Information recording method and electronic equipment

Abstract

The invention provides an information recording method and electronic equipment, and relates to the technical field of terminals. The method comprises the steps that first memory information is obtained, the first memory information is extracted according to display content of a first interface and matched with a preset memory type, and the memory type corresponds to at least one attribute item; and displaying the first memory information on the intelligent assistant. According to the scheme, the user does not need to memorize a large amount of information by himself / herself and can acquire the required information through the intelligent assistant at any time according to requirements, so that the operation complexity of information acquisition can be reduced, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and more specifically, to an information recording method and an electronic device. Background Technology

[0002] As the functionalities of terminal applications continue to increase, the information they carry is also becoming richer, such as train ticket information, hotel booking information, movie ticket booking information, and related card information. This information is often used multiple times in other scenarios or applications. In order to reuse this information, users need to record it in advance, which is a significant burden for users and often results in incomplete records. Alternatively, users can search for this information in the application where it originated when they need it again; however, this operation is very inconvenient, and sometimes the desired information cannot be found. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides an information recording method and an electronic device that can reduce the operational complexity for users to repeatedly use information in terminal applications and improve user experience.

[0004] Firstly, an information recording method is provided, which can be executed by an electronic device, or by a component of the electronic device (such as a processor, chip, or chip system), or by a logic module or software that can implement all or part of the functions of the electronic device.

[0005] The method includes: acquiring first memory information, which is extracted based on the content displayed on a first interface, and the first memory information matches a preset memory type, which corresponds to at least one attribute item; and displaying the first memory information on a smart assistant.

[0006] According to the information recording method provided in this application, electronic devices can record information based on the content displayed on the terminal and retrieve the remembered information through a smart assistant. This method eliminates the need for users to manually record large amounts of information and allows them to retrieve the required information at any time via a smart assistant, thereby reducing the operational complexity of reusing information from terminal applications and improving the user experience.

[0007] In one possible implementation, before displaying the first memory information on the smart assistant, the method further includes: obtaining a first user operation, which is used to instruct the saving of the first memory information, or the first user operation is used to instruct the display content.

[0008] Based on this scheme, the electronic device can save the first memory information based on the first user's operation. Alternatively, the electronic device can determine the display content based on the first user's operation and retrieve the first memory information based on the display content.

[0009] In one possible implementation, the first user operation is used to instruct the saving of first memory information; wherein, before acquiring the first user operation, the method further includes: displaying a prompt message, the prompt message being used to indicate that there is currently memory information to be saved.

[0010] Based on this scheme, the electronic device can display a prompt after acquiring the initial memory information. The user can then perform a first user operation based on the prompt, and the electronic device can save the initial memory information based on this operation. This method can actively acquire the initial memory information without user intervention.

[0011] In one possible implementation, the first user operation is used to instruct the display content, which is one of the following: the interface content of the first interface of the first application, the display content in the first area of ​​the first interface, the first text in the first interface, and the first image in the first interface.

[0012] In one possible implementation, the display of the prompt message includes: if the display interface is not switched out from the first interface, the prompt message is displayed in the dialog window of the smart assistant, and the dialog window is floating on the first interface; wherein, the display of the first memory information on the smart assistant includes: in response to the first user operation on the dialog window, the first memory information is displayed in the dialog window.

[0013] For example, the first user action can be a voice or text command entered by the user in the dialog window, or an action such as clicking on the prompt information.

[0014] This solution can support users in retrieving memory information in both voice-interaction and non-voice-interaction scenarios, and can also guide users to save memory information by prompting them with information when the user has not switched the display interface.

[0015] In one possible implementation, the display of the prompt information includes: if the display interface switches from the first interface to the second interface, displaying the prompt information on the second interface; wherein, the display of the first memory information on the smart assistant includes: in response to the first user operation on the prompt information or in response to the first user operation on the application corresponding to the prompt information, displaying the first memory information on a third interface of the smart assistant, the third interface including the memory information to be saved.

[0016] For example, the prompt message could be a push notification.

[0017] This solution supports guiding users to save memory information via push notifications even after the user has switched display interfaces. This allows users to be aware in real-time that the electronic device has retrieved memory information and to save that information whenever needed.

[0018] In one possible implementation, the third interface includes a confirmation control corresponding to the first memory information, and the method further includes: in response to an operation on the confirmation control, saving the first memory information on the smart assistant.

[0019] In one possible implementation, obtaining the first memory information includes: scanning the display interface of the first application; and obtaining the first memory information based on the currently scanned first interface, wherein the first interface is a display interface of the first application.

[0020] In one possible implementation, the first application is a target application, wherein the electronic device is able to acquire memory information based on a scan of the display interface of the target application.

[0021] In one possible implementation, the displayed content is the interface content of the first interface of the first application; wherein, obtaining the first memory information includes: in response to the first user operation on the dialogue window of the smart assistant, obtaining the first memory information according to the interface content of the first interface, the dialogue window being displayed floating on the first interface, and the first interface being a display interface of the first application.

[0022] Based on this solution, users can retrieve memory information by waking up a smart assistant when they have a memory need.

[0023] In one possible implementation, the displayed content is the content of a first area of ​​the first interface; wherein, before acquiring the first memory information, the method further includes: determining the first area in response to the first user operation on the first interface.

[0024] In one possible implementation, before determining the first region in response to the first user operation on the first interface, the method further includes: in response to an operation on the fourth interface, performing intelligent screen recognition on the fourth interface to obtain the first interface, the first interface including a first control; and in response to an operation on the first control, activating a selection function.

[0025] Based on this scheme, users can indicate the content to be displayed by circling or drawing lines, and the electronic device can obtain the memory information based on the content indicated by the user.

[0026] In one possible implementation, the first interface further includes a second control; wherein, acquiring the first memory information includes: in response to an operation on the second control, waking up the smart assistant and displaying an image corresponding to the first area in the smart assistant's dialog box; and acquiring the first memory information based on the content in the image.

[0027] In one possible implementation, the displayed content is the first text; wherein, before acquiring the first memory information, the method further includes: copying the first text in response to the first user operation on the first interface.

[0028] Based on this scheme, electronic devices can obtain memory information from copied text.

[0029] In one possible implementation, obtaining the first memory information includes: in response to an operation on a third control in a fifth interface of the smart assistant, displaying a first window on the fifth interface, the first window including a fourth control, the fifth interface displaying saved memory information, the third control being used to add new memory information; in response to an operation on the fourth control, displaying the first text in the first window; and extracting the first text from the first window to obtain the first memory information.

[0030] This solution allows users to access memory information whenever they need it by setting up an entry point for adding new memory information.

[0031] In one possible implementation, the displayed content is the first image; wherein, before acquiring the first memory information, the method further includes: displaying the first interface, which includes the first image, in response to the first user operation on a thumbnail of the first image in the sixth interface.

[0032] Based on this scheme, electronic devices can obtain memory information from images.

[0033] In one possible implementation, acquiring the first memory information includes: performing text recognition on the first image to obtain recognized text; displaying a seventh interface of the smart assistant in response to an operation on a fifth control in the first interface, the seventh interface of the smart assistant displaying a second window, the second window including the recognized text; and acquiring the first memory information based on the recognized text in response to an operation on a sixth control in the second window.

[0034] In one possible implementation, acquiring the first memory information includes: in response to the first user operation on the seventh control in the first interface, acquiring the first memory information based on the first image, wherein the seventh control is located in a third window of the first interface and the first image is displayed in the third window.

[0035] In one possible implementation, prior to the first user operation in response to a thumbnail of the first image in the second interface, the method further includes: in response to an operation on an eighth control in an eighth interface of the smart assistant, displaying a fourth window on the eighth interface, the fourth window including a ninth control, the eighth interface displaying saved memory information, the eighth control being used to add new memory information; in response to an operation on the ninth control, displaying a first option and a second option in the fourth window, the first option being used to select an image from the gallery, the second option being used to take a photo; and in response to an operation on the first option, displaying a sixth interface, the sixth interface including a thumbnail of at least one image from the gallery.

[0036] In one possible implementation, before displaying the first memory information on the smart assistant, the method further includes: displaying on the smart assistant a memory type that matches the display content of the first interface and at least one attribute item corresponding to the memory type.

[0037] In one possible implementation, some or all of the attribute items in the at least one attribute item are editable.

[0038] Based on this solution, users can edit attribute items according to their needs.

[0039] In one possible implementation, the smart assistant includes a tenth control, and the method further includes: displaying a first menu bar in response to an operation on the tenth control, the first menu bar including a plurality of memory type options; and displaying on the smart assistant, in response to an operation on a first memory type option among the plurality of memory type options, the first memory type matching the display content of the first interface and at least one attribute item corresponding to the first memory type.

[0040] Based on this solution, users can select the memory type according to their needs to obtain the information they need to remember.

[0041] In a second aspect, embodiments of this application provide an electronic device comprising: one or more processors and a memory; the memory being coupled to the one or more processors; the memory being used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform the method shown in the first aspect above.

[0042] Thirdly, embodiments of this application provide a chip applied to an electronic device. The chip includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the methods shown in the first aspect above.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method shown in the first aspect above.

[0044] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when run by an electronic device, causes the electronic device to implement the method shown in the first aspect above.

[0045] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0046] Figures 1 to 12 The user interface diagram for obtaining memory information provided in this application embodiment is shown.

[0047] Figure 13 This is an exemplary flowchart of an information recording method provided in an embodiment of this application;

[0048] Figures 14 to 17 This is a schematic diagram of the interface for displaying memory types and corresponding attribute items provided in an embodiment of this application;

[0049] Figure 18 This is an exemplary flowchart of an information recording method provided in an embodiment of this application;

[0050] Figure 19 This is a schematic diagram of a system architecture provided in an embodiment of this application;

[0051] Figure 20 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0052] Figure 21 This is a schematic diagram of the chip structure provided in the embodiments of this application. Detailed Implementation

[0053] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0054] This application relates to electronic devices equipped with a display screen. For example, the electronic device may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), artificial intelligence (AI) device, wearable device, in-vehicle device (such as a car infotainment system), smart home device, or smart city device, etc. The embodiments of this application do not impose any special limitations on the specific type of the electronic device.

[0055] One technology allows users to save screen information, such as train ticket information, hotel booking information, movie ticket booking information, and relevant card information, as images through screenshots. However, this method does not help users understand and remember the content of the images. Furthermore, users need to manually browse through numerous images to find the information they need, which is a cumbersome process.

[0056] To address the current problem of inconvenience for users in obtaining and searching for information, this application provides an information recording method. Electronic devices can actively or based on user triggers acquire information suitable for memorization, such as images, text, and application interfaces, and display and save the acquired information on a smart assistant, thereby reducing the operational complexity of information acquisition and improving user experience.

[0057] In this application, the intelligent assistant may also be referred to as a voice assistant, smart assistant, artificial intelligence (AI) assistant, intelligent voice, digital assistant, virtual assistant, etc. It can be understood as an information processing system that can recognize natural language input in the form of speech and / or text to infer the user's intent and perform corresponding actions based on the inferred user intent. Furthermore, the system can respond to the user's input through audible (e.g., voice) or visual (e.g., text, cards) output formats.

[0058] In this embodiment, the memory information can be displayed and stored on the smart assistant in the form of a card. It should be understood that this application does not limit the form in which the memory information is displayed and stored; for example, the memory information can also be displayed and stored on the smart assistant in the form of text or images.

[0059] For example, the stored information can be address information, card information, or order information. Order information could include train ticket orders, movie ticket orders, hotel orders, attraction ticket orders, etc. Card information could include ID card information, passport information, business card information, etc. It should be understood that this application does not limit the specific type of stored information.

[0060] The following illustrations, with reference to the accompanying diagrams, use the smart assistant "Xiaoyi" as an example to illustrate the interactions between the user and the electronic device involved in acquiring memory information in different scenarios. It should be understood that "Xiaoyi" is an application (APP), and it is only one specific example of a smart assistant; a smart assistant can also be other APPs or mini-programs, etc.

[0061] Scenario 1: Obtain memory information based on the application's display interface.

[0062] Figure 1 This is a schematic diagram of a user interface for actively acquiring memory information, provided in an embodiment of this application. See also... Figure 1 In (a) of the diagram, interface 10 is the ticket order information interface, which is the current display interface of a certain application on the electronic device. The electronic device actively retrieves the memory information from interface 10. If the user does not switch the display interface, the electronic device will wake up (or launch or invoke) Xiaoyi after retrieving the memory information. See also Figure 1 In (b), after Xiaoyi is awakened, its window 11 floats on the interface 10. Window 11 displays a prompt message 12, which informs the user that the electronic device has acquired the memory information. If the user wishes to save the memory information acquired from the interface 10, in one way, the user can input a command in window 11 that is the same as or substantially the same as the prompt message via voice or text. For example, the user can issue a voice command based on prompt message 12 such as "Remember the screen information," "Remember the screen information," "Record the screen information," or "Remember the ticket information on the picture." In another way, the user can click on prompt message 12; in response to the user's click, the electronic device simulates the user issuing the command "Remember the screen information." See also Figure 1In step (c), the user inputs the command "Remember the screen information" via voice or text input or clicks on prompt 12. The electronic device executes the "Remember the screen information" command and displays the execution result, namely, "Okay, I've remembered it for you" and card 13. The content of card 13 is the remembered information obtained from the content displayed on interface 10. For example, the electronic device can also verbally announce "Okay, I've remembered it for you" and / or the content of card 13. It should be noted that, similarly, for any of the figures or embodiments described below, the content in the dialog box and / or the content in the card displayed in Xiaoyi can also be verbally announced, which will not be elaborated on further below.

[0063] In one example, card 13 could also be Figure 1 (d) in the text. See also Figure 1 In (d), card 13 may also include image 14, which is a partial screenshot of interface 10, indicating that the time and location information in card 13 comes from interface 10.

[0064] Figure 2 This is a schematic diagram of another user interface for actively acquiring memory information provided in an embodiment of this application. See also... Figure 2 In (a) of the diagram, interface 20 is the movie ticket order information interface, and it is the current display interface of the electronic device. The electronic device actively retrieves the memory information from interface 20. If the user does not switch interfaces, the electronic device will wake up Xiaoyi after retrieving the memory information. See also... Figure 2 In (b), after Xiaoyi is awakened, her window 21 floats on the interface 20. Window 21 can display a prompt message 22, which is related to... Figure 1 The prompt message 12 is the same. For example, if the user clicks on prompt message 22, in response to the user's click on prompt message 22, the electronic device simulates the user issuing and executing the "remember the screen information" command, and displays "Okay, I've remembered it for you" and card 23. The content of card 23 is the memorized information obtained from the content displayed on interface 20.

[0065] In one example, card 23 could also be Figure 2 (d) in the text. See also Figure 2 In (d), card 23 may also include image 24, which is the movie ticket collection QR code in interface 20.

[0066] Figure 3 This is a schematic diagram of another user interface for actively acquiring memory information provided in an embodiment of this application. See also... Figure 3In (a), interface 30 is the current display interface of the electronic device. The electronic device actively retrieves the memory information in interface 30. If the user switches the electronic device's interface from interface 30 to interface 30 before retrieving the memory information from interface 30, the electronic device may fail to retrieve the memory information from interface 30. Figure 3 Interface 31 shown in (b) is referenced. Figure 3 In (c), the electronic device displays a pushed notification 32 on interface 31. Notification 32 is used to prompt the user for currently pending confirmation of memory information. See also Figure 3 In section (d), the user can access interface 33 by clicking notification 32 or through the relevant path of the application corresponding to notification 32. Interface 33 is the interface for the memory information to be confirmed. Interface 33 includes the memory information to be confirmed, i.e., card 34, and controls 35 and 36 for the user to save and delete the memory information to be confirmed. For example, if the user clicks control 35, the electronic device will save card 34 in response to the user's click operation. Or, if the user clicks control 36, the electronic device will delete card 34 in response to the user's click operation. It should be understood that interfaces 30 and 34 are not directly related to the memory information. Figure 1 The interface shown is the same as 10, and cards 34 and... Figure 1 The card shown is the same as card 13.

[0067] It should be noted that interface 33 may include one or more pieces of memory information awaiting confirmation. For example, after the electronic device displays notification 32, the user does not enter interface 33, but instead browses... Figure 2 The interface 20 shown. Furthermore, the electronic device acquires the memory information from interface 20, and before the electronic device acquires the memory information, the user switches the electronic device's interface from interface 20 to another interface, such as... Figure 3 Interface 31 is shown in (b) above. Therefore, there will be two pieces of memory information to be confirmed, and interface 33 will include the card 23 to be confirmed (e.g., ...). Figure 2 (c) or Figure 2 (as shown in (d)) and the card 34 to be confirmed. Similarly, the user can save or delete card 23 using the "Save" and "Delete" controls corresponding to card 23.

[0068] Figure 4 This is a schematic diagram of a memory information management interface provided in an embodiment of this application. See also... Figure 4 Interface 40 is a memory information management interface. Interface 40 may include at least one card, the content of which is the memory information acquired by the electronic device. For example, the at least one card may be card 41 and card 42. Card 41 may be... Figure 1 Card 13 is shown. Figure 1 In the example shown, when the electronic device displays card 13 in window 11, interface 40 can save card 13. Alternatively, card 41 can be... Figure 3Card 34 is shown. Figure 3 In the example shown, after the user clicks control 35, card 34 can be saved in interface 40. Card 42 can be... Figure 2 Card 23 is shown. Figure 2 In the example shown, when the electronic device displays card 23 in window 21, interface 40 can save card 23.

[0069] Figure 4 The "Important Dates," "Frequently Used Addresses," and "Cards" labels shown are memory type tags. In one example, users can swipe left and right on the memory type tags to view memory information belonging to different memory types. In one example, cards 41 and 42 could be memory information for the "Important Dates" type; however, it should be understood that cards 41 and 42 could also be memory information for other memory types (such as orders). Memory types will be explained in detail below and will not be discussed here.

[0070] It should be understood that Figure 4 The following explanation uses only the current storage of cards 41 and 42 on interface 40 as an example. It should be understood that if the electronic device also stores other memory information, it will also be stored on interface 40. Users can view all cards by swiping up and down on the screen.

[0071] In one example, interface 40 and Figure 3 Interface 33 shown in (d) belongs to different pages in Xiaoyi, and the two can be switched between each other.

[0072] Figure 5 This is a schematic diagram of a user interface that retrieves memory information based on user triggering (i.e., passively) according to the display content of the application's display interface, as provided in an embodiment of this application. See also... Figure 5 In (a) of the diagram, interface 50 is the movie ticket order information interface, and interface 50 is the current display interface of the electronic device. When the user wants to access the stored information in interface 50, they can wake up Xiaoyi. For example, the user can wake up Xiaoyi by voice inputting "Xiaoyi, Xiaoyi, remember the screen information." See also Figure 5 In (b), after Xiaoyi is awakened by the user, its window 51 floats on interface 50. Xiaoyi executes the user's command to "remember the screen information," and after obtaining the remembered information from interface 50, displays "Okay, I've remembered it for you" and card 52. The content of card 52 is the remembered information obtained based on the content displayed on interface 50. It should be understood that interface 50 and... Figure 2 The interface shown is the same as 20, and card 52 and Figure 2 The card shown is the same as card 23.

[0073] In one example Figure 4The card 42 shown can be card 52. When the electronic device displays card 52 in window 51, Figure 4 The interface 40 shown can save card 52.

[0074] Figure 6 This application provides a schematic diagram of a user interface for an electronic device that retrieves memory information based on user triggering and the content displayed on the application's screen. See also: Figure 6 In (a), interface 60 is the chat interface, which is the current display screen of the electronic device. When a user wants to access the stored information in interface 60, they can wake up Xiaoyi. For example, a user can wake up Xiaoyi by voice inputting "Xiaoyi, Xiaoyi, remember the screen information." See also Figure 6 In (b), after Xiaoyi is awakened, its window 61 floats on the interface 60. Xiaoyi executes the user's command to "remember the screen information", and after obtaining the memory information in the interface 60, it displays "Okay, I've remembered it for you", card 62 and card 63. The contents of card 62 and card 63 are the memory information obtained by the electronic device from the interface 60.

[0075] In one example, cards 62 and 63 can also be saved in Figure 4 On the interface 40 shown.

[0076] As another example, if an electronic device acquires multiple pieces of memory information, it can also support users in selectively saving those memories.

[0077] For example, see Figure 6 In (b) of this example, Xiaoyi can ask the user, "Multiple pieces of memory information have been identified. Which ones do you want to remember?" and display "Remember All" and "Remember Selected" controls. If the user wishes to save all memory information, they can click the "Remember All" control. See also Figure 6 In step (c), in response to the user clicking the "Record All" control, the electronic device can display "Okay, all recorded" and re-display cards 62 and 63 to indicate that cards 62 and 63 have been saved. Further, at this time... Figure 4 The interface 40 shown can display cards 62 and 63. If the user only wants to save one of the memory entries, they can click on the desired entry, such as card 62, and then click the "Remember Selected" control. In response to the user's click on the "Remember Selected" control, the electronic device can display "Okay, selected has been remembered" and re-display card 62, indicating that card 62 has been saved. Further, at this time... Figure 4Card 62 can be displayed on the interface 40 shown. Furthermore, the user can also instruct the electronic device to save card 62 and / or card 63 via voice or text. For example, if the user voice-inputs "Remember all," the electronic device will save card 62 and card 63. Similarly, if the user voice-inputs "Remember the first piece of information," the electronic device will save card 62.

[0078] In some embodiments, if Xiaoyi is only woken up on the current display screen without being commanded to retrieve the memory information of the current display screen, Xiaoyi can automatically retrieve the memory information from the current display screen and display a prompt message on the Xiaoyi window after retrieving the memory information, informing the user that the electronic device has retrieved the memory information. For example, the prompt message could be... Figure 1 The prompt message 12 in the text. Afterwards, the interaction between the user and the electronic device can refer to the... Figure 1 or Figure 2 The description of execution.

[0079] Scenario 2: Obtain memory information based on the selected image.

[0080] Figure 7 This is a schematic diagram of a user interface for retrieving memory information based on a user's selection operation, as provided in this application. See also... Figure 7 In (a), interface 70 is the current display interface of the electronic device. Interface 70 can be a picture in the gallery or an application interface. The user can enter Smart Screen Recognition by pressing the screen with two fingers or by other triggering methods. By performing Smart Screen Recognition on interface 70, interface 70 becomes... Figure 7 Interface 71 is shown in (b) of the diagram. See also... Figure 7 In (b) of the example, the user clicks the "image selection" control 72 and then selects content on the screen. For example, see... Figure 7 In step (c), based on the user's selection operation, the electronic device draws corresponding lines 73, and the area enclosed by lines 73 is the selected area. Based on the content in the selected area, the electronic device can obtain the memory information.

[0081] For example, in one instance, when a user clicks the "Memory Generation" control 74, the electronic device wakes up Xiaoyi in response to the user's click. See also Figure 7In step (d), after the electronic device wakes up Xiaoyi, it displays Xiaoyi's window 75. For example, window 75 can be displayed floating on interface 71. Window 75 displays the user's dialog box 76, and the electronic device simulates the user issuing the command "Remember the content in the picture." The picture 77 in dialog box 76 is the image corresponding to the circled area in interface 71. The electronic device executes the command "Remember the content in the picture," retrieves the memory information based on picture 77, and then displays the execution result "Okay, I've remembered it for you" and card 78. The content of card 78 is the memory information retrieved based on picture 77.

[0082] In another example, when a user clicks the "Memory Generation" control 74, the electronic device, in response to the user's click, can retrieve memory information based on the content in the selected area. After retrieving the memory information, the electronic device can prompt the user on interface 71 that there is currently memory information awaiting confirmation, for example, through a push notification. The user can then save the memory information based on this notification. For instance, this notification might be similar to... Figure 3 Notification 32 shown in (c) allows users to... Figure 3 The memory information is stored in the interface shown in (d).

[0083] Scenario 3: Obtaining memory information from text.

[0084] Figure 8 This is a schematic diagram of an interface for adding memory information via text, as provided in this application. See also... Figure 8 In (b), interface 81 is Figure 4 Based on the interface 40 shown, a control 84 for adding memory information is added. Cards 82 and 83 in interface 81 can be cards 41 and 42 in interface 41, respectively.

[0085] See Figure 8 In (a), the user copied the information "I need to buy books for my daughter tomorrow" from interface 80 to the clipboard. See also Figure 8 In (b), the user enters interface 81 and clicks control 84 on interface 81. See [link to example] for the response to the user clicking control 84. Figure 8 In section (c), window 85 is displayed in interface 81. Window 85 is the smart memory retrieval window. The user can click on control 86 in window 85 to trigger the electronic device to automatically paste the text "I need to buy books for my daughter tomorrow" that the user copied to the clipboard.

[0086] In one example, a user clicking control 86 could also trigger the electronic device to retrieve memory information based on "I need to buy a book for my daughter tomorrow." For example, in response to a user clicking control 86, see... Figure 8In step (d), window 85 displays dialog box 87 and card 88. Dialog box 87 contains the text "Buy books for my daughter tomorrow" copied to the clipboard by the user, and card 88 is the memory information retrieved based on the text "Buy books for my daughter tomorrow." If the user wishes to save this memory information, they click control 89 in window 85. In response to the user clicking control 89, the electronic device saves card 88. For example, in response to the user clicking control 89, window 85 is no longer displayed, and interface 80 changes to... Figure 10 The interface 100 shown is where the electronic device saves the card 88 in the memory information management interface. Optionally, the interface 100 may also include a prompt box 101, which displays the message "New memory information created successfully" to remind the user that the memory information has been successfully added.

[0087] In another example, after an electronic device automatically pastes the text "I need to buy books for my daughter tomorrow," it can retrieve the memory information based on "I need to buy books for my daughter tomorrow" based on a user trigger. For example, in response to a user clicking control 86 in window 85, see [reference needed]. Figure 9 In (a) of the diagram, window 85 displays a dialog box 87 and a control 90 for content recognition. After the user clicks the control 90, the electronic device retrieves the stored information based on the content of the dialog box 87 and displays, as shown in (a). Figure 9 The prompt message 91 is shown in (b) above. In one design, after the electronic device acquires the memory information, it no longer displays the prompt message 91, but instead displays the card 88 in window 85. That is, after the electronic device acquires the memory information, the content displayed in window 85 is as follows: Figure 8 As shown in (d) above. Similar to the previous example, if the user wishes to save the memory information, they click control 89. In response to the user's click on control 89, the electronic device will save card 88. In another design, after the electronic device obtains the memory information, it no longer displays window 85, but instead displays... Figure 10 The interface shown is 100.

[0088] Scenario 4: Obtaining memory information based on images.

[0089] Figure 11 This is a schematic diagram of an interface for adding memory information through text in an image, as provided in this application. See also... Figure 11 (a) in the text, interface 81, refer to the above text. Figure 8 The description in (b) is as follows. When the user clicks control 84, in response to the user's click on control 84, see [link to relevant documentation]. Figure 11 In (b) of the interface, window 110 is displayed in interface 81. Window 110 is the intelligent memory retrieval window. Compared to Figure 8Window 85 (c) shown in the diagram, and window 110 also include a "Voice Recognition" control and an "Image Recognition" control. After the user clicks the "Image Recognition" control, window 110 will display an option box 111, which includes two options: "Gallery Selection" and "Take Photo". The "Gallery Selection" option allows the user to select images from the gallery; the "Take Photo" option allows the user to take a picture. For example, if the user clicks the "Gallery Selection" option, the electronic device will display... Figure 11 As shown in (c) of the diagram, interface 112 allows users to select an image for information retrieval. For example, after selecting image 113 from interface 112, the electronic device displays the image as shown in the diagram. Figure 11 The interface 114 is shown in (d) above, and the electronic device recognizes the image in the interface 114 to obtain information from the image, which is the content in the recognition box 115. In response to a user clicking the control 116 in the interface 114, see [reference needed]. Figure 11 In (e), the electronic device will obtain Figure 11 The information in the image shown in (d) is "Shop transfer contact person: Ms. Zhang 130XXX". Window 110 also includes a control 117, which the user can click. In response to the user's click on control 117, the electronic device retrieves the stored information based on "Shop transfer contact person: Ms. Zhang 130XXX" and obtains the following... Figure 11 Card 118 is shown in (f) in the diagram.

[0090] Figure 12 This is a schematic diagram of an interface for adding memory information through images, as provided in this application. Figure 12 (a) to Figure 12 Figure (c) in the middle and Figure 11 (a) to Figure 11 The same as Figure (c) above; please refer to the text above. Figure 11 The relevant description is as follows: After the user selects image 113 in interface 112, the display jumps to interface 81, and window 110 in interface 81 displays image 113. In response to the user clicking control 117 in window 110, see [link to relevant documentation]. Figure 12 In (e), the electronic device will obtain the memory information based on image 113 to obtain, as shown in (e). Figure 12 Card 118 is shown in (e) of the diagram.

[0091] It can be seen that, with Figure 11 The example shown is different in that... Figure 12 In the example shown, after the user clicks on the thumbnail of image 113, the electronic device does not display the image as shown. Figure 11The interface 114 shown in (d) does not recognize the image 113, but directly displays the image 113 in the window 110 and obtains the memory information based on the image 113.

[0092] It should be understood that the various operations described in the embodiments of this application, such as clicking or swiping, are merely possible operation examples to achieve the corresponding functions, and these operation examples should not constitute any limitation on this application. The names, patterns, positions, etc. of the controls in the user interface diagrams shown in the embodiments of this application, as well as the arrangement of various contents in the user interface diagrams, are all examples, and these examples should not constitute any limitation on this application.

[0093] The information recording method provided in this application will be explained below in conjunction with the user interface described above.

[0094] Figure 13 This is an exemplary flowchart of an information recording method provided in this application. The method 1300 can be applied to the electronic device described above; for example, the method can be executed by an intelligent agent in the electronic device. The method 1300 may include steps S1310 and S1320. The steps of the method 1300 are described below.

[0095] S1310, acquire first memory information.

[0096] The first memory information is extracted from the content displayed on the first interface and matches a preset memory type, which corresponds to at least one attribute item. For example, the content displayed on the first interface can be all the content of the first interface, or it can be part of the content of the first interface, such as text or images in the first interface.

[0097] In the embodiments of this application, at least one memory type (also referred to as category, memory type, or identification type, etc.) can be preset. For example, the memory type can be an important date, address, card, order, or event (or matter).

[0098] A memory type can include at least one attribute item. For example, the attributes for an important day can include: name, related person, time, etc. The attributes for an event can include: time, location, name, subject, etc.

[0099] For example, some memory types can be further divided into multiple subtypes, and the attribute items corresponding to different subtypes may differ. For instance, an important date may include one or more of the following subtypes: birthday, anniversary, payment due date, exam date, payday. For instance, an order may include one or more of the following subtypes: train ticket order, plane ticket order, or movie ticket order. For instance, a card may include one or more of the following subtypes: business card, ID card, or driver's license.

[0100] For example, the attributes of a memory type with the subtype "train ticket" could include: train number, departure station, destination station, time, gate, seat number, etc. See also... Figure 1 The card 13 shown contains a memory information, and the memory type corresponding to this memory information is an order, with the subtype being a train ticket.

[0101] For example, the attribute items for the memory type of driver's license subtype can include: name, number, validity period, etc.

[0102] For example, if the first interface is a map-related application, the corresponding memory type could be an event or an address. The first memory information could include one or more of the following: time, address, name, card number, or event. The event could be, for example, taking a taxi or traveling. Similarly, if the first interface is a music-related application, the corresponding memory type could be an important day or event. The first memory information could include one or more of the following: song title, artist, or song release date.

[0103] It should be understood that the memory types and corresponding attribute items shown in the embodiments of this application are merely illustrative and should not constitute any limitation on this application.

[0104] In one implementation, the content displayed on the first interface can be matched with some or all of the memory types in at least one preset memory type; if the content displayed on the first interface is successfully matched with a certain memory type, the matching information is used as the first memory information; or, if the content displayed on the first interface is successfully matched with multiple memory types, the matching information of the memory type with the most matching information can be used as the first memory information.

[0105] In another implementation, different interfaces or different content can correspond to a memory type by default. If the content displayed on the first interface matches its corresponding default memory type, the matching information can be used as the first memory information.

[0106] In another implementation, users can choose a memory type themselves, and the content displayed on the first interface is matched with the selected memory type. If the match is successful, the matched information can be used as the first memory information.

[0107] It should be understood that the matching information refers to the information of the corresponding attribute items obtained through matching.

[0108] For example, the aforementioned matching operation can be performed using a lager language model (LLM).

[0109] In some embodiments, the electronic device can perform corresponding processing based on the displayed content of the first interface to obtain first memory information. For example, the electronic device can match the displayed content of the first interface with at least one preset memory type to obtain first memory information.

[0110] In other embodiments, the electronic device may send the displayed content of the first interface or a preprocessing result obtained by preprocessing the displayed content of the first interface to the server. The server can then perform corresponding processing based on the displayed content of the first interface or the preprocessing result to obtain the first memory information. In this case, S1310 may specifically include receiving the first memory information from the server. For example, the server may match the displayed content of the first interface with at least one preset memory type to obtain the first memory information. For example, the preprocessing may be visual analysis, interface framework analysis, and / or interface text recognition, and the corresponding preprocessing result may be an application page, text, and / or image data with descriptions. The application page may be, for example, an application name or / or a page name.

[0111] S1320, the first memory information is displayed on the smart assistant.

[0112] For example, a first card could be displayed on the smart assistant, and this first card could include first memory information. Alternatively, the first memory information could also be displayed in other forms, such as as an image.

[0113] According to the information recording method provided in this application, an electronic device can acquire initial memory information and display it on a smart assistant. This method eliminates the need for users to memorize large amounts of information themselves, allowing them to retrieve the required information at any time through the smart assistant, thereby reducing the operational complexity of information acquisition and improving the user experience.

[0114] In some embodiments, prior to S1320, the method may further include: obtaining a first user action.

[0115] In one scenario, the first user operation instructs the storage of first memory information. In this scenario, the electronic device, in response to the first user operation, may execute S1320.

[0116] In one example, before obtaining the first user action, the method 1300 may further include: displaying a prompt message. This prompt message indicates that there is currently memory information to be saved.

[0117] Specifically, after the electronic device acquires the first memory information, it can display a prompt message. Based on the prompt message, the user can perform a first user operation. In response to the first user operation, the electronic device can display the first memory information on the smart assistant.

[0118] Based on this scheme, electronic devices can actively acquire first memory information and prompt the user to save the memory information after acquiring the first memory information. The user can save the first memory information based on the prompts from the electronic device.

[0119] In another scenario, the first user operation is used to instruct the displayed content. In this case, the electronic device, in response to the first user operation, can execute S1310.

[0120] Based on this scheme, electronic devices can retrieve initial memory information based on user triggers.

[0121] For example, the displayed content is one of the following: the interface content of the first interface of the first application, the displayed content in the first area of ​​the first interface, the first text in the first interface, and the first image in the first interface.

[0122] The following section provides a detailed explanation of two scenarios: electronic devices actively acquiring first memory information and passively (i.e., based on user triggering) acquiring first memory information.

[0123] Actively acquire first memory information

[0124] In some embodiments, S1310 may specifically include: scanning the display interface of the first application; obtaining first memory information based on the display content of the currently scanned first interface, wherein the first interface is a display interface of the first application.

[0125] In other words, electronic devices can actively acquire memory information based on the application's display interface. For example, an electronic device can scan the display interface of a first application at a preset period, or scan the display interface of the first application when it switches. It should be understood that for the same application, such as the first application, the interface scanned by the electronic device at different times may be different. Memory information may be acquired from some interfaces, but not from others. Failure to acquire memory information means that the interface does not contain information matching the relevant memory type.

[0126] Based on the above solution, electronic devices can proactively acquire first-memory information without user intervention, eliminating the need for manual user intervention.

[0127] For example, for the first interface currently being scanned, visual analysis, interface framework analysis, and / or interface text recognition can be combined to extract the application page, text, and descriptive image data of the first interface. Then, the first memory information can be obtained based on the application page, text, and descriptive image data of the first interface. For instance, the memory type adapted to the first application can be determined based on the application page, text, and descriptive image data of the first interface. Then, the corresponding schema can be called to generate a prompt for obtaining memory information. The first memory information can be generated by calling the LLM through the prompt.

[0128] Optionally, the first application can be a target application. The target application can also be referred to as a memory source. The electronic device is able to acquire memory information based on scanning the display interface of the target application.

[0129] For example, if an electronic device repeatedly retrieves memory information from an application's display interface based on user triggers, the device can designate that application as a target application, or the user can mark it as a target application. For instance, if the electronic device retrieves memory information from an application's display interface more than a preset number (denoted as N times), it can display a pop-up message on the current screen prompting the user whether to mark the application as a target application. The user can then mark the application as a target application based on this prompt.

[0130] In one example, S1320 may specifically include: if the display interface has not switched from the first interface, waking up the smart assistant and displaying the first memory information in the smart assistant's dialogue window.

[0131] In other words, the electronic device obtains the first memory information based on the first interface. If the electronic device's display interface is still on the first interface when the first memory information is obtained, the electronic device can actively wake up the smart assistant and display the first memory information obtained based on the first interface on the smart assistant.

[0132] For example, the smart assistant could be Xiaoyi, and the first interface could be... Figure 1 The interface 10 shown can contain the first memory information. Figure 1 The contents of card 13 are shown. If the user remains on interface 10, the electronic device, after obtaining the memory information based on interface 10, will wake up Xiaoyi, display card 13 in window 11, and save card 13 to Xiaoyi, for example, by saving it to... Figure 4 The interface shown is 40.

[0133] In another example, the display prompt message may specifically include: if the display interface is not switched from the first interface, the prompt message is displayed in the dialogue window of the smart assistant, and the dialogue window of the smart assistant is displayed floating on the first interface.

[0134] Accordingly, S1320 may specifically include: displaying first memory information in the dialogue window of the smart assistant in response to a first user operation on the dialogue window of the smart assistant. That is, the electronic device obtains the first memory information based on the first interface. If the display interface of the electronic device is still on the first interface when the first memory information is obtained, the electronic device can actively wake up the smart assistant and display the first prompt information in the dialogue window of the smart assistant. Then, the user can perform a first user operation on the dialogue window of the smart assistant, and in response to the first user operation, the electronic device can display the first memory information. For example, the user can input a first user command in the dialogue window of the smart assistant. The first user command can be input via voice or text, and the electronic device executes the first user command and displays the first memory information. Alternatively, the user can operate on the prompt information, and the electronic device executes the command corresponding to the operation and displays the first memory information.

[0135] The above solution can support users to obtain memory information in both voice interaction and non-voice interaction scenarios, and can also guide users to save memory information by prompting them with information when the user has not switched the display interface.

[0136] For example, the smart assistant could be Xiaoyi, and the first interface could be... Figure 1 The interface 10 shown can contain the first memory information. Figure 1 The content of card 13 is shown. If the user remains on interface 10, the electronic device will retrieve the memory information based on the content on interface 10, wake up Xiaoyi, and display prompt message 12 (an example of a prompt message). The user can enter the first user command "Remember the screen information" or click on prompt message 12. The electronic device will then display card 13 and save card 13 to Xiaoyi, for example, by saving it to... Figure 4 The interface shown is 40.

[0137] For example, the smart assistant could be Xiaoyi, and the first interface could be... Figure 2 The interface 20 shown can contain the first memory information. Figure 2 Card 23 is shown. If the user remains on interface 20, the electronic device will retrieve the memory information based on the content on interface 20, wake up Xiaoyi, and display prompt message 22 (an example of a prompt message). The user can enter the first user command "Remember the screen information" or click on prompt message 22. The electronic device will then display card 23 and save card 23 to Xiaoyi, for example, by saving it to... Figure 4 The interface shown is 40.

[0138] In yet another example, the display prompt information may include: if the display interface switches from the first interface to the second interface, the prompt information is displayed on the second interface.

[0139] Accordingly, S1320 specifically involves: in response to a first user operation on the prompt information, displaying the first memory information on the third interface of the smart assistant, the third interface including the memory information to be saved (or confirmed).

[0140] In other words, the electronic device obtains initial memory information from the first interface. If the displayed interface of the electronic device is no longer the first interface when the initial memory information is obtained, such as the second interface, the electronic device will push a prompt message to remind the user that there is currently memory information to be saved. The user can then view the memory information to be saved on the third interface of the smart assistant based on this prompt message. For example, the user can click on the prompt message, and the electronic device will display the third interface in response to the user's click. Alternatively, the user can click on the APP icon corresponding to the prompt message, enter the APP, and then switch pages to access the third interface.

[0141] The above solution supports guiding users to save memory information via push notifications even after the user has switched display interfaces. This allows users to be aware in real-time that the electronic device has performed a memory retrieval operation and to save the retrieved information whenever needed.

[0142] For example, the first interface could be Figure 3 The interface shown is 30. The electronic device obtains the first memory information based on the content of interface 30. If the electronic device displays interface 31 instead of interface 30 when the first memory information is obtained, the electronic device will display a prompt message, i.e., notification 32, on interface 31. The user can click on notification 32, click on the Xiaoyi icon, or enter a third interface, such as interface 33, through other means. The user can view the memory information to be saved, i.e., the content in card 34, on interface 33.

[0143] Furthermore, the third interface includes a confirmation control corresponding to the first memory information, and the method 1300 may further include: in response to an operation on the confirmation control, saving the first memory information on the smart assistant.

[0144] For example, in Figure 3 In the example shown, the confirmation control is control 35. The user can click control 35, and in response to this click, Xiaoyi will save card 34. For example, after Xiaoyi saves card 34, the user can... Figure 4 View card 34 in the interface 40 shown. Figure 4 Card 41 is the same as card 34.

[0145] Passively acquiring first memory information, that is, acquiring first memory information based on user triggers.

[0146] For example, electronic devices can support various interaction methods such as voice and touch operation (e.g., circle selection, line drawing, text selection, etc.) to trigger the electronic device to obtain memory information based on the entire screen information or part of the screen information, so as to meet the user's needs for obtaining memory information in different situations.

[0147] The following examples illustrate the methods of passively acquiring first memory information, based on the different displayed content.

[0148] Example 1: The displayed content is the interface content of the first interface of the first application.

[0149] In some embodiments, S1310 may specifically include: in response to a first user operation on a dialogue window of a smart assistant, obtaining first memory information based on the content of a first interface. The dialogue window of the smart assistant is displayed floating on the first interface, which is a display interface of a first application.

[0150] Based on this solution, users can retrieve memory information by waking up Xiaoyi when they have a memory need.

[0151] For example, the first application can be a non-target application. As mentioned earlier, the electronic device can acquire memory information based on scanning the display interface of the target application. Conversely, for non-target applications, the electronic device can acquire memory information based on user triggers.

[0152] For example, the first user action may take the form of voice or text.

[0153] For example, the first interface could be Figure 5 The interface 50 shown can contain the first memory information. Figure 5 The content of card 52 shown can be used for the first user operation, which may be user voice input. Figure 5 The command shown is "Hey Celia, write down the screen information".

[0154] Example 2: The displayed content is the content in the first area of ​​the first interface.

[0155] In some embodiments, prior to S1310, the method may further include: determining a first region in response to a first user action on a first interface.

[0156] For example, the first user operation could be a selection or a line drawing. Based on this solution, users can perform selection or line drawing operations on the first interface, and the electronic device can obtain the first memory information based on the selected area or the line drawing content.

[0157] For example, the first interface could be Figure 7 The interface 71 shown can have the first area as follows: Figure 7 The area enclosed by line 73, as shown in (d) in the diagram.

[0158] In one example, before determining the first region in response to a first user operation on the first interface, the method 1300 may further include: in response to an operation on the fourth interface, performing intelligent screen recognition on the fourth interface to obtain the first interface, the first interface including a first control; and in response to an operation on the first control, activating the selection function.

[0159] The fourth interface can be any display interface, such as an image or an application page. For example, the fourth interface could be... Figure 7 The interface 70 is shown. Users can perform corresponding operations on the fourth interface to enable intelligent screen recognition. For example, a user can press the screen with two fingers on interface 70. In response to this operation, the electronic device will intelligently recognize the screen. Through intelligent screen recognition on interface 70, the electronic device will display the first interface, which could be, for example, [the first interface could be...]. Figure 7 Interface 71 is shown in (b) above. It should be understood that the first interface includes the content obtained by intelligently recognizing the fourth interface. Interface 71 may include a control 72 as a first control. By clicking the control 72, the electronic device can activate the selection function. After the electronic device activates the selection function, the user can select the content in interface 71 to obtain a first area, such as the area enclosed by the line 73.

[0160] In one implementation, the first interface further includes a second control; S1310 may specifically include: responding to the operation of the second control, waking up the smart assistant, and displaying the image corresponding to the first area in the dialog box of the smart assistant; and obtaining the first memory information based on the content in the image.

[0161] Accordingly, S1320 may specifically include: displaying the first memory information in the dialog box of the smart assistant.

[0162] For example, see Figure 7 In (c), the second control can be control 74, which the user can click. In response to the user clicking control 74, the electronic device will wake up Xiaoyi and... Figure 7 In the window 75 shown in (d), the image in the selected area is displayed, i.e., picture 77 in dialog box 76. Afterwards, the electronic device can simulate a user issuing and executing the command "Remember the content in the picture," displaying the first memory information obtained from picture 77, i.e., the content in card 78.

[0163] In another implementation, the method 1300 may further include: waking up the smart assistant after a preset time and displaying the image corresponding to the first region in the smart assistant's dialog box.

[0164] That is, in response to the user's first user operation on the first interface, the smart assistant is activated after a preset time after the first area in the first interface is determined, and the image corresponding to the first area is displayed in the smart assistant's dialog box.

[0165] For example, in Figure 7 In the example shown, after the electronic device displays line 73 based on the user's first operation on interface 71, it can wake up Xiaoyi after a preset time, such as 5 seconds, and then... Figure 7 In the window 75 shown in (d), the image in the selected area is displayed, i.e., picture 77 in dialog box 76. Afterwards, the electronic device can simulate a user issuing and executing the command "Remember the content in the picture," displaying the first memory information obtained from picture 77, i.e., the content in card 78.

[0166] Example 3: The displayed content is the first text.

[0167] In some embodiments, prior to S1310, the method may include: in response to a first user action on a first interface, copying first text in the first interface.

[0168] For example, the first interface could be Figure 8 The interface shown in 80 allows for a copy operation as the second operation, and the first text can be "I need to buy books for my daughter tomorrow".

[0169] Based on the above solution, it is possible to obtain memory information from a portion of the text displayed on the screen, thus meeting the needs of users who only want to remember information from a portion of the text displayed on the screen.

[0170] In one example, S1310 may specifically include: in response to an operation on a third control in the fifth interface of the smart assistant, displaying a first window on the fifth interface, the first window including a fourth control, the fifth interface displaying saved memory information, and the third control being used to add new memory information; in response to an operation on the fourth control, displaying first text in the first window; and extracting the first text from the first window to obtain the first memory information.

[0171] For example, the fifth interface could be the memory information management interface of a smart assistant or the interface / page for users to manage memory information. For instance, the fifth interface could be... Figure 8 In the interface 81 shown in (b), the third control can be control 84 in interface 81, and the first window can be... Figure 8Window 85 is shown in (c) above. Control 84 is used to add memory information. The user can click control 84, and in response to the click operation on control 84, the electronic device will display window 85 on interface 81. Window 85 includes a fourth control, such as control 86. The user can click control 86, and in response to the click operation on control 86, the first text "Buy books for my daughter tomorrow" which has been copied to the clipboard will be displayed in window 85. The electronic device can extract "Buy books for my daughter tomorrow" and obtain the first memory information based on "Buy books for my daughter tomorrow". For example, see... Figure 8 In step (d), in response to a click on control 86, window 85 not only displays the first text, but the electronic device can also retrieve "I need to buy a book for my daughter tomorrow." Based on "I need to buy a book for my daughter tomorrow," it obtains the first memory information and displays that first memory information, i.e., the content in card 88. See also... Figure 9 As shown in (a), in response to a click on control 86, the first text is displayed in window 85, and control 90 is also displayed; control 86 is no longer displayed. The user can click on control 90, and in response to the click, the electronic device can retrieve "I need to buy a book for my daughter tomorrow." Based on "I need to buy a book for my daughter tomorrow," after obtaining the first memory information, card 88 is displayed. Furthermore, during the process of obtaining the first memory information, a prompt message can also be displayed indicating that the first memory information is being obtained. For example, this prompt message could be... Figure 9 The prompt message 91 is shown in (b) above. After the electronic device acquires the first memory information, it may no longer display the prompt message 91.

[0172] Example 4: The displayed content is the first image.

[0173] In some embodiments, prior to S1310, the method may further include: displaying a first interface, the first interface including the first image, in response to a first user action on a thumbnail of a first image in a sixth interface.

[0174] For example, the sixth interface could be Figure 11 In the interface 112 shown in (c), the first image can be image 113. The user can click on the thumbnail of image 113. In response to the click on the thumbnail of image 113, the electronic device can display interface 114, which is an example of the first interface, and interface 114 includes image 113.

[0175] Further, S1310 may specifically include: performing text recognition on the first image in the first interface to obtain recognized text; in response to the operation of the fifth control in the first interface, displaying the seventh interface of the smart assistant, on which a second window is displayed, the second window including the recognized text; in response to the operation of the sixth control in the second window, obtaining the first memory information based on the recognized text.

[0176] For example, the first interface could be Figure 11 In the interface 114 shown in (d), the first image can be image 113, and the fifth control can be control 116 in interface 114. The user clicks... Figure 11 After viewing the thumbnail of image 113 in interface 112 shown in (c), the electronic device not only displays interface 114 but also performs text recognition on image 113 to obtain the text in image 113, i.e., text recognition. See also Figure 11 In (d), the recognized text is the content in recognition box 115. After obtaining the recognized text, the user can click control 116. In response to the click on control 116, the electronic device will display the seventh interface, for example... Figure 11 Interface 81, shown in (e), displays a second window, such as window 110, which displays the identification text "Shop transfer contact person: Ms. Zhang 130XXX". Window 110 may include a sixth control, such as control 117; the user can click control 117, and in response to the click, the electronic device will obtain first memory information based on the identification text, for example, the obtained first memory information is... Figure 11 The contents of card 118 shown in (f) are shown in the diagram.

[0177] In some embodiments, S1310 may specifically include: in response to a first user operation on a seventh control in the first interface, obtaining first memory information based on a first image in the first interface, wherein the seventh control is located in a third window of the first interface and the first image is displayed in the third window.

[0178] For example, the first interface could be Figure 12 The interface 81 shown in (d) can have a third window. Figure 12 In the interface 81 shown in (d), window 110 can have a first image, which can be an image in window 110, and a seventh control, which can be control 117 in window 110. The user can click control 117. In response to the click on control 117, the electronic device can retrieve first memory information based on the image in window 110. For example, the first memory information could be... Figure 12 The contents of card 118 shown in (e) of the diagram.

[0179] Optionally, before displaying the first interface in response to the first user action on a thumbnail of the first image in the second interface, the method 1300 may further include:

[0180] In response to an operation on the eighth control in the eighth interface of the smart assistant, a fourth window is displayed on the eighth interface. The fourth window includes a ninth control. The eighth interface displays saved memory information, and the eighth control is used to add new memory information. In response to an operation on the ninth control, a first option and a second option are displayed in the fourth window. The first option is used to select an image from the gallery, and the second option is used to take a photo. In response to an operation on the first option, the sixth interface is displayed.

[0181] For example, the eighth interface could be a memory information management interface or a user interface / page for managing memory information. For instance, the eighth interface could be... Figure 11 In the interface 81 shown in (a), the eighth control can be control 84 in interface 81, and the fourth window can be... Figure 11 Window 110 is shown in (b) above. A user can click control 84, and in response to the operation of control 84, the electronic device can display window 110. The ninth control can be an "Image Recognition" control in window 110. A user can click the "Image Recognition" control, and in response to the operation of the "Image Recognition" control, the electronic device can display a "Gallery Selection" option (an example of the first option) and a "Take Photo" option (an example of the second option). A user can click the "Gallery Selection" option, and in response to the click operation of the "Gallery Selection" option, the electronic device can display interface 112 (an example of the sixth interface), interface 112 including a thumbnail of at least one image in the gallery. The first image can be image 113 displayed in interface 112.

[0182] Based on the above method, memory information can be retrieved from images in the photo library. Alternatively, by clicking the second option, memory information can be retrieved from images taken by the user.

[0183] In some embodiments, the first memory information may include multiple memory information entries. The electronic device can selectively save a portion of the multiple memory information entries or save all of the memory information entries based on user instructions or operations via a smart assistant. This solution supports selective saving of memory information by the user.

[0184] For example, see Figure 6In the example shown, the first memory information includes cards 62 and 63. The user can click the "Remember All" control to save cards 62 and 63. Alternatively, the user can click card 62 and then click the "Remember Selected" control; in response to the user's click on the "Remember Selected" control, the electronic device will save card 62. Alternatively, window 61 may include controls such as "Remember First Memory" and / or "Remember Second Memory," and the user, for example, clicking "Remember First Memory" will save card 62.

[0185] It should be understood that, although the embodiments of this application are only used in... Figure 6 The scenario shown illustrates multiple memory information entries, but it should be understood that in other scenarios shown in the accompanying figures, the first memory information may also include multiple memory information entries.

[0186] In some embodiments, prior to S1320, the method 1300 may further include: displaying on the smart assistant a memory type that matches the display content of the first interface and attribute items corresponding to the memory type.

[0187] For example, see Figure 14 The user interface diagram shown is as follows. Figure 14 The interface shown can be responsive to user input. Figure 11 The interface diagram showing the operation of control 117 as shown in (e) is, or, Figure 14 The interface shown can be responsive to user input. Figure 12 The diagram shows the interface display of control 117, indicated by (d) in the figure. See also... Figure 14 Window 110 includes the memory type: Card. The attributes corresponding to the Card include: (1) Name: Ms. Zhang's Business Card; (2) Subject: Shop Transfer; (3) Type: Business Card; (4) Number: 130XXX. It should be noted that "Business Card" is a subtype of Card. Additionally, in some attached diagrams, such as... Figure 14 , Figure 15 or Figure 16 In Chinese, "memory type" can also be described as "recognition type," or a similar or related description.

[0188] For example, see Figure 15 The user interface diagram shown is as follows. Figure 15 The interface shown can be responsive to user input. Figure 8 The diagram shows the interface display of control 86 shown in (c). See also... Figure 15Window 85 includes the memory type: Important Day. The attributes corresponding to Important Day include: (1) Name: Buying books for my daughter; (2) Related person: Daughter; (3) Type: Other; (4) Date: 12 / 10. It should be noted that "Other" is a subtype of Important Day. For example, Important Day can include one or more of the following subtypes: birthday, anniversary, repayment date, exam date, payday, or others.

[0189] For example, see Figure 16 The user interface diagram shown illustrates that window 161 can include multiple memory types and corresponding attribute items for each memory type. This information is obtained based on "Son's information: Wang XX; ID card: 420XXX11; Phone number: 123XXX12". Memory type 1 and memory type 2 shown in window 161 are identical, both being cards / certificates. It should be understood that whether multiple memory types are identical is determined by the first content; multiple memory types may be identical or different. In the case of multiple identical memory types, the subtypes of multiple memory types may be identical or different.

[0190] When displaying multiple memory types, controls for viewing each memory type can also be displayed. For example, see... Figure 16 Users can view all memory types in window 161 by sliding control 163 up and down.

[0191] Optionally, at least some of the attribute items corresponding to the memory type can be edited. For example, see Figure 14 The type and number can be edited. For example, see [link to relevant documentation]. Figure 15 The type and date can be edited.

[0192] In some embodiments, in response to user operations on memory information, the memory type and the attribute information corresponding to the memory type are displayed.

[0193] For example, for Figure 4 The user can click on card 41, and in response, the electronic device can display the memory type and its corresponding attribute information. Furthermore, the user can edit some or all of the attribute information.

[0194] In some embodiments, the smart assistant further includes a tenth control, and method 1300 may further include: in response to an operation on the tenth control, displaying a first menu bar, the first menu bar including a plurality of memory type options; in response to an operation on a first memory type option among the plurality of memory type options, displaying on the smart assistant a first memory type that matches the display content of the first interface and at least one attribute item corresponding to the first memory type.

[0195] In other words, electronic devices can display attribute items corresponding to the selected memory type based on the chosen memory type. It should be understood that the retrieved memory information may differ depending on the memory type.

[0196] For example, the tenth control could be Figure 17 The control 171 shown can have a first menu bar that is Figure 17 The menu bar shown is 172. See also... Figure 17 The example shown, Figure 17 The memory types and corresponding attribute items in each of the diagrams shown are all obtained based on "Shop transfer contact person: Ms. Zhang 130XXX". Figure 17 The memory type shown in (a) can be the default memory type of the electronic device. If the user wishes to modify the memory type, they can click control 171. In response to clicking control 171, the following will be displayed: Figure 17 The menu bar 172 shown in (b) includes various memory types. For example, if the user clicks "Events," in response to the click, the electronic device displays the attributes corresponding to "Events," such as... Figure 17 As shown in (c) above. For example, if a user clicks "Orders," in response to the click, the electronic device displays the attribute items corresponding to "Orders" as follows: Figure 17 As shown in (d) in the diagram, it can be seen that when the memory type is "order", the electronic device does not obtain information matching the attribute item, and therefore does not obtain the first memory information.

[0197] In some embodiments, when displaying the memory type and the attribute information corresponding to the memory type, an eleventh control is also displayed; wherein, S1320 specifically involves: displaying first memory information in response to an operation on the eleventh control.

[0198] For example, see Figure 14 Window 110 may also include control 141, which can be clicked by the user. In response to clicking control 141, the electronic device can display, for example... Figure 11 In the case of window 110 (f), the memory type and its corresponding attribute information are no longer displayed. Instead, card 118 is displayed, which represents the first memory information. Alternatively, in response to a click on control 141, the electronic device no longer displays window 110 but instead displays card 118 on interface 81.

[0199] For example, see Figure 15 Window 85 may also include control 151, which the user can click. In response to clicking control 151, the electronic device can display, for example... Figure 8In the case of window 85 (d), the memory type and its corresponding attribute information are no longer displayed. Instead, card 88 is displayed, which represents the first memory information. Alternatively, in response to a click on control 151, the electronic device no longer displays window 85 but instead displays card 88 on interface 81.

[0200] Figure 18 This is an exemplary flowchart of an information recording method provided in this application. The following is a description of... Figure 18 The steps in method 1800 shown are explained.

[0201] S1801, Scan application display interface.

[0202] For example, it can scan the application's display interface based on a preset cycle, or it can scan the display interface when the display interface changes.

[0203] S1802, determine whether the application to which the first interface of the currently scanned application belongs is marked as the target application.

[0204] If the application to which the first interface belongs is marked as the target application, then S1803, S1804, or S1805 can be executed; otherwise, S1801 will continue to be executed.

[0205] In one example, if an electronic device retrieves memory information multiple times based on the application's display interface based on user triggers, the electronic device can use that application as the target application, or the user can mark that application as the target application.

[0206] For example, when an electronic device retrieves memory information based on a user's trigger and the display interface of a certain application, it can also count the number of times memory information has been retrieved based on that application's display interface. This count may or may not include the current retrieval. If the count exceeds a preset number, a pop-up message can prompt the user to mark the application as a target application. Based on this pop-up message, the user can mark the application as a target application.

[0207] S1803, determine whether the first probability Ta exceeds the preset probability.

[0208] Here, the first probability is Ta, which represents the probability that a user saves the first memory information obtained from the first interface based on the target prompt. The target prompt is used to indicate whether memory information has been obtained or whether there is currently memory information awaiting confirmation. Users can save memory information based on the target prompt. For example, the target prompt could be the first prompt information or the first notification mentioned earlier. Ta = Number of times the user saves the first memory information obtained from the first interface based on the target prompt / Number of times the electronic device displays the target prompt.

[0209] For example, suppose the first interface is Figure 1 As shown in interface 10, if the electronic device actively retrieves the first memory information 8 times based on interface 10, and the user saves the first memory information 5 times based on prompt information 12, then Ta = 5 / 8. It should be understood that the user saving the first memory information based on prompt information 12 means that the user clicked on prompt information 12 or entered a command similar to "remember the screen information".

[0210] If the execution result of S1803 is yes, then S1804 or S1805 can be executed. If the execution result of S1803 is no, then S1801 can be returned or S1804 can be executed.

[0211] S1804, determine whether the time interval since the last (i.e., the most recent) display of the target prompt is greater than the threshold Tt.

[0212] If the result of S1804 is yes, then S1805 can be executed; otherwise, S1801 can be returned.

[0213] S1805, extract information from the first interface to obtain the first content.

[0214] For example, visual analysis, interface framework analysis, and / or interface text recognition can be combined to extract the application page, text, and descriptive image data of the first interface. The content extracted based on the first interface is the first content.

[0215] S1806, determine whether the parsing value corresponding to the first interface is consistent with the parsing value of the application's display interface obtained most recently.

[0216] For example, the parsed value may be calculated based on the first content corresponding to the application's display interface, and the calculation may be a hash value, etc. It should be understood that the first content may be different for different display interfaces of the application.

[0217] In S1806, if it is determined that the parsed value obtained this time is consistent with the most recently obtained parsed value, then the first interface is considered to be a different display interface from the display interface of the application in the previous scan. Then S1807 can be executed, and the parsed value obtained this time can be recorded.

[0218] S1807, Obtain memory information based on the first content.

[0219] For example, the memory type that is compatible with the first application can be determined based on the application page, text and image data with descriptions of the first interface. Then, the corresponding mode is called to generate a Prompt to obtain memory information. The first memory information can be generated by calling LLM through the Prompt.

[0220] S1807 can be followed by S1808 or S1809.

[0221] S1808, determine whether the memory information obtained in S1807 is memory information that has not been saved.

[0222] Compare the information with the currently saved memory information to confirm whether the memory information obtained in S1807 is already saved memory information. If it is, S1801 can be executed; otherwise, S1809 can be executed.

[0223] S1809, determine whether the display interface is still on the first screen.

[0224] That is, determine whether the current display interface is the first interface. If it is, then S1810 can be executed; otherwise, S1811 can be executed. For ease of description, in the following text, when the current display interface is different from the first interface, the current display interface will be referred to as the second interface.

[0225] S1810 wakes up the smart assistant and displays the first prompt message in the smart assistant's dialogue window. That is, it prompts the user to save the remembered information in real time.

[0226] This step can be referred to in the description of the corresponding content above, and will not be repeated here.

[0227] S1812 can be executed after S1810.

[0228] In one example, if S1810 is executed once for a display interface of an application, the number of times the electronic device displays the target prompt can be incremented by 1. It should be understood that when S1810 is executed for the first time for a display interface of an application, the number of times the electronic device displays the target prompt is 1.

[0229] S1811, the first notification is displayed on the second application interface. That is, a non-real-time prompt for the user to save the remembered information.

[0230] This step can be referred to in the description of the corresponding content above, and will not be repeated here.

[0231] S1812 can be executed after S1811.

[0232] S1812, save memory information and / or user preference storage.

[0233] For details regarding the storage of memory information in S1810, S1811, and S1812, please refer to the descriptions of the relevant schemes above; they will not be repeated here.

[0234] As mentioned earlier, users can manually select memory types, and the user preference storage in S1812 is used for users to store their selection preferences for memory types for this application.

[0235] In one example, if S1812 is executed once for a display interface of an application, the number of times the user saves the memory information retrieved from that display interface according to the target prompt can be incremented by 1. It should be understood that when S1812 is executed for the first time for a display interface of an application, the number of times the user saves the memory information retrieved from that display interface according to the target prompt is 1.

[0236] According to the solution provided in this application, when a user is browsing certain applications, the electronic device can proactively acquire memory information, eliminating the need for the user to trigger the memory information acquisition step. Furthermore, by learning the user's habits in receiving memory information, the electronic device can develop a more automatic and intelligent memory information acquisition capability that better meets the user's expectations. Simultaneously, the memory information saving method, which supports both real-time and non-real-time prompts, is compatible with multiple user operation scenarios, ensuring that the recognized memory information is retained even after the user switches screens.

[0237] Figure 19 This is a schematic diagram of a system architecture provided in an embodiment of this application. The system architecture 1900 may include one or more modules shown in the figure, and the functions of each module are described below.

[0238] The intelligent voice module 1910: uses voice to acquire the corpus of first memory information, such as "remember the screen information", and sends the corresponding instruction to the memory trigger module 1914.

[0239] Clipboard module 1911: Supports copying text within the display interface.

[0240] Screen touch module 1912: Recognizes the user's touch operations on the interface, such as two-finger pressing, clicking, etc.

[0241] Screen drawing module 1913: Receives drawing instructions and draws lines / circles on the image.

[0242] Memory trigger module 1914: Receives signals / instructions sent by the smart voice module 1910, screen touch module 1912 and screen drawing module 1913, and passively triggers the acquisition of memory information; it can also actively trigger the acquisition of memory information according to the application's display interface.

[0243] Information extraction module 1915: can combine visual analysis, interface framework analysis and / or interface text recognition technologies to extract application pages, text and / or image data with descriptions.

[0244] Memory Platform 1916: After receiving the content sent by the information extraction module 1915, it determines the memory type that matches the received content and calls the corresponding pattern to generate a Prompt for the user to retrieve memory information. The Prompt then calls the LLM to generate the memory information. Furthermore, it can filter the memory information finally returned to the user based on the user's preference for memory type selection.

[0245] For example, the memory platform can be deployed on the electronic device side or on the server side.

[0246] User memory preference recording module 1917: Records the user's preference for the memory type of the application's display interface.

[0247] Memory Source Confirmation Module 1918: When it is detected that the same application has been frequently retrieved multiple times based on user triggers, it can be considered to remind the user to mark this application as a memory source or target application, and supports electronic devices to actively retrieve memory information based on the display interface of the memory source.

[0248] Memory storage module 1919: Used to store acquired memory information.

[0249] The system architecture provided in the embodiments of this application can support the acquisition of memory information according to different scenarios.

[0250] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the system architecture. In other embodiments of this application, the system architecture may include more or fewer modules than illustrated, or combine some modules, or split some modules, or have different module arrangements. The illustrated modules may be implemented in hardware, software, or a combination of software and hardware.

[0251] Figure 20 This is a schematic diagram of the structure of an electronic device provided in this application. The electronic device 2000 may include a processor 2010, an external memory interface 2020, an internal memory 2021, an antenna 1, an antenna 2, a mobile communication module 2030, a wireless communication module 2040, and a display screen 2050.

[0252] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 2000. In other embodiments of this application, the electronic device 2000 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0253] Processor 2010 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0254] The wireless communication function of the electronic device 2000 can be achieved through antenna 1, antenna 2, mobile communication module 2030, wireless communication module 2040, etc.

[0255] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Figure 1 The structures of antennas 1 and 2 shown are merely one example. Each antenna in electronic device 2000 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0256] The mobile communication module 2030 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 2000. The mobile communication module 2030 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 2030 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0257] The wireless communication module 2040 can provide at least one of the following wireless communication solutions for use in the electronic device 2000: wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). The wireless communication module 2040 can be one or more devices integrating at least one communication processing module. The wireless communication module 2040 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 2010.

[0258] Electronic device 2000 implements display function through display screen 2050. Display screen 2050 is used to display interactive interfaces, etc.

[0259] It should be understood that the electronic device 2000 can implement any of the technical solutions of this application.

[0260] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the information recording methods shown in the above embodiments.

[0261] This application also provides a chip, see [link to relevant documentation] Figure 21 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, it implements the information recording methods shown in the above embodiments.

[0262] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the information recording methods shown in the above embodiments.

[0263] This application also provides a computer program product, which includes a computer program that, when run by a terminal device, enables the terminal device to implement the information recording methods shown in the above embodiments.

[0264] It should be noted that the click operation in this application is merely an example, and the click operation can be replaced by touch, voice, or other operations. The names of pages / controls / windows, etc., provided in the embodiments of this application are also merely examples and do not constitute any limitation on this application.

[0265] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0266] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0267] In the embodiments provided in this application, the division of each framework or module is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple frameworks or modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0268] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0269] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0270] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0271] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0272] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An information recording method, characterized in that, include: First memory information is obtained, which is extracted based on the content displayed on the first interface, and the first memory information matches a preset memory type, wherein the memory type corresponds to at least one attribute item; The first memory information is displayed on the smart assistant.

2. The method as described in claim 1, characterized in that, Before displaying the first memory information on the smart assistant, the method further includes: Obtain a first user operation, which is used to instruct the saving of the first memory information, or the first user operation is used to instruct the display content.

3. The method as described in claim 2, characterized in that, The first user operation is used to instruct the saving of the first memory information; Prior to obtaining the first user action, the method further includes: A prompt message is displayed, indicating that there is currently memory information to be saved.

4. The method as described in claim 2, characterized in that, The first user operation is used to instruct the display content, which is one of the following: the interface content of the first interface of the first application, the display content in the first area of ​​the first interface, the first text in the first interface, and the first image in the first interface.

5. The method as described in claim 3, characterized in that, The displayed prompt information includes: If the display interface is not switched from the first interface, the prompt information is displayed in the dialogue window of the smart assistant, and the dialogue window is displayed floating on the first interface; The step of displaying the first memory information on the smart assistant includes: In response to the first user action on the dialog window, the first memory information is displayed in the dialog window.

6. The method as described in claim 3, characterized in that, The displayed prompt information includes: If the display interface switches from the first interface to the second interface, the prompt information is displayed on the second interface; The step of displaying the first memory information on the smart assistant includes: In response to the first user operation on the prompt information or in response to the first user operation on the application corresponding to the prompt information, the first memory information is displayed on the third interface of the smart assistant, the third interface including the memory information to be saved.

7. The method as described in claim 6, characterized in that, The third interface includes a confirmation control corresponding to the first memory information, and the method further includes: In response to the operation of the confirmation control, the first memory information is saved on the smart assistant.

8. The method according to any one of claims 2-7, characterized in that, The acquisition of the first memory information includes: Scan the display interface of the first application; The first memory information is obtained based on the first interface currently being scanned, where the first interface is a display interface of the first application.

9. The method as described in claim 8, characterized in that, The first application is the target application, wherein the electronic device is able to acquire memory information based on scanning the display interface of the target application.

10. The method as described in claim 4, characterized in that, The displayed content is the interface content of the first interface of the first application; The acquisition of the first memory information includes: In response to the first user operation on the dialogue window of the smart assistant, the first memory information is obtained according to the interface content of the first interface, the dialogue window is floating on the first interface, and the first interface is a display interface of the first application.

11. The method as described in claim 4, characterized in that, The displayed content is the content in the first area of ​​the first interface; Prior to acquiring the first memory information, the method further includes: In response to the first user action on the first interface, the first area is determined.

12. The method as described in claim 11, characterized in that, Before determining the first region in response to the first user action on the first interface, the method further includes: In response to an operation on the fourth interface, intelligent screen recognition is performed on the fourth interface to obtain the first interface, the first interface including a first control; In response to the operation on the first control, the selection function is activated.

13. The method as described in claim 12, characterized in that, The first interface also includes a second control; The step of obtaining the first memory information includes: In response to the operation of the second control, the smart assistant is activated, and the image corresponding to the first area is displayed in the dialog box of the smart assistant; The first memory information is obtained based on the content of the image.

14. The method as described in claim 4, characterized in that, The displayed content is the first text; Prior to acquiring the first memory information, the method further includes: In response to the first user action on the first interface, the first text is copied.

15. The method as described in claim 14, characterized in that, The step of obtaining the first memory information includes: In response to an operation on the third control in the fifth interface of the smart assistant, a first window is displayed on the fifth interface, the first window including a fourth control, the fifth interface displays saved memory information, and the third control is used to add new memory information; In response to an operation on the fourth control, the first text is displayed in the first window; Extract the first text from the first window to obtain the first memory information.

16. The method as described in claim 14, characterized in that, The displayed content is the first image; Prior to acquiring the first memory information, the method further includes: In response to a first user action on a thumbnail of the first image in a sixth interface, the first interface, including the first image, is displayed.

17. The method as described in claim 16, characterized in that, The acquisition of the first memory information includes: Perform text recognition on the first image to obtain the recognized text; In response to an operation on the fifth control in the first interface, a seventh interface of the smart assistant is displayed, on which a second window is displayed, the second window including the recognized text; In response to an operation on the sixth control in the second window, the first memory information is obtained based on the recognized text.

18. The method as described in claim 16, characterized in that, The step of obtaining the first memory information includes: In response to the first user operation on the seventh control in the first interface, the first memory information is obtained based on the first image, wherein the seventh control is located in the third window of the first interface and the first image is displayed in the third window.

19. The method according to any one of claims 16-18, characterized in that, Prior to the first user action in response to a thumbnail of the first image in the second interface, the method further includes: In response to an operation on the eighth control in the eighth interface of the smart assistant, a fourth window is displayed on the eighth interface, the fourth window including a ninth control, the eighth interface displays saved memory information, and the eighth control is used to add new memory information; In response to an operation on the ninth control, a first option and a second option are displayed in the fourth window, the first option being used to select an image from the gallery, and the second option being used to take a photo; In response to an action on the first option, the sixth interface is displayed, the sixth interface including a thumbnail of at least one image from the gallery.

20. The method according to any one of claims 1-19, characterized in that, Prior to displaying the first memory information on the smart assistant, the method further includes: The smart assistant displays a memory type that matches the content displayed on the first interface and at least one attribute item corresponding to the memory type.

21. The method as described in claim 20, characterized in that, Some or all of the at least one attribute item are editable.

22. The method as described in claim 20 or 21, characterized in that, The smart assistant includes a tenth control, and the method further includes: In response to an operation on the tenth control, a first menu bar is displayed, the first menu bar including multiple memory type options; In response to an operation on the first memory type option among the plurality of memory type options, the first memory type and at least one attribute item corresponding to the first memory type are displayed on the smart assistant, matching the display content of the first interface.

23. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors; The memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-22.

24. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-22.

25. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-22.

Images