Information processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an information processing method and device, electronic equipment and a storage medium. The method comprises the steps that in response to an information recording instruction, screen information of a current page is obtained, the screen information comprises in-screen visible information and in-screen invisible information, and the in-screen invisible information comprises content associated with the in-screen visible information; extracting at least one type of target information from the screen information; and recording the at least one type of target information. According to the method, the target information in the screen information of the current page can be quickly recorded through the information recording instruction.

Description

Technical Field

[0001] The present application belongs to the field of information processing technology, and specifically relates to an information processing method, device, electronic device and storage medium. Background Art

[0002] In the related art, when screen information is extracted, it is impossible to record the screen information comprehensively and quickly. Summary of the Invention

[0003] In view of the above problems, the present application proposes an information processing method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides an information processing method, the method comprising: in response to an information recording instruction, obtaining screen information of a current page, the screen information comprising visible information on the screen and invisible information on the screen, the invisible information on the screen comprising content associated with the visible information on the screen; extracting at least one target information from the screen information; and recording the at least one target information.

[0005] In second aspect, an embodiment of the present application provides an information processing device, which includes: a first acquisition unit, used to obtain screen information of the current page in response to an information recording instruction, wherein the screen information includes visible information on the screen and invisible information on the screen, and the invisible information on the screen includes content associated with the visible information on the screen; a second acquisition unit, used to obtain at least one target information from the screen information; and a recording unit, used to record the at least one target information.

[0006] In a third aspect, an embodiment of the present application provides an electronic device comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, wherein the above method is executed when the program code is run.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0009] Embodiments of the present application provide an information processing method, apparatus, electronic device, and storage medium. First, in response to an information recording instruction, screen information of the current page is obtained. Then, at least one target information is obtained from the screen information and recorded. Through the above method, the target information in the screen information of the current page can be quickly recorded using the information recording instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram showing an application scenario of an information processing method proposed in one embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram showing an application scenario of an information processing method proposed in one embodiment of the present application is shown;

[0013] Figure 3 A flowchart of an information processing method proposed in one embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram showing a control trigger in an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram showing a three-finger upward swipe trigger in an embodiment of the present application is shown;

[0016] Figure 6 A schematic diagram showing the triggering of the favorites button in an embodiment of the present application is shown;

[0017] Figure 7 A flowchart of an information processing method proposed in another embodiment of the present application is shown;

[0018] Figure 8 A schematic diagram showing the arrangement of a first control and a second control in another embodiment of the present application is shown;

[0019] Figure 9 A schematic diagram showing the arrangement of a first control and a second control in another embodiment of the present application is shown;

[0020] Figure 10 A schematic diagram showing the display of prompt information in another embodiment of the present application is shown;

[0021] Figure 11 A schematic diagram showing the display of prompt information in another embodiment of the present application is shown;

[0022] Figure 12 A schematic diagram of the process described in steps S210 to S240 in another embodiment of the present application is shown;

[0023] Figure 13 A flowchart of an information processing method proposed in another embodiment of the present application is shown;

[0024] Figure 14 A schematic diagram of the process described in step S320 in another embodiment of the present application is shown;

[0025] Figure 15 A schematic diagram of card switching in another embodiment of the present application is shown;

[0026] Figure 16 A schematic diagram of card switching in another embodiment of the present application is shown;

[0027] Figure 17 A schematic diagram showing the effect after card switching in another embodiment of the present application is shown;

[0028] Figure 18 A schematic diagram showing a card stacking display in another embodiment of the present application is shown;

[0029] Figure 19 A schematic diagram showing card stacking display after card switching in another embodiment of the present application is shown;

[0030] Figure 20 A flowchart of an information processing method proposed in another embodiment of the present application is shown;

[0031] Figure 21 A schematic diagram of a fusion acquisition framework in another embodiment of the present application is shown;

[0032] Figure 22 A schematic diagram of the process described in steps S410 to S440 in another embodiment of the present application is shown;

[0033] Figure 23 A structural block diagram of an information processing device proposed in an embodiment of the present application is shown;

[0034] Figure 24 A structural block diagram of an information processing device proposed in an embodiment of the present application is shown;

[0035] Figure 25 A structural block diagram of an electronic device or server for executing an information processing method according to an embodiment of the present application is shown in real time in the present application;

[0036] Figure 26A storage unit in real time of the present application is shown for storing or carrying program codes for implementing the information processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] In the context of rapid iteration of intelligent systems, efficient extraction of screen information has become a core technical requirement in fields such as industrial control, mobile application testing, and accessibility services. While researching related information processing methods, the inventors discovered that these methods are unable to quickly record screen information.

[0039] Therefore, the inventors have proposed the information processing method, apparatus, electronic device, and storage medium of the present application. First, in response to an information recording instruction, screen information of the current page is obtained, and then at least one target information is obtained from the screen information, and the at least one target information is recorded. Through the above method, the target information in the screen information of the current page can be quickly recorded using the information recording instruction.

[0040] In the embodiment of the present application, the information processing method provided can be executed by an electronic device. In this manner, all steps in the information processing method provided by the embodiment of the present application can be executed by the electronic device. For example, Figure 1 As shown, the information collection device of the electronic device 100 can respond to the information recording instruction to obtain the screen information of the current page, wherein the screen information includes visible information and invisible information on the screen, and the invisible information on the screen includes content associated with the visible information on the screen. The obtained screen information is then transmitted to the processor, so that the processor can extract at least one target information from the screen information in real time and record the at least one target information.

[0041] Furthermore, the information processing method provided in the embodiments of the present application can also be executed by a server (cloud). Correspondingly, in this server-side execution mode, the electronic device can respond to the information recording instruction, obtain the screen information of the current page, and synchronously send the screen information to the server. The server can then extract at least one target information from the screen information in real time and record the at least one target information.

[0042] In addition, the electronic device and the server may collaborate to perform the process. In this manner, some steps of the information processing method provided in the embodiment of the present application are performed by the electronic device, while other steps are performed by the server.

[0043] For example, Figure 2 As shown, the electronic device 100 can execute the information processing method including: responding to the information recording instruction, obtaining the screen information of the current page, and then the server 200 executes the extraction of at least one target information from the screen information, and then the electronic device 100 executes the recording of the at least one target information.

[0044] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.

[0045] It should be noted that the electronic device 100 can be used for Figure 1 and Figure 2 In addition to the smartphone shown in FIG, it can also be a car device, a wearable device, a tablet computer, a laptop computer, a smart speaker, etc. The server 200 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers.

[0046] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0047] See also Figure 3 , an information processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method includes:

[0048] Step S110: in response to the information recording instruction, obtaining screen information of the current page, wherein the screen information includes visible information on the screen and invisible information on the screen, wherein the invisible information on the screen includes content associated with the visible information on the screen.

[0049] In an embodiment of the present application, the information recording instruction is a triggered instruction for automatically collecting, preprocessing and recording the screen information of the current page. Among them, collecting refers to collecting screen information; preprocessing refers to extracting target information from the screen information; recording refers to writing the extracted target information into a preset storage area. The current page refers to the page that currently displays at least a partial area on the screen of the electronic device. Partial area means that the current page may not be fully displayed on the screen of the electronic device. In an embodiment of the present application, no matter what application scenario, as long as the information recording instruction is triggered, the screen information of the current page will be automatically obtained. Among them, the application scenario can be understood as the current operating scenario of the electronic device, such as a call scenario, a navigation scenario, a chat dialogue scenario, an information browsing scenario, etc., which is not specifically limited here.

[0050] In one embodiment, the on-screen visible information includes information currently displayed on the screen, and the on-screen invisible information includes content that is not currently displayed on the screen but is associated with the on-screen visible information.

[0051] Visible on-screen information includes information currently displayed on the screen, which refers to information on the current page currently displayed on the screen, and may include text, images, links, controls, screenshots, page layout information, etc., without specific limitations here. Invisible on-screen information includes content not currently displayed on the screen but associated with the visible on-screen information, which refers to content on the current page not currently displayed on the screen but associated with the visible on-screen information. "Associated" means that the invisible on-screen information may be a portion of the current page that is not displayed on the screen, or an invisible on-screen area of ​​the current page that requires a page turn or pull-down operation to be displayed. For example, if the current page is a long document, the visible on-screen information may be the portion of text currently displayed on the long document, and the invisible on-screen information may be the portion of text not currently displayed on the long document. Optionally, in embodiments of the present application, the invisible on-screen information may include screenshots, text, controls, jump links, page layout information, file paths, large image paths, etc., which are not currently displayed on the screen, without specific limitations here. Jump links may support jumps to the original text or to applications / applets.

[0052] Furthermore, to better meet the user's information recording needs, the screen information obtained can include only the type of information the user wants to record. For example, if the user only wants to record the text information of the current page, then when obtaining the screen information of the current page, only the text information visible on the screen of the current page and the text information not visible on the screen of the current page can be obtained. The type of screen information obtained can be pre-set in the corresponding information collection rules.

[0053] In an embodiment of the present application, the triggering mode of the information recording instruction includes one of key triggering, gesture triggering, voice triggering, page switching triggering and control triggering.

[0054] Key triggering means that an electronic device may be provided with a key (which may be a physical key) specifically for triggering the information recording function. When it is detected that the user switches the key from the first state to the second state, it is determined that the information recording instruction has been triggered. The first state is the initial state of the key, that is, the state in which the information recording function cannot be triggered, and the second state is the state after switching from the initial state, that is, the state in which the information recording function can be triggered. Optionally, in order to facilitate the control of the information recording function, the key in the embodiment of the present application may only include the first state and the second state. For example, Figure 4 As shown, Figure 4 A button 110 for controlling the information recording function is provided on the right side of the electronic device. When it is detected that the button 110 is in a pressed state, it can be determined that the information recording instruction is triggered.

[0055] Gesture triggering means that when a designated operation acting on the screen of an electronic device is detected, it is determined that an information recording instruction has been triggered. Among them, the designated operation is a pre-set gesture operation that can trigger an information recording instruction. For example, the designated operation can be a multi-finger swipe up operation (such as a three-finger swipe up), a multi-finger swipe left / right operation, etc., which are not specifically limited here; the designated operation acting on the screen of an electronic device can refer to a designated operation in which the finger touches the screen of the electronic device, or a designated operation within a preset distance range above the screen of the electronic device without touching the screen of the electronic device. For example, Figure 5 As shown, when the gesture trigger is set to when a three-finger swipe-up operation is detected on the screen of the electronic device, it is determined that the information recording instruction is triggered.

[0056] Voice triggering means that when a voice control command containing a preset keyword is detected, it can be determined that a message recording instruction has been triggered. The preset keyword is a pre-set keyword that can trigger the message recording function, for example, the preset keyword can be set to "message recording" or "open message recording", etc., without specific limitation here.

[0057] Page switching trigger means that when the switching of the designated page is detected, it can be determined that the information recording instruction is triggered. The designated page can be a pre-set page that can trigger the information recording function, for example, the designated page can be set to a page containing private information.

[0058] Control triggering means that when an operation is detected on a designated control displayed on the screen, it can be determined that an information recording instruction has been triggered. The designated control can be a pre-set virtual control that can trigger the information recording function. For example, Figure 6 As shown, the specified control can be set as a favorites button. Figure 6 In the example embodiment, when a click operation on the favorites button 120 is detected, it can be determined that an information recording instruction is triggered.

[0059] In the embodiments of the present application, the triggering of the above-mentioned information recording instruction can be monitored in various ways. For example, page switching triggering and control triggering can be achieved through control instrumentation technology, accessibility, AppSwitch, and other technologies; gesture triggering can be monitored by a touch sensor provided in the electronic device; voice triggering can be monitored by a voice recognition module provided in the electronic device; key triggering can be monitored by a listener provided in the electronic device, etc., without further specific limitations here.

[0060] When it is detected in the above manner that the information recording instruction is triggered, the screen information of the current page is obtained in response to the information recording instruction.

[0061] In the above-mentioned triggering methods of the information recording instruction, page switching triggering and control triggering can be understood as automatic triggering of the information recording instruction; key triggering, gesture triggering and voice triggering can be understood as manual triggering of the information recording instruction.

[0062] Step S120: extracting at least one target information from the screen information.

[0063] In the embodiments of the present application, target information is information obtained by extracting target fields from screen information and summarizing them according to preset rules. The extracted target information can be of various types, where the type refers to the category to which the extracted target information belongs, such as schedule, personal information, to-do list, text summary, structured information, etc. Preset rules are pre-set rules for obtaining target fields from screen information, and target fields are pre-set fields from which the user wants to extract information.

[0064] The visible information and invisible information included in the screen information can be embodied in various forms, such as text, pictures, numbers, special symbols and patterns, etc., which are not specifically limited here.

[0065] When extracting at least one target information from the screen information according to the preset rules, since the screen information may exist in various forms, different preset rules may be used to extract screen information in different forms.

[0066] Specifically, after the screen information of the current page is acquired in step S110 , at least one target information may be extracted from the acquired screen information of the current page according to a preset rule.

[0067] Step S130: Record the at least one target information.

[0068] In an embodiment of the present application, after obtaining at least one type of target information, the at least one type of target information may be recorded. Recording here means that the at least one type of target information may be saved.

[0069] When recording at least one type of target information, the at least one type of target information may be stored in different formats based on the category to which the target information belongs. For example, if the target information is schedule information, the target information may be recorded in the form of a schedule; if the target information is personal information, the target information may be recorded in an encrypted form, etc., without further limitation.

[0070] In the embodiment of the present application, when the information recording instruction is triggered, the processes described in step S110, step S120 and step S130 can be automatically performed, that is, the collection, preprocessing and recording processes of the screen information can be automatically performed.

[0071] The present application provides an information processing method, which first responds to an information recording instruction to obtain the screen information of the current page, then obtains at least one target information from the screen information, and records the at least one target information. Through the above method, the target information in the screen information of the current page can be quickly recorded through the information recording instruction.

[0072] See also Figure 7 , an information processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method includes:

[0073] Step S210: in response to the information recording instruction, obtaining screen information of the current page, wherein the screen information includes visible information on the screen and invisible information on the screen, wherein the invisible information on the screen includes content associated with the visible information on the screen.

[0074] Step S220: extracting at least one target information from the screen information.

[0075] Step S230: If the at least one type of target information includes target information belonging to a target type, determine an application corresponding to the target information belonging to the target type.

[0076] In an embodiment of the present application, a target type is a pre-set information type that requires the corresponding target information to be written to the corresponding application. The target type can be one or more and can be customized according to actual needs. For example, the target type can be a schedule, personal information, and a to-do list. Among them, the target information of one or more target types can be written to one or more corresponding applications. For example, the schedule, personal information, and to-do list can all be written to one or more corresponding applications. That is, the target information of one target type can be written to one application, or the target information of one target type can be written to multiple applications, or the target information of multiple target types can be written to one application, or the target information of multiple target types can be written to multiple applications.

[0077] As a method, the correspondence between target types and applications can be preset. After extracting at least one target information from the screen information, the application corresponding to the at least one target information can be determined based on the preset correspondence between target types and applications.

[0078] As an approach, the target type includes at least one of a schedule, to-do items, and personal identity information.

[0079] Among them, the schedule can be divided into multiple schedules, such as car rental schedule, travel schedule, flight schedule, performance / movie schedule, train schedule, hotel check-in schedule, attraction ticket schedule, meeting schedule, general schedule, etc., which are not specifically limited here. In the embodiment of the present application, different schedules can contain different fields respectively, and the application corresponding to the target information of the schedule class can be a calendar, which is not specifically limited here. For example, as shown in the following table, the car rental schedule can include:

[0080]

[0081] To-do refers to a to-do list, which refers to tasks that need to be completed within a certain period of time but have not yet been processed. In the embodiments of the present application, the to-do list can also be divided into multiple types of to-do lists, such as cooking to-do lists, performance to-do lists, purchase to-do lists, and movie ticket to-do lists, etc., which are not specifically limited here; different to-do lists can also contain different fields, and the application corresponding to the to-do list can be a notepad or a suggestion card, which are not specifically limited here.

[0082] Personal identity information can also be divided into multiple types of personal identity information, such as basic identity information and encrypted identity information, which are not specifically limited here. Similarly, different personal identity information can include different fields, and the application corresponding to the personal identity information can be a password book, which is not specifically limited here.

[0083] In the embodiment of the present application, since different types of target information may include different fields, when determining whether at least one type of target information includes target information of a target type, it may be determined by the fields included therein.

[0084] After the target information including the target type is determined through the fields included in at least one type of target information, the application corresponding to the target information of the target type may be further determined based on the correspondence between the target type and the application.

[0085] Step S240: Add the target information belonging to the target type to the corresponding application program.

[0086] In an embodiment of the present application, after the target information of the target type corresponds to the application program determined in the aforementioned manner, the target information of the target type can be written into the corresponding application program. For example, the target information of the schedule type can be written into the calendar, or personal identity information can be written into the password book, or to-do items can be written into a notepad or suggestion card, etc., without specific limitation here.

[0087] As a method, adding the target information belonging to the target type to the corresponding application includes: displaying prompt information to prompt confirmation whether to add the target information belonging to the target type to the corresponding application; and adding the target information belonging to the target type to the corresponding application in response to confirmation information of the prompt information.

[0088] Among them, the prompt information is generated based on the target information, and is used to prompt whether to add the target information belonging to the target type to the corresponding application. The prompt information can be displayed in a fluid cloud manner, wherein the fluid cloud refers to the function of dynamically displaying the status of high-priority real-time tasks in the status bar, lock screen, notification center, etc. in the form of capsules, bubbles or cards.

[0089] In an embodiment of the present application, prompt information corresponding to target information of different priorities can be displayed in different forms. Specifically, the target information can be divided into multiple priorities in advance according to the service priority, and then the prompt information corresponding to the target information can be displayed in different forms according to the priority of the target information, wherein the service priority can be customized in advance according to the needs or the system default, and the service priority can also be understood as the application priority. For example, the target information can be divided into high-priority target information, medium-priority target information and low-priority target information in advance according to the service priority, wherein the prompt information corresponding to the high-priority target information can be displayed in the form of a card, the prompt information corresponding to the medium-priority target information can be displayed in the form of a capsule, and the prompt information corresponding to the low-priority target information can be displayed in the form of a bubble, etc. The above display form is only an example, and of course the prompt information can also be displayed in other ways, which is not specifically limited here.

[0090] The confirmation information of the prompt information is information for confirming that the target information of the target type is added to the corresponding application. The confirmation information of the prompt information can be information triggered by a designated control in a display interface displaying the prompt information. The designated control in the display interface displaying the prompt interface can be a "yes" control for confirming that the target information of the target type is added to the corresponding application.

[0091] Specifically, the display interface for displaying prompt information may include two controls: a first control and a second control, wherein the first control is used to determine whether to add target information belonging to the target type to the corresponding application, and the second control is used to determine whether to add target information belonging to the target type to the corresponding application. In an embodiment of the present application, the first control and the second control can be set at different positions in the display interface for displaying prompt information. As a way, the first control and the second control can be set in the display interface in any display area other than the display area for prompt information, such as Figure 8 As shown, in the display interface 201, the display area 2011 is the area where the prompt information is displayed, and the display area 2012 can be understood as the area other than the prompt information. The first control and the second control can be set in the display area 2012. As another way, the first control and the second control can be set in the display interface, and the prompt information display area, such as Figure 9 As shown, the first control and the second control may be set in the display area 2011 .

[0092] Furthermore, in an embodiment of the present application, the setting method of the first control and the second control can be determined according to the display method of the prompt information. For example, if the prompt information is displayed in a fluid cloud manner in the current page, the first control and the second control can be set in the display area of ​​the prompt information; if the prompt information is displayed in a fluid cloud manner in a new display page, then at this time, the first control and the second control can be set in an area outside the display area of ​​the prompt information. Optionally, the first control and the second control can also be set in the display area of ​​the prompt information.

[0093] In an embodiment of the present application, if there are multiple target information of the determined target type, then when displaying the prompt information corresponding to the target information of the multiple target types, the multiple prompt information can be displayed at the same time, or the multiple prompt information can be displayed in sequence according to the service priority, which is not specifically limited here.

[0094] When displaying multiple prompt information at the same time, since the multiple prompt information may be displayed in different forms, the different forms of prompt information can be displayed in different positions of the display page according to the preset arrangement. The preset arrangement is the display area where the different forms of prompt information are to be displayed. Optionally, the multiple forms of prompt information can be displayed in a stacked manner, such as Figure 10 As shown, it can also be displayed in a tiled manner, such as Figure 11 As shown, no specific limitation is given here.

[0095] When multiple forms of prompt information are displayed in a tiled manner, if there are multiple forms of prompt information of the same form among the multiple forms of prompt information, the multiple forms of prompt information of the same form may be displayed in a stacked manner.

[0096] After determining that at least one type of target information includes target information of a target type and an application corresponding to the target information of the target type, corresponding prompt information can be generated based on the target information of the target type. When an operation acting on the first control is detected, confirmation information is determined to have triggered the prompt information, and the target information of the target type is added to the corresponding application. When generating prompt information based on target information of the target type, the prompt information can include a specified field in the target information of the target type, where the specified field is a field that characterizes the type of target information or a specific event. For example, the prompt information can be "Are you sure you want to add the current travel itinerary to the calendar?"

[0097] In an embodiment of the present application, when displaying prompt information, the target display duration of the prompt information can be determined based on the current display impact parameters.

[0098] Among them, the display influencing parameters are parameters obtained that affect the display of the information to be displayed. Among them, the display influencing parameters include at least one of the display size, pop-up method and light intensity of the information to be displayed, and may include the following situations. The first is that the display influencing parameters include one of the display size, pop-up method and light intensity of the prompt information. Specifically, the display influencing parameters may only include the display size of the prompt information; or, the display influencing parameters may only include the pop-up method of the prompt information; or, the display influencing parameters may only include the light intensity. The second is that the display influencing parameters include any two of the display size, pop-up method and light intensity of the prompt information. Specifically, the display influencing parameters may include the display size and pop-up method of the prompt information; or, the display influencing parameters may include the display size and light intensity of the prompt information; or, the display influencing parameters may include the pop-up method and light intensity of the prompt information. The third is that the display influencing parameters may include parameters such as the display size, pop-up method and light intensity of the prompt information.

[0099] The prompt information may be at least one of a prompt text or a prompt picture. In the case where the prompt information is a prompt text, the display size of the prompt information may refer to the font size of the prompt text, and the font may be any one of all fonts supported by the electronic device; in the case where the prompt information is a prompt picture, the display size of the prompt information may refer to the picture size of the prompt picture. The pop-up mode of the prompt information may include an active pop-up mode and a passive pop-up mode. Among them, the active pop-up mode refers to the pop-up of a prompt related to a certain functional module when clicking on it; the passive pop-up mode refers to the display of random content when the electronic device is used naturally. That is, the active pop-up mode refers to the pop-up mode of the prompt information triggered by a preset operation, and the passive pop-up mode refers to the mode of automatically popping up the prompt information. Among them, the preset operation may be an operation acting on a certain functional module of the electronic device, such as a click operation, a sliding operation or a long press operation, etc., which are not specifically limited here. The pop-up mode of the prompt information can be determined based on the generation method of the prompt information.

[0100] The light intensity may refer to the light intensity of the environment in which the electronic device is located; alternatively, the light intensity may refer to the screen brightness of the electronic device, without specific limitation herein. As one approach, when the light intensity refers to the light intensity of the environment in which the electronic device is located, a photosensor in the electronic device may be configured to obtain the current light intensity.

[0101] The target display duration refers to the target display duration of the prompt information on the screen of the electronic device. Different display influencing parameters may correspond to different target display durations.

[0102] As a method, different types of display influence parameters can be divided into multiple levels, and then different target display times can be set for different types of display influence parameters at different levels, that is, a correspondence between the display influence parameters and the target display times is established, so that after obtaining the current display influence parameters, the target display time corresponding to the current display influence parameters can be determined based on the correspondence between the display influence parameters and the target display time.

[0103] In the case where the current display influence parameter is a parameter of one type, after obtaining the current display influence parameter, the target display duration corresponding to the current display influence parameter can be determined based on the correspondence between the display influence parameter and the target display duration, that is, the target display duration of the prompt information can be determined; in the case where the current display influence parameter includes multiple types of parameters, the target display duration corresponding to each of the current display influence parameters can be determined based on the correspondence between the display influence parameter and the target display duration, and then the target display duration corresponding to each of the multiple types of display influence parameters can be weightedly summed to obtain the target display duration of the prompt information.

[0104] As another approach, determining the target display duration of the prompt information based on the current display influencing parameters may refer to adjusting the display duration of the prompt information based on the current display influencing parameters to obtain the target display duration of the prompt information. Adjustment refers to making adjustments based on a baseline display duration. The baseline display duration may be a default display duration, or the baseline display duration may refer to the display duration of the prompt information under a set of default display influencing parameters. For example, the baseline display duration may be the display duration of the prompt information when the display size is regular font, the illumination intensity is normal light, and the pop-up mode is active pop-up mode.

[0105] By associating different display impact parameters with the display duration of the prompt information, the display duration of the prompt information can be flexibly set, thereby helping users to better read, write and understand the prompt information, thereby improving the user's cognitive experience.

[0106] Optionally, in an embodiment of the present application, after target information belonging to a target type is added to a corresponding application, at least one target information may also be recorded. Recording here may refer to storing at least one target information in a preset storage area.

[0107] For example, the process described in step S210 to step S240 can be as follows: Figure 12As shown, screen capture is triggered by means such as buttons / gestures / voices (equivalent to responding to information recording instructions in this application) to obtain the screen information of the current page. After obtaining the screen information of the current page, at least one target information is extracted from the screen information through operations such as OCR (Optical Character Recognition) / parsing, key information extraction, and classification. After extracting at least one target information, it is determined whether the at least one target information contains target information of the type of schedule / to-do / personal identity information. If so, a prompt is given in a fluid cloud manner to determine whether to add the target information of the type of schedule / to-do / personal identity information to the corresponding application. If the user confirms to add, the target information of the type of schedule / to-do / personal identity information is added to the corresponding application, and the at least one target information and the created status are stored in a preset storage area. If there is no target information of the type of schedule / to-do / personal identity information, or the user confirms not to add, the at least one target information is directly stored in the preset storage area. The created status may include the fact that the target information of the type of schedule / to-do / personal identity information has been added to the corresponding application, the time when it was added to the corresponding application, etc.

[0108] This application provides an information processing method that first responds to an information recording instruction, obtains screen information of the current page, then extracts at least one type of target information from the screen information, and if the at least one type of target information includes target information of a target type, determines the application corresponding to the target information of the target type, and adds the target information of the target type to the corresponding application. This method can quickly add target information of the target type to the corresponding application, improving the user experience.

[0109] See also Figure 13 , an information processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method includes:

[0110] Step S310: in response to the information recording instruction, obtaining screen information of the current page, wherein the screen information includes visible information on the screen and invisible information on the screen, wherein the invisible information on the screen includes content associated with the visible information on the screen.

[0111] Step S320: input the screen information into multiple pre-trained large models respectively, obtain the target information of the screen information output by each of the multiple large models, and obtain at least one target information, wherein different large models are used to extract information from different types of screen information.

[0112] In an embodiment of the present application, multiple pre-trained large models are used to extract target information from different types of screen information, that is, different large models have different capabilities. For example, multiple pre-trained large models can be OCR models, NER (Named Entity Recognition) models, and image classification and summarization models. Specifically, the OCR model provides the ability to extract text from images; the NER model is used to extract schedule / personal identity information; the image classification and summarization model is used for structured extraction, such as extracting movie ticket information from an order page. Different types can refer to types such as images or texts, or they can refer to different extracted fields.

[0113] Furthermore, multiple pre-trained large models can be deployed in the cloud server. When the capabilities of multiple pre-trained large models need to be invoked, large models with different capabilities can be invoked by calling different AI (Artificial Intelligence) capability interfaces. One large model corresponds to one AI capability interface.

[0114] As a method, multiple pre-trained large models can be called at the same time. After obtaining the screen information, the screen information is input into the multiple pre-trained large models respectively to obtain at least one target information output by the multiple pre-trained large models.

[0115] Optionally, multiple pre-trained large models can also be deployed in the electronic device. When the capabilities of multiple pre-trained large models need to be called, the multiple pre-trained large models can be called directly in the electronic device.

[0116] After the target information of the screen information is output by the various large models, it can be further determined whether the target information output by the large model complies with safety regulations. If it does, the target information is recorded; if it does not, the target information is discarded.

[0117] For example, the process described in step S320 can be as follows: Figure 14 As shown, in Figure 14 The electronic device side and the cloud side can be included. Among them, the electronic device side includes a screen recognition application and an AI unit, and the cloud side is used to provide back-end services. The cloud side is configured with a variety of pre-trained large models (such as OCR models, NER models, etc.). When the target information of the screen information of the current page obtained is extracted by the recognition application, the AI ​​capability interface in the AI ​​unit can be called to call large models with different functions to distribute and process the screen information. Specifically, all the screen information can be given to the large model, and the relevant fields can be extracted by the large model to obtain at least one target information.

[0118] Step S330: Record the at least one target information.

[0119] Step S340: If the at least one target information includes card data, generate and display a corresponding card based on the card data.

[0120] In the embodiments of the present application, card data refers to target information that can generate cards. A card is a form of information presented by an electronic device to a user, and can include pictures, text, links, controls, and other information related to the same topic. For example, weather cards, stock cards, and news cards, etc. In the embodiments of the present application, a pre-trained data recognition model can be used to determine whether at least one target information contains card data. Specifically, at least one target information can be input into a pre-trained data recognition model, and the data recognition model can recognize the at least one target information and then output the card data in the at least one target information.

[0121] As a way, a card can be the entrance to the application (application, APP) corresponding to the card. The user can operate the card to open the application corresponding to the card, so that the electronic device presents the interface of the application corresponding to the card to the user, so that the user can view more detailed information on the interface. Furthermore, after opening the application corresponding to the card, the user can perform corresponding operations on the interface of the application to meet their own needs. For example, the application corresponding to the weather card is weather. The user can open the application weather by operating the weather card, so that the electronic device presents the weather interface to the user. The user can set the information displayed on the weather card, or view the weather conditions of a certain city, etc. by operating the weather interface to meet their own needs.

[0122] As another way, the card can also be the entrance to one or more services provided by the application corresponding to the card. The user can operate the card to open the services provided by the application corresponding to the card, so that the electronic device presents the service interface of the application corresponding to the card to the user.

[0123] Among them, the application can be a native application or a third-party application in the electronic device, which is not specifically limited here. Among them, the native application refers to an application that is pre-installed on the electronic device, which is an application that comes with the electronic device. When the electronic device is turned on for the first time, the desktop of the electronic device will present an icon of the native application. Third-party applications are different from native applications. Third-party applications refer to applications produced by a third party and need to be installed by the user on the electronic device. Optionally, the electronic device can be any one of a mobile phone, tablet computer, desktop, laptop, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), wearable electronic device, smart watch and other devices, which is not specifically limited here.

[0124] Furthermore, in an embodiment of the present application, a card can be displayed on the desktop or the negative one screen of an electronic device, which is not specifically limited here. Among them, the negative one screen refers to sliding the screen to the right on the main screen of the electronic device until sliding to the user interface of the leftmost split screen. The negative one screen can be used to place some quick service functions and notification messages, such as global search, quick entry to a page of an application (payment code, WeChat, etc.), instant messages and reminders (express information, expenditure information, commuting conditions, taxi travel information, schedule information, etc.) and follow-up dynamics (football stands, basketball stands, stock information, etc.).

[0125] Multiple cards may be included in the desktop or the negative one screen. The cards included in the desktop are different from the cards included in the negative one screen. The difference here may be at least one of the following: different card types, different card displayed contents, different card corresponding applications, different card corresponding services, etc.

[0126] In an embodiment of the present application, a corresponding card is generated in response to a first operation applied to card data. The first operation may be at least one of a two-finger press operation, a long press operation, a click operation, or a slide operation, which is not specifically limited here. The first operation may be at least one of a two-finger press operation, a click operation, a slide operation, or a long press operation. The two-finger press operation and the long press operation may be operations that trigger card conversion of card data, the click operation may be an operation that triggers card conversion of one type of card data, and the slide operation may be an operation that triggers card conversion of at least one type of card data, which is not specifically limited here.

[0127] In the embodiment of the present application, different types of card data can generate different types of cards.

[0128] Different cards have different card functions. Therefore, each card has its corresponding card display rules. The card display rules include the rules for generating the graphic style and color of the card itself, as well as the display rules for the card display data. For example, when a card is used to display the user's exercise status, different card colors are used to represent the user's current exercise intensity according to the length of the exercise time. The data displayed on the card also includes mapped user exercise items. When the user is walking, a walking person silhouette icon is displayed; when the user is running, a running person silhouette icon is displayed, and so on. The main purpose of the different display rules for cards is to display the data that the card needs to display in a striking manner, so as to facilitate user identification. Therefore, in this embodiment, the differentiated display that is conducive to user identification can be used in cards that implement different functions, and is not limited to the above-mentioned embodiments.

[0129] After different cards are generated based on different types of card data, they can be displayed based on the card display rules corresponding to the different cards.

[0130] In the embodiment of the application, when generating cards based on card data, multiple cards may be generated at the same time, and then multiple cards can be displayed by card switching. Among them, the card switching method can be assumed to be a click switch, referring to Figure 15 , Figure 15 If a card switching instruction based on the upward sliding of card 1 is detected, card 1 will slide upward and switch to the next card. The switching process refers to Figure 16 . Figure 16 Card 1 continues to slide upwards, and the next card is Card 2, which also slides upwards. The final technical effect of the sliding operation is referenced Figure 17 , Card 1 slides up and disappears, and Card 2 will be displayed on the currently displayed page.

[0131] Optionally, when displaying multiple cards, the multiple cards can also be displayed in a stacked manner. When displaying multiple cards in a stacked manner, the card displayed in the top layer of the currently displayed page can be switched by sliding or clicking on the card switch. When switching the card displayed in the top layer of the currently displayed page by sliding or clicking on the card switch, if the card displayed in the top layer of the currently displayed page is clicked or slid once, the card will be switched to be displayed in the bottom layer of the currently displayed page. For example, Figure 18 As shown, in Figure 18The card 1, card 2, card 3, card 4 and card 5 are included. Currently, card 1, card 2, card 3, card 4 and card 5 are stacked and displayed in the display page 101. Card 1 is displayed on the top layer of the display page 101, and the layers displayed one layer below are card 2, card 3, card 4 and card 5. When a switching operation is detected on card 1, card 1 is displayed in the layer where card 5 was originally displayed, and card 2 is displayed in the layer where card 1 was originally displayed. The layers displayed by card 2, card 3, card 4 and card 5 are moved up one layer in turn, as shown in FIG. Figure 19 As shown, Figure 19 This is a schematic diagram after card 1 is switched and displayed.

[0132] Optionally, when multiple cards are displayed, the multiple cards may be divided into two parts for display, wherein one part of the cards is displayed on the desktop of the electronic device, and the other part of the cards is displayed on the negative one screen of the electronic device.

[0133] Of course, the generated multiple cards can also be displayed in other ways, which are not specifically limited here.

[0134] In order to facilitate users to view various types of information, users can set cards corresponding to different types of information, so that the electronic device can display a certain type of information in units of cards, making it convenient for users to view information intuitively.

[0135] The present application provides an information processing method that displays target information in the form of cards, which can facilitate users to view information intuitively.

[0136] See also Figure 20 , an information processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method includes:

[0137] Step S410: In response to the information recording instruction, determine whether the current page is in the page whitelist.

[0138] In an embodiment of the present application, the page whitelist may include a plurality of pre-set legal pages that the user is allowed to access. The page whitelist may store page identifiers of legal pages that the user is allowed to access, and then when determining whether the current page is in the page whitelist, it may be determined whether the current page is in the page whitelist based on the page identifier of the current page. Specifically, if the page identifier of the current page can be found in the page whitelist, it can be determined that the current page is in the page whitelist; otherwise, or if the page identifier of the current page is not found in the page whitelist, it can be determined that the current page is not in the page whitelist. Among them, the page identifier can be understood as an identifier for uniquely identifying a page, and one page may correspond to one page identifier.

[0139] Step S420: If it is determined that the current page is in the page whitelist, the screen information of the current page is obtained.

[0140] In an embodiment of the present application, after determining that the current page is in the page whitelist, the screen information of the current page can be obtained through pre-set information collection rules.

[0141] Among them, Figure 21 As shown, Figure 21 The fusion collection framework shown may include a variety of information collection rules, and the comprehensive collection of screen information of the current page may be achieved through the fusion collection framework.

[0142] As one approach, if it is determined that the current page is in the page whitelist, obtaining the screen information of the current page includes: obtaining the information collection rules of the current page; if it is determined that the current page is in the page whitelist, obtaining the screen information of the current page based on the information collection rules.

[0143] Different pages may correspond to different information collection rules, and one page may correspond to at least one information collection rule, so as to comprehensively collect target information in the page.

[0144] In an embodiment of the present application, the correspondence between pages and information collection rules can be pre-set. After determining that the current page is on the page whitelist, the information collection rules for the current page can be determined based on the correspondence between the page and the information collection rules. After determining the information collection rules for the current page, the screen information of the current page can be obtained according to the information collection rules of the current page.

[0145] Step S430: extracting at least one target information from the screen information.

[0146] Step S440: Record the at least one target information.

[0147] For example, the process described in step S410 to step S440 can be as follows: Figure 22 As shown, in Figure 21 In the process, the target information in the screen information can be quickly recorded through the following steps: triggering the information recording instruction -> extracting the screen information -> preprocessing the screen information -> storing -> manually writing -> querying and deleting.

[0148] The present application provides an information processing method that collects the screen information of the current page through corresponding information collection rules only after determining that the current page is in the page whitelist, thereby ensuring that the collected screen information is valid information. At the same time, different pages collect screen information through different information collection rules, thereby realizing comprehensive collection of the screen information of the current page.

[0149] See also Figure 23 , an embodiment of the present application provides an information processing device 500, the device 500 comprising:

[0150] The first acquiring unit 510 is configured to acquire screen information of the current page in response to an information recording instruction, wherein the screen information includes visible information on the screen and invisible information on the screen, wherein the invisible information on the screen includes content associated with the visible information on the screen.

[0151] As one approach, the first acquiring unit 510 is specifically configured to determine whether the current page is in the page whitelist in response to the information recording instruction; if it is determined that the current page is in the page whitelist, acquire screen information of the current page.

[0152] As another way, the first acquisition unit 510 is specifically used to acquire the information acquisition rule of the current page; if it is determined that the current page is in the page whitelist, the screen information of the current page is acquired based on the information acquisition rule.

[0153] The triggering mode of the information recording instruction includes one of key triggering, gesture triggering, voice triggering, page switching triggering and control triggering.

[0154] The second acquiring unit 520 is configured to acquire at least one type of target information from the screen information.

[0155] As a method, the second acquisition unit 520 is specifically used to input the screen information into multiple pre-trained large models respectively, obtain the target information of the screen information output by each of the multiple large models, and obtain at least one target information, wherein different large models are used to extract information from different types of screen information.

[0156] The recording unit 530 is configured to record the at least one type of target information.

[0157] As a method, the recording unit 530 is specifically used to determine the application corresponding to the target information belonging to the target type if the at least one target information includes target information belonging to the target type; and add the target information belonging to the target type to the respective corresponding applications.

[0158] As another method, the recording unit 530 is specifically used to display prompt information, prompting confirmation whether to add the target information belonging to the target type to the corresponding application; in response to the confirmation information of the prompt information, the target information belonging to the target type is added to the corresponding application.

[0159] The target type includes at least one of schedule, to-do list and personal identity information.

[0160] Optional, see Figure 24 , the apparatus 500 may further include:

[0161] The generating unit 540 is configured to generate and display a corresponding card based on the card data if the at least one target information includes card data.

[0162] It should be noted that the device embodiment in this application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0163] The following will be combined Figure 25 An electronic device or server provided in this application is described.

[0164] See also Figure 25 Based on the above-mentioned information processing method and apparatus, the embodiments of the present application also provide another electronic device or server 800 that can execute the above-mentioned information processing method. The electronic device or server 800 includes one or more (only one is shown in the figure) processors 802, a memory 804, and a network module 806 that are coupled to each other. The memory 804 stores a program that can execute the content of the above-mentioned embodiments, and the processor 802 can execute the program stored in the memory 804.

[0165] The processor 802 may include one or more processing cores. The processor 802 utilizes various interfaces and circuits to connect various components within the server 800. It executes instructions, programs, code sets, or instruction sets stored in the memory 804, and accesses data stored in the memory 804 to perform various server 800 functions and process data. Optionally, the processor 802 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 802 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 802 and may be implemented separately via a communications chip.

[0166] The memory 804 may include a random access memory (RAM) or a read-only memory (ROM). The memory 804 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 804 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat history data) created by the electronic device or server 800 during use.

[0167] The network module 806 is used to receive and transmit electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 806 may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, a memory, etc. The network module 806 can communicate with various networks such as the Internet, an enterprise intranet, a wireless network, or communicate with other devices via a wireless network. The above-mentioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 806 can exchange information with a base station.

[0168] Please refer to Figure 26 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 900 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0169] The computer-readable storage medium 900 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 for executing any of the method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 910 can be compressed, for example, in a suitable form.

[0170] This application provides an information processing method, apparatus, electronic device, and storage medium. The method first responds to an information recording instruction to obtain screen information of the current page, then obtains at least one target information from the screen information, and records the at least one target information. The method, using the information recording instruction, can quickly record the target information in the screen information of the current page.

[0171] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. An information processing method, characterized in that: The method comprises: In response to the information recording instruction, screen information of the current page is acquired, where the screen information includes visible information on the screen and invisible information on the screen, where the invisible information on the screen includes content associated with the visible information on the screen; extracting at least one target information from the screen information; The at least one target information is recorded.

2. The method according to claim 1, characterized in that The recording of the at least one target information includes: If the at least one target information includes target information belonging to a target type, determining an application corresponding to the target information belonging to the target type; The target information belonging to the target type is added to the corresponding application program.

3. The method according to claim 2, characterized in that The adding the target information belonging to the target type to the corresponding application includes: Displaying a prompt message for prompting confirmation of whether to add the target information belonging to the target type to the corresponding application; In response to confirmation information of the prompt information, the target information belonging to the target type is added to the corresponding application.

4. The method according to claim 2, characterized in that The target type includes at least one of a schedule, to-do items, and personal identity information.

5. The method according to claim 1, wherein The method further comprises: If the at least one target information includes card data, a corresponding card is generated and displayed based on the card data.

6. The method according to claim 1, characterized in that The extracting at least one target information from the screen information includes: The screen information is input into multiple pre-trained large models respectively, and the target information of the screen information output by each of the multiple large models is obtained to obtain at least one target information, wherein different large models are used to extract information from different types of screen information.

7. The method according to claim 1, characterized in that The step of obtaining screen information of the current page in response to the information recording instruction includes: In response to the information recording instruction, determining whether the current page is in the page whitelist; If it is determined that the current page is in the page whitelist, the screen information of the current page is obtained.

8. The method according to claim 7, characterized in that If it is determined that the current page is in the page whitelist, the screen information of the current page is obtained, including: Get the information collection rules of the current page; If it is determined that the current page is in the page whitelist, the screen information of the current page is obtained based on the information collection rule.

9. The method according to claim 1, characterized in that The triggering mode of the information recording instruction includes one of key triggering, gesture triggering, voice triggering, page switching triggering and control triggering.

10. The method according to claim 1, characterized in that The on-screen visible information includes information currently displayed on the screen, and the on-screen invisible information includes content that is not currently displayed on the screen but is associated with the on-screen visible information.

11. An information processing device, characterized in that: The device comprises: a first acquiring unit, configured to acquire screen information of a current page in response to an information recording instruction, wherein the screen information includes visible information on the screen and invisible information on the screen, wherein the invisible information on the screen includes content associated with the visible information on the screen; a second acquiring unit, configured to acquire at least one target information from the screen information; A recording unit is used to record the at least one target information.

12. An electronic device, characterized in that: The method comprises one or more processors; one or more programs are stored in the memory and configured to execute the method according to any one of claims 1 to 10 by the one or more processors.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images