Screen identification method and electronic equipment

Abstract

The invention discloses a screen recognition method and electronic equipment, relates to the technical field of terminals, can mark a matched text entity based on an article entity included in a user interface so as to provide a mark for quickly acquiring information from the user interface for a user, and is beneficial to improving the man-machine interaction efficiency. The method comprises the steps that a first interface is displayed, and the first interface comprises a first article entity, a first text entity and a second text entity; and in response to a first triggering operation on the first interface, adding a first text mark to the first text entity, and not adding a mark to the second text entity. And displaying a second interface, wherein the second interface comprises a second article entity, a third text entity and a fourth text entity. And in response to a second trigger operation on the second interface, adding a second text mark to the third text entity, and not adding a mark to the fourth text entity.

Description

Screen recognition method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 3, 2023, with application number 202311461131.5 and invention name “A method and electronic device for intelligent identification”, the entire contents of which are incorporated by reference into this application.

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 26, 2023, with application number 202311821050.1 and invention name “A screen recognition method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The embodiments of the present application relate to the field of terminal technology, and in particular to a screen recognition method and electronic device. Background Art

[0004] In the daily use of electronic devices such as mobile phones and tablets, the mobile phone may need to recognize the content in the user interface, such as identifying products and text in the user interface.

[0005] In the prior art, although there are some solutions for recognizing content in user interfaces, such as recognizing text in user interfaces through optical character recognition (OCR) technology, the recognition results cannot be presented well, thereby failing to help users obtain information efficiently.

[0006] Summary of the Invention

[0007] The present application provides a screen recognition method and electronic device that can identify the content in the user interface and recommend functions that match the interface based on the recognition results, so that the user can perform corresponding processing on the content in the user interface and improve the efficiency of human-computer interaction.

[0008] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0009] In a first aspect, the present application provides a screen recognition method, which is applied to an electronic device. A first interface is displayed, and the first interface includes a first object entity, a first text entity, and a second text entity. In response to a first trigger operation on the first interface, a first text tag is added to the first text entity, and no tag is added to the second text entity. A second interface is displayed, and the second interface includes a second object entity, a third text entity, and a fourth text entity. In response to a second trigger operation on the second interface, a second text tag is added to the third text entity, and no tag is added to the fourth text entity.

[0010] Among them, the entity categories of the first item entity and the second item entity are different, such as the first item entity is a building and the second item entity is an animal. That is to say, the first interface and the second interface include item entities of different entity categories. The entity category of the first text entity and the fourth text entity is the same, such as both are telephone numbers, and the entity category of the second text entity and the third text entity is the same, such as both are addresses. The entity categories of the first text entity and the second text entity are different. That is, not all text entities of entity categories will be marked. Moreover, text entities of the same entity category are not marked in both the first interface and the second interface.

[0011] In summary, by using the present application, an electronic device can mark text entities of the first category (the entity category of the first text entity and the fourth text entity) in an interface with a certain category (the entity category of the first item entity), but not mark text entities of the second category (the entity category of the second text entity and the third text entity); and an electronic device can mark text entities of the second category in an interface with another category (the entity category of the second item entity), but not mark text entities of the first category. It can be seen from this that an electronic device can mark matching text entities based on the item entities included in the user interface, thereby providing users with marks for quickly obtaining information from the user interface, which is conducive to improving the efficiency of human-computer interaction.

[0012] In one possible design of the first aspect, the entity categories of the text entity include at least two of the following: address, phone number, flight information, express delivery number, email address, website link, identification document number, and graphic code. The entity categories of the object entity include at least two of the following: animal, plant, building, and food.

[0013] That is, the electronic device can mark the above text entities that match the animals, plants, buildings and foods in the user interface.

[0014] In a possible design of the first aspect, the first text marker indicates the entity category of the first text entity, such as the first text marker is a category icon of the entity category. For example, if the first text entity is a phone number, the first marker can be a phone icon.

[0015] Alternatively, the first text entity is associated with multiple services. Taking the first text entity as a phone number as an example, the phone number can be associated with multiple services such as making a call, adding to contacts, etc. The first text tag can indicate the first service that the user is most interested in among multiple services, such as the first text tag being the service icon of the first service. Among them, the electronic device can regard the service that the user has selected the most times under the text entity of the first category as the first service of greatest interest, and the first category is the entity category of the first text entity. In this way, the electronic device can mark the text entity with a tag corresponding to the service that the user is interested in (such as a service icon).

[0016] It should be understood that the specific content of the second text mark can also refer to the first text entity, which will not be repeated here.

[0017] In one possible design of the first aspect, after adding a first text tag to the first text entity in response to a first triggering operation on the first interface, the method further includes: displaying a third interface in response to a third triggering operation on the first text entity or the first text tag, wherein the third interface includes multiple service options, the multiple service options corresponding one-to-one to the multiple services. Among the multiple service options, the service option for the first service is displayed first.

[0018] That is, the electronic device ranks the option of the first service that the user is most interested in first among the multiple service options so that the user can use the first service through the option of the first service. Of course, for other unlabeled text entities of the first category, the electronic device can also respond in the same way, that is, display the service option of the first service first.

[0019] It should be understood that the electronic device responds to the third trigger operation on the third text entity or the second text mark in the same way, which will not be elaborated here.

[0020] In a possible design of the first aspect, the first interface further includes a fifth text entity. The method further includes: in response to a first triggering operation on the first interface, and if the fifth text entity and the first text entity have different entity categories, adding a third text tag to the fifth text entity. If the fifth text entity and the first text entity have the same entity category, no tag is added to the fifth text entity.

[0021] That is to say, for text entities of the same entity category, the electronic device only displays a mark for one of the text entities to avoid repeated marks for text entities of the same entity category.

[0022] In a possible design of the first aspect, the first interface further includes a sixth text entity. The method further includes: in response to a first triggering operation on the first interface, and if the fourth entity marker of the sixth text entity does not obstruct the first text marker, adding a fourth text marker to the sixth text entity. If the fourth text marker obstructs the first text marker, no marker is added to the sixth text entity.

[0023] That is to say, the electronic device will display all the markers only when the markers do not block each other.

[0024] In one possible design of the first aspect, the first interface further includes a third item entity. The method further includes: in response to a first triggering operation on the first interface, highlighting the first item entity and displaying a first shortcut entry around the first item entity. In response to a fourth triggering operation on the third item entity, highlighting the third item entity and displaying a second shortcut entry around the first item entity. The highlighted item entity can be understood as a focused item entity.

[0025] That is, in response to a triggering operation by the user, the electronic device may switch the focus item entity and display a shortcut entry corresponding to the focus item entity so that the user can obtain information about the focus item entity.

[0026] In a possible design of the first aspect, the above-mentioned response to the first trigger operation on the first interface, highlighting the first item entity, and displaying the first quick entry around the first item entity includes: responding to the first trigger operation on the first interface, and the first item entity meets the first condition, highlighting the first item entity, and displaying the first quick entry around the first item entity.

[0027] The first condition includes at least one of the following:

[0028] Condition 1: The area of ​​the first item entity is greater than the area of ​​the third item entity. For example, the area of ​​the first item entity is the largest item entity in the first interface. In other words, the electronic device may prioritize the item entity with the larger area as the focus item entity.

[0029] Condition 2: The obscured area of ​​the first item entity is smaller than the obscured area of ​​the third item entity. It can be understood that a smaller obscured area of ​​the first item entity indicates a higher degree of integrity and a more comprehensive display of the first item entity. Thus, the electronic device can prioritize the item entity that can be fully displayed as the focal item entity.

[0030] Condition 3: The edge clarity of the first object entity is higher than the edge clarity of the third object entity. The higher the edge clarity of the first object entity, the more accurately the electronic device can extract the first object entity. In other words, the electronic device can prioritize the object entity with the more accurate extraction as the focal object entity.

[0031] At this point, it should be noted that the combination of conditions 1, 2, and 3 above—that is, the area of ​​the first object entity is larger than the area of ​​the third object entity, the area obscured by the first object entity is smaller than the area obscured by the third object entity, and the edge clarity of the first object entity is higher than the edge clarity of the third object entity—indicates that the first object entity has a larger area, is more complete, and has a clearer boundary. In other words, the electronic device can prioritize the object entity with a larger area and more accurate and complete cutout as the focal object entity.

[0032] In one possible design of the first aspect, the method further includes: in response to a first triggering operation on the first interface, displaying a first item marker on the third item entity, e.g., the first item marker is a circle. The fourth triggering operation includes a triggering operation on the first item marker. That is, the electronic device can use the first item marker as a clear trigger point.

[0033] In a possible design of the first aspect, when the first item entity is the focus item entity, the electronic device marks the first text entity. The above method further includes: in response to a fourth triggering operation on the third item entity, adding a third text mark to the second text entity.

[0034] That is, as the focus item entity switches, the text entity marked by the electronic device will also switch, such as switching from the first text entity to the second text entity. In this way, the electronic device can always ensure that the marked text entity matches the current focus item entity.

[0035] In a possible design of the first aspect, the method further includes: in response to a move operation on the highlighted item entity, displaying a fourth interface, the fourth interface including multiple associated entries, each associated entry corresponding to an application or a service, the multiple associated entries including a first associated entry, the first associated entry corresponding to the first application or the second service. That is, for the focus item entity, the electronic device can quickly provide associated entries. In response to moving the highlighted item entity to the first associated entry, the electronic device displays a fifth interface, the fifth interface is an interface of the first application or the second service, and the fifth interface includes associated information of the highlighted item entity. That is, the user only needs to move the focus item entity to the first association, and the electronic device can present the associated information of the focus item entity in the first application or the second service.

[0036] In this way, the user's operation can be simplified. The user does not need to first exit the current user interface, such as the first interface, and enter the desktop, then enter the interface of the first application or the second service, and finally search for the focus item entity in the interface of the first application or the second service, thereby improving the efficiency of human-computer interaction.

[0037] In one possible design of the first aspect, displaying the fourth interface in response to a move operation on the highlighted item entity includes: moving the highlighted item entity in the first interface in response to the move operation on the highlighted item entity; and displaying the fourth interface in response to the highlighted item entity moving to a target area in the first interface.

[0038] That is, the electronic device determines that there is a need to provide an associated entry only after the focus object physically moves to the target area, thereby improving the accuracy of the timing of providing the associated entry.

[0039] In one possible design of the first aspect, the highlighted item entity is a first item entity, and the plurality of associated entries include a second associated entry. The highlighted item entity is a third item entity, and the plurality of associated entries include a third associated entry. The second associated entry is different from the third associated entry.

[0040] That is to say, the associated entrances provided by the electronic device may be different depending on the focus object entity, so as to improve the pertinence of the provided associated entrances.

[0041] In a possible design of the first aspect, the first interface is a camera viewfinder interface, and the first triggering operation includes a shooting operation. That is, the shooting operation can trigger the electronic device to recognize and mark text entities in the interface.

[0042] In a possible design of the first aspect, before adding a first text tag to the first text entity in response to a first trigger operation on the first interface, the above method further includes: displaying an identification control in the first interface when the first interface satisfies the second condition. The second condition includes: the first interface is a non-blank interface, the first interface includes text entities and / or object entities, and the first trigger operation includes a trigger operation on the identification control. That is to say, when the first interface includes useful information such as text and objects, the electronic device will actively push the identification control to identify and mark the text entity in the first interface. In this way, the electronic device can realize the function of intelligently pushing identification and marking text entities (such as the intelligent recognition function described below).

[0043] In one possible design of the first aspect, displaying the recognition control on the first interface when the first interface satisfies the second condition includes: displaying the recognition control on the first interface in response to a fifth triggering operation (e.g., a two-finger press operation) performed by the user on the first interface when the first interface satisfies the second condition. The user performing the fifth operation on the first interface indicates that the user wants to recognize and mark the text entity.

[0044] That is to say, the electronic device can include useful information such as text and objects in the first interface, and only push the function of identifying and marking text entities when the user wants to identify and mark the text entity, so as to accurately meet the user's needs.

[0045] In a possible design of the first aspect, the method further includes:

[0046] In response to the user's fifth trigger operation on the first interface, the fifth trigger operation is used to trigger the electronic device to analyze the content of the first interface to determine the recommended functions for the first interface. Several typical contents and their matching functions are shown below:

[0047] If the number of foreign languages ​​in the first interface exceeds a first number, a translation control is displayed in the first interface. If the number of foreign languages ​​in the first interface does not exceed the first number, the translation control is not displayed, and the translation control is used to trigger the electronic device to translate the foreign languages ​​in the first interface. In this way, the electronic device can recommend the translation function for interfaces with a large number of foreign languages ​​to facilitate translation of foreign languages ​​in the interface.

[0048] If the first interface includes private information, a privacy protection control is displayed on the first interface. If the first interface does not include private information, the privacy protection control is not displayed. The privacy protection control is used to trigger the electronic device to block the private information in the first interface. In this way, the electronic device can recommend a privacy protection function for the interface including private information to protect the private information in the interface.

[0049] If the first interface includes text but no entity, a selection control is displayed on the first interface. If the first interface does not include text or includes an entity, the selection control is not displayed, and the selection control is used to trigger the electronic device to select the text in the first interface. In this way, the electronic device can recommend a text selection function for interfaces containing ordinary text, so that operations such as copying and cutting can be performed on the text in the interface.

[0050] In all the above descriptions of the possible design methods of the first aspect, the first interface is mainly used for description. It is understandable that in various possible design methods, the specific implementation of the second aspect is similar to that of the first interface, and this article will not go into details about this.

[0051] In a second aspect, the present application further provides an electronic device comprising a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is configured to store computer program code, which comprises computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method of the first aspect and any possible design thereof.

[0052] In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as in the first aspect and any possible design method thereof.

[0053] In a fourth aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible design thereof.

[0054] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method according to the first aspect and any possible design thereof.

[0055] It can be understood that the beneficial effects that can be achieved by the electronic device of the second aspect, the chip system of the third aspect, the computer storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] FIG1 is a hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0057] FIG2 is one of the mobile phone interface diagrams provided in an embodiment of the present application;

[0058] FIG3 is a second mobile phone interface diagram provided in an embodiment of the present application;

[0059] FIG4 is a third mobile phone interface diagram provided in an embodiment of the present application;

[0060] FIG5 is a fourth mobile phone interface diagram provided in an embodiment of the present application;

[0061] FIG6 is a fifth mobile phone interface diagram provided in an embodiment of the present application;

[0062] FIG7 is a sixth mobile phone interface diagram provided in an embodiment of the present application;

[0063] FIG8 is a seventh mobile phone interface diagram provided in an embodiment of the present application;

[0064] FIG9A is an eighth mobile phone interface diagram provided in an embodiment of the present application;

[0065] FIG9B is a ninth mobile phone interface diagram provided in an embodiment of the present application;

[0066] FIG10 is a tenth diagram of a mobile phone interface provided in an embodiment of the present application;

[0067] FIG11 is an eleventh mobile phone interface diagram provided in an embodiment of the present application;

[0068] FIG12 is a twelfth mobile phone interface diagram provided in an embodiment of the present application;

[0069] FIG13A-FIG13B is a thirteenth mobile phone interface diagram provided in an embodiment of the present application;

[0070] FIG14 is a fourteenth diagram of a mobile phone interface provided in an embodiment of the present application;

[0071] FIG15 is a fifteenth mobile phone interface diagram provided in an embodiment of the present application;

[0072] FIG16 is a sixteenth diagram of a mobile phone interface provided in an embodiment of the present application;

[0073] FIG17 is a seventeenth mobile phone interface diagram provided in an embodiment of the present application;

[0074] FIG18 is the eighteenth mobile phone interface diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0076] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0077] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0078] The screen recognition method provided in the embodiment of the present application enables the electronic device to recognize the content in the user interface, such as text, objects, graphic codes (including barcodes, QR codes, etc.), and mark part of the content in the user interface based on the recognition results, thereby providing users with marks for quickly obtaining information from the user interface, which is conducive to improving the efficiency of human-computer interaction.

[0079] Taking a picture displayed in the user interface as an example, the electronic device can identify the text, objects and graphic codes included in the picture, and based on the identified objects, mark some of the text and graphic codes in the picture, such as marking text and graphic codes that are highly correlated with the objects.

[0080] For example, the electronic device may be a mobile phone, tablet, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook computer, as well as a cellular phone, personal digital assistant (PDA), augmented reality (AR) or virtual reality (VR) device, etc., which has a camera. The embodiments of the present application do not impose any particular restrictions on the specific form of the electronic device.

[0081] 1 , the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc.

[0082] It is understood that the structures illustrated in the embodiments of the present invention do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0083] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0084] In some embodiments, the electronic device may complete the identification method through the processor 110 to obtain an identification result.

[0085] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0086] The electronic device implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0087] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device can display the recognition result through the display screen 194.

[0088] The electronic device can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. In some embodiments, the electronic device can capture images through the camera 193 for recognition.

[0089] The electronic device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0090] The button 190 may include a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The mobile phone may receive the button input and generate key signal input related to the user settings and function control of the mobile phone. The motor 191 may generate a vibration prompt. The motor 191 may be used for incoming call vibration prompts or for touch vibration feedback. The indicator 192 may be an indicator light that may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card may be connected to or separated from the mobile phone by inserting it into or removing it from the SIM card interface 195.

[0091] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture, which is not specifically limited in the present embodiment.

[0092] The screen recognition method provided in the embodiment of the present application can be implemented in the above-mentioned electronic device. The following takes the electronic device being a mobile phone as an example to illustrate the screen recognition method provided in the embodiment of the present application.

[0093] The phone provides an application (Application, APP) specifically for identifying content in the user interface, referred to herein as the Smart Vision APP. The Smart Vision APP can be a system-level APP. The user interface includes the application interfaces of various applications, such as the lock screen interface, desktop interface, gallery interface, chat interface, video playback interface, etc.

[0094] In particular, the app interface also includes the camera app's viewfinder interface, which means the Smart Vision app can also be used to identify content in the viewfinder.

[0095] There are several ways to open the Smart Vision APP on your mobile phone, as shown in Method 1 to Method 4 below.

[0096] Method 1: The mobile phone interface provides an opening entrance for the Smart Vision APP.

[0097] The mobile phone provides an opening entrance for the Smart Vision APP. In response to a trigger operation on the opening entrance, such as a click operation, the mobile phone can open the Smart Vision APP for identification. The embodiment of the present application does not specifically limit the form and position of the opening entrance. This article only lists several position examples as shown in Figure 2 below:

[0098] Example 1: The opening entrance is located on the lock screen interface.

[0099] On the lock screen of a mobile phone, in response to a user pulling up from the bottom of the lock screen, the phone can display portals to various functions provided by the phone. For example, in response to a pull-up operation, the phone can display lock screen 201 shown in Figure 2. Lock screen 201 includes portals to shortcut functions such as a recorder, calculator, and compass. In addition, lock screen 201 also includes a portal 200 for opening the Smart Vision app.

[0100] Example 2: The opening entrance is located in the drop-down control center.

[0101] When the phone screen is on, in response to the user pulling down from the top of the display screen, the phone can display the pull-down control center interface 202 shown in Figure 2. The pull-down control center interface 202 also includes entrances to various shortcut functions provided by the phone, including an opening entrance 200 for the Smart Vision APP.

[0102] Example 3: The opening entrance is located in the global search interface.

[0103] When the mobile phone displays the desktop (including the home screen and the negative first screen), in response to the user sliding down from the middle area of ​​the desktop, the mobile phone can display the global search interface 203 shown in Figure 2. The global search interface 203 can be used to search for information in the mobile phone. For example, the global search interface 203 includes a search box 2031. In response to the user entering search text in the search box, the mobile phone can search for information including the searched text in the mobile phone, such as chats, files, text messages, applications, etc. The search box 2031 includes the opening entrance 200 of the Smart Vision APP.

[0104] Example 4: The opening entrance is located at the negative one screen.

[0105] The mobile phone can display the negative one screen 204 shown in Figure 2, and the negative one screen 204 includes a search box 2041, and the search box 2041 includes an opening entrance 200 of the Smart Vision APP.

[0106] Example 5: The opening entrance is located in the application interface of the camera application.

[0107] After the mobile phone opens the camera application, the mobile phone can display the application interface 205 of the camera application shown in Figure 2. The application interface 205 includes an opening entrance 200 of the smart vision APP.

[0108] Method 2: In response to voice wake-up, the phone opens the Smart Vision APP.

[0109] Users can also trigger the phone to open the Smart Vision APP by voice wake-up. Specifically, after the phone's voice assistant is woken up, in response to the user's voice input to open the Smart Vision APP, the phone can open the Smart Vision APP for recognition.

[0110] Taking the voice command "Use Smart Vision" to open the Smart Vision app as an example, after the user inputs the voice command "Use Smart Vision", the mobile phone can recognize "Use Smart Vision" and display the interface 301 shown in Figure 3, which includes the voice result "Use Smart Vision" 3011. Subsequently, the mobile phone can open the Smart Vision app.

[0111] After triggering the Smart Vision app through method 1 or method 2 above, the phone can display the Smart Vision app's application interface. For example, the phone can display interface 401 shown in Figure 4, which is the application interface of the Smart Vision app. Interface 401 includes a viewfinder area 4010 for displaying the camera's viewfinder image.

[0112] Furthermore, the Smart Vision APP provides a variety of recognition functions. For example, the interface 401 shown in Figure 4 includes a variety of recognition function options, such as "Text Extraction" option 4011, "Smart Recognition" option 4012, "Scan Code" option 4013 (usually the default selected option), and "Cutout" option 4014. In response to the user's sliding operation from right to left in the function option area 4015 (shown in the dotted box in the figure, the dotted box does not actually exist), the "Translation" option 4016, "Scan Document" option 4017, "Scan Card" option 4018, "Test Paper / Homework" option 4019, etc. can also be presented in the interface 401.

[0113] In response to a user selecting any function option, the electronic device can switch to the corresponding recognition function to achieve recognition. For example, in response to a user clicking on the "Smart Recognition" option 4012 in the interface 401 shown in FIG4 , the mobile phone can switch from the default selected code scanning function to the smart recognition function, such as displaying the interface 402 shown in FIG4 , in which the "Smart Recognition" 4012 is currently selected.

[0114] The text extraction function is used to identify and extract text from the user interface and can also be used to mark text entities. Text entities include identification document numbers (such as ID numbers and passport numbers), addresses, phone numbers, flight information, express delivery numbers, email addresses, links, and more. For example, if a phone number is identified in an image, it will be marked as a phone number.

[0115] Intelligent Recognition: This function integrates the aforementioned text extraction functionality. It can also be used to identify objects in the user interface and label them. These include animals, plants, buildings, and food. For example, if an animal is identified in an image, an animal label is added to the corresponding area. This means that the intelligent recognition function can recognize and label text, as well as identify, crop, and label objects.

[0116] In addition, the intelligent recognition function can also recognize and mark graphic codes, including barcodes, QR codes, etc.

[0117] The translation feature can be used to recognize text in the user interface and then translate it.

[0118] After triggering the opening of the Smart Vision APP through the above-mentioned method 1 or method 2, the objects identified by the corresponding recognition function include: pictures taken under the corresponding recognition function, or pictures selected from the gallery. The embodiment of this application does not make specific limitations on this.

[0119] For example, in response to the user clicking the shooting button 4020 in the interface 402 shown in Figure 4 (also referred to as the first trigger operation or the second trigger operation), the mobile phone can take a picture and use the currently selected recognition function (i.e., the intelligent recognition function) to identify the captured picture to obtain a recognition result.

[0120] As another example, in response to the user clicking on the gallery entry 4021 in the interface 402, the mobile phone can provide pictures in the gallery for the user to select, such as the interface 403 shown in Figure 4, the interface 403 includes thumbnails of multiple pictures in the gallery, and in response to the user selecting any thumbnail, such as thumbnail 4031, the mobile phone can use the currently selected recognition function to recognize the picture corresponding to the selected thumbnail to obtain a recognition result.

[0121] Furthermore, after the Smart Vision app is triggered and launched through the above-mentioned method 1 or method 2, the objects recognized by the corresponding recognition function also include: the content that is framed by the mobile phone but not yet captured under the corresponding recognition function, that is, the content of the framing interface. For example, the recognized object can be the content corresponding to the framing screen 4022 displayed in the framing interface 402 shown in Figure 4. In this case, the mobile phone can recognize the content of the framing interface through augmented reality (AR) recognition technology to obtain a recognition result.

[0122] Method 3: After launching the Gallery app or displaying a picture in the Gallery app, the phone automatically launches the Smart Vision app.

[0123] The phone can also automatically start the Smart Vision APP to analyze the pictures in the Gallery App after starting the Gallery App or displaying the pictures in the Gallery App, and recommend the intelligent recognition function in the Smart Vision APP based on the analysis results.

[0124] In some embodiments, after launching the gallery application, the phone can launch the Smart Vision app, which analyzes the images in the gallery and identifies target images. Target images can include at least one of the following: non-blank images, images containing objects, images containing text, and images containing graphic codes. In other words, target images typically contain useful information such as text, objects, and graphic codes. Images that are not target images do not contain this useful information.

[0125] In response to a user viewing a target image, such as clicking on a thumbnail corresponding to the target image, the phone can display the target image and provide access to a smart recognition function. This smart recognition function can be used to trigger the phone to identify and tag content such as text, objects, or graphic codes within the target image. In this way, in response to a viewing operation, the phone can quickly recommend the smart recognition function and provide the user with access to identify and tag content within the image.

[0126] Exemplarily, after starting the gallery application, the mobile phone can display interface 501 shown in Figure 5, which is the application interface of the gallery application. Interface 501 includes thumbnails of multiple pictures in the gallery, such as thumbnail 5011. Taking the target picture as the picture corresponding to thumbnail 5011 as an example, in response to the user clicking on thumbnail 5011, the mobile phone can display interface 502 shown in Figure 5. Interface 502 includes picture 5021 corresponding to thumbnail 5011, and also includes "Smart Vision" 5022. "Smart Vision" 5022 is the entrance to the intelligent recognition function. That is, the mobile phone recognizes picture 5021 as the target picture.

[0127] Each time the gallery is launched, the phone can only analyze the images added between the two gallery launches, while other images can refer to the historical recognition results. This way, the phone can only analyze a small number of images each time the gallery is launched, thus reducing phone power consumption.

[0128] In other embodiments, after receiving a user's request to view an image in the gallery application, the mobile phone can launch the Smart Vision app and use it to analyze the currently viewed image to determine whether it is the target image. In this way, the mobile phone can only analyze the currently viewed image, thereby reducing power consumption during each analysis.

[0129] Of course, the timing of triggering the mobile phone to automatically start the Smart Vision APP in the above-mentioned method 3 is only a few typical situations, and is not actually limited to this. For example, the mobile phone can also automatically start the Smart Vision APP to analyze the pictures in the gallery when charging or at a preset time (such as in the early morning) to determine the target icon. On this basis, after starting the gallery application, the mobile phone can start the Smart Vision APP to analyze the pictures that have not been analyzed, that is, the pictures newly added between the last analysis and the current startup of the gallery application; or, on this basis, the mobile phone will only start the Smart Vision APP to analyze the currently viewed picture after receiving the user's viewing operation for the picture that has not been parsed in the gallery application.

[0130] If the currently viewed image is the target image, the phone will provide access to the smart recognition function. If the currently viewed image is not the target image, the phone will not provide access to the smart recognition function. For example, in interface 501 shown in Figure 5 , where thumbnail 5012 is the second thumbnail, in response to a click on thumbnail 5012, the phone will display interface 504 shown in Figure 5 . Interface 504 includes image 5041 corresponding to thumbnail 5012. However, interface 504 does not include access to the smart recognition function. In other words, interface 504 is the fourth interface.

[0131] After automatically starting the Smart Vision APP and providing the entrance to the smart recognition function using the above-mentioned method three, if the display time of the entrance to the smart recognition function reaches time 1, the mobile phone can retract the entrance to the smart recognition function to the edge of the user interface to avoid affecting the user's viewing of pictures. Taking time 1 as an example of 5s, after the display time of "Smart Vision" 5022 in the interface 502 shown in Figure 5 reaches 5s, the mobile phone can display the interface 503 shown in Figure 5. Interface 503 also includes "Smart Vision" 5022. However, unlike interface 502, "Smart Vision" 5022 in interface 503 is displayed at the edge of interface 503. It should be noted that the entrance to the smart recognition function after retraction has the same function as the entrance to the expanded smart recognition function, and both can trigger the mobile phone to use the smart recognition function to identify the target picture, which will not be repeated here.

[0132] Furthermore, during a usage process from starting the gallery process to closing the gallery process, after receiving the user's trigger operation on the entrance of the recommended smart recognition function, even if the mobile phone responds to the user's viewing operation on the target picture again, the mobile phone may no longer display the expanded smart recognition function entrance, but directly display the collapsed smart recognition function entrance.

[0133] After automatically launching the Smart Vision APP and providing access to the smart recognition function using the third method described above, in response to a user triggering the access to the smart recognition function (which may be referred to as a first triggering operation or a second triggering operation), such as a click operation, the mobile phone may use the smart recognition function to identify the target image and obtain a recognition result. For example, in response to a user clicking on "Smart Vision" 5022 in the interface 502 shown in FIG. 5 , the mobile phone may use the smart recognition function provided by the Smart Vision APP to identify image 5021 and obtain a recognition result.

[0134] Method 4: In response to the user's operation 1 (also called the fifth trigger operation), the mobile phone starts the Smart Vision APP.

[0135] The mobile phone can also start the Smart Vision APP to analyze the content of the current interface (which can be called the first interface or the second interface) after receiving the user's operation 1 on the current interface (which can be called the first interface or the second interface), and recommend processing functions based on the analysis results. In other words, in method 4, the mobile phone can be triggered by operation 1 to start the Smart Vision APP to analyze the corresponding needs and make recommendations.

[0136] Among them, operation 1 can be a long press operation, a sliding operation, a two-finger press operation, etc. The embodiment of the present application does not make specific limitations on this. The following mainly takes operation 1 being a two-finger press operation as an example for explanation.

[0137] The current interface can be any user interface displayed during the use of the mobile phone. For example, the current interface can be an application interface of a social application (such as a chat application, a life sharing application, etc.), a picture viewing interface, etc. The embodiment of the present application does not specifically limit this. That is to say, in method 4, the mobile phone can not only intelligently recommend processing functions for pictures in the gallery application, but also recommend processing functions for other interfaces.

[0138] At this point, it should be noted that: the current interface is a picture viewing interface, and the mobile phone can directly use the picture as the content of the current interface for the smart vision APP to analyze. However, if the current interface is not a picture viewing interface, it is usually difficult for the mobile phone to directly obtain the content of the current interface. Based on this, in a specific implementation method, in response to the user's two-finger press operation on the current interface, the mobile phone can take a screenshot of the current interface and start the smart vision APP to analyze the screenshot. That is, the mobile phone can obtain the content of the current interface by taking a screenshot and the user's smart vision APP analyzes it. Accordingly, the smart vision APP analyzes the screenshot, which is equivalent to analyzing the content of the current interface.

[0139] In another specific implementation, in response to a user performing a two-finger press operation on the current interface, and if the current interface is a picture viewing interface, the mobile phone can launch the Smart Vision app to analyze the currently viewed image. In response to a user performing a two-finger press operation on the current interface, and if the current interface is not a picture viewing interface, the mobile phone can take a screenshot of the current interface and launch the Smart Vision app to analyze the screenshot. In this way, the mobile phone can take targeted screenshots for analysis by the Smart Vision app.

[0140] In the following, the example in which a mobile phone takes a screenshot of the current interface and starts the Smart Vision APP to analyze the screenshot in response to a two-finger press operation on the current interface is mainly used for explanation. The following content of this application can also be a technical solution after parsing the image or interface content after starting the Smart Vision APP using Methods 1 to 3.

[0141] The processing function can be a recognition function provided by the Smart Vision app, such as translation, intelligent recognition, or text extraction. Alternatively, the processing function can be another function, such as a privacy protection function. The privacy protection function is used to identify and block private information to protect it.

[0142] In some embodiments, after the Smart Vision app analyzes and determines that the number of foreign words in the current interface reaches a threshold (also referred to as a first number), the mobile phone may provide an entry to a translation function (also referred to as a translation control). In this way, the mobile phone can recommend the translation function in the Smart Vision app for scenarios requiring translation.

[0143] The quantity threshold may be a fixed quantity, or the quantity of text included in the current interface (i.e., the interface is entirely in foreign languages), or a fixed ratio (e.g., 90%, 80%, etc.) of the quantity of text included in the current interface. This embodiment of the present application does not specifically limit this.

[0144] Taking the example of the quantity threshold being the amount of text included in the current interface, the mobile phone may display interface 601 shown in FIG6 , which is entirely in English. In response to the user's two-finger press operation on interface 601, the mobile phone may take a screenshot of interface 601 and launch the Smart Vision app to analyze the screenshot. After parsing that the text included in the screenshot is entirely in a foreign language (different from the language set by the system), the mobile phone may display interface 602 shown in FIG6 . Different from interface 601, interface 602 includes "Translate" 6021. "Translate" 6021 is the entry point for the translation function.

[0145] On the other hand, if the Smart Vision app determines that the number of foreign languages ​​in the current interface does not reach the threshold, the phone will not provide access to the translation function. In other words, the phone does not recommend the translation function for every interface when you press two fingers, but instead recommends it dynamically based on the number of foreign languages.

[0146] In other embodiments, after the Smart Vision APP analyzes the current interface and finds that it contains private information, the mobile phone can provide an entry to the privacy protection function (also called a privacy protection control). In this way, the mobile phone can recommend privacy protection functions for scenarios where privacy protection is required.

[0147] Among them, private information includes address, phone number, email address, ID number / picture for identity identification, etc.

[0148] For example, a mobile phone may display interface 701 shown in FIG7 , which is the application interface of a chat application. In response to a user's two-finger press on interface 701, the mobile phone may take a screenshot of interface 701 and launch the Smart Vision app to analyze the screenshot. After analyzing the screenshot to find private information such as an address and email address, the mobile phone may display interface 702 shown in FIG7 . Unlike interface 701 , interface 702 includes "Privacy Code" 7021 . "Privacy Code" 7021 is the entry point for the privacy protection function.

[0149] Conversely, if the Smart Vision app determines that the current interface does not contain private information, the phone will not provide access to the privacy protection function. In other words, the phone does not recommend the privacy protection function for every interface you perform a two-finger press. Instead, it will dynamically recommend the privacy protection function based on whether the current interface contains private information.

[0150] In other embodiments, after the Smart Vision App parses and determines that the current interface includes text but does not include entities (including text entities and object entities), the mobile phone may provide an entry for a selection function (also known as a selection control). It should be noted that the mobile phone can only select text after extracting it, so the selection function can be understood as a subfunction of the aforementioned text extraction function.

[0151] It is understandable that if there is no entity in the current interface, it means that the user has no need to perform an operation on the entity, and the mobile phone can directly provide an entry for the selection function. In this way, the mobile phone can quickly perform a selection operation on the text in the current interface for subsequent copying, cutting or sharing of the text.

[0152] For example, the mobile phone can display interface 801 shown in Figure 8, which includes picture 8011. Picture 8011 includes text but no entity. In response to the user's two-finger press operation on interface 801, the mobile phone can take a screenshot of interface 801 and start the Smart Vision APP to parse the screenshot. After parsing the screenshot and finding that it includes text but no entity, the mobile phone can display interface 802 shown in Figure 8. The difference from interface 801 is that interface 802 includes "Select All" 8021. "Select All" 8021 is the entrance to the selection function.

[0153] Conversely, if the Smart Vision app analyzes the current interface and determines that it contains no text, or contains both text and entities, the phone will not provide an entry for selecting a function. In other words, the phone does not recommend a selection function for every two-finger press on each interface; instead, it dynamically recommends a function based on whether the current interface contains text and / or entities.

[0154] The above-mentioned solution of the recommendation selection function is particularly suitable for some interfaces that cannot perform copy operations, such as the above-mentioned interface 801, or some application interfaces that restrict users from copying, so that users can complete text operations in these interfaces.

[0155] As can be seen, using method 4, for different interfaces, in response to the user's two-finger press operation on the interface, the phone can use the Smart Vision app to analyze the user interface and recommend different recognition functions. For example, if the current interface is interface 2, in response to the user's two-finger press operation on interface 2, the phone can recommend processing function 1; if the current interface is interface 3, in response to the user's two-finger press operation on interface 3, the phone can recommend processing function 2. Processing function 1 and processing function 2 are different processing functions.

[0156] Furthermore, the same interface may meet the recommendation criteria for different processing functions, such as meeting the recommendation criteria for both the translation function and the selection function. In this case, the mobile phone can simultaneously recommend all processing functions that meet the criteria; alternatively, the mobile phone can configure the matching order between different (types of) interfaces and processing functions, and the mobile phone can only recommend the processing function that has the highest degree of match with the current interface. This embodiment of the application does not specifically limit this.

[0157] For example, for a user interface that does not allow long press to copy, the mobile phone can be configured with the highest matching degree of the selection function to facilitate text operations in the user interface; for the application interface of the chat application, the mobile phone can be configured with the highest matching degree of the privacy protection function to encode the private information involved in the chat content before sending it; and for foreign language websites, the mobile phone can be configured with the highest matching degree of the translation function to facilitate foreign language translation.

[0158] After automatically starting the Smart Vision APP and providing the corresponding processing function entrance using the aforementioned method 4, in response to the user's trigger operation on the processing function entrance (which can be called the first trigger operation or the second trigger operation), such as a click operation, the mobile phone can use the recommended processing function to process the current interface.

[0159] At this point, it should be noted that the conditions for triggering the launch of the Smart Vision APP in the above-mentioned methods three and four can also be interchanged. That is, launching the gallery or receiving an operation to view pictures in the gallery in method three can be interchanged with the double-finger press operation in method four. Exemplarily, in response to launching the gallery or receiving an operation to view pictures in the gallery, the mobile phone can launch the Smart Vision APP to analyze whether the conditions for recommending various processing functions in the above-mentioned method four are met. If so, the corresponding processing functions are recommended. Another exemplary example is that in response to a double-finger press operation, the mobile phone can launch the Smart Vision APP and analyze whether the conditions for recommending the intelligent recognition function in the above-mentioned method three are met. If so, the intelligent recognition function is recommended.

[0160] After going through the aforementioned methods one to four, the mobile phone can use the corresponding functions (such as the various recognition functions provided by the Smart Vision APP selected in method one and method two, or the intelligent recognition function provided by the Smart Vision APP recommended in method three, and the various processing functions recommended in method four) to perform processing.

[0161] It should be noted that when the mobile phone uses the corresponding function to perform processing, it needs to identify the text, objects, graphic codes, etc. in the current interface and perform various operations such as marking, selecting, and translating, which will change the information in the current interface. Based on this, in a specific implementation method, in response to the user triggering the operation of using the corresponding function to perform processing, such as the click operation on the shooting button 4020 and the selection operation on the thumbnail 4031 in method 1 and method 2, the click operation on the "Smart Vision" 5022 in method 3, and the triggering operation of the entrance of various recommended processing functions in method 4, the mobile phone can first take a screenshot of the current interface and display the screenshot image, and then identify the screenshot image, and then perform various operations such as marking, selecting, and translating. In this way, the mobile phone can perform operations on the screenshot image without affecting the current interface. It should be noted that when browsing large images in the gallery and identifying and processing the image content, the mobile phone does not need to perform screenshot processing.

[0162] The following describes the intelligent recognition function, text extraction function, translation function, and privacy protection function. It should be noted that in the following, the process of the mobile phone taking a screenshot and performing an operation on the screenshot in response to the user triggering the corresponding function to perform processing will not be repeated one by one.

[0163] First, intelligent recognition function.

[0164] The mobile phone uses intelligent recognition function to identify and mark the physical objects.

[0165] Taking the example of triggering the execution of processing using the smart recognition function through the aforementioned method 1, after selecting the function option of the smart recognition function, the mobile phone can display interface 901 shown in Figure 9A. In response to the user clicking the capture button 9011 in interface 901, the mobile phone can display interface 902 shown in Figure 9A. Interface 902 is the recognition result page (also referred to as the first interface). In the embodiment of the present application, in response to the user clicking the capture button in interface 901 corresponding to the smart recognition function, the mobile phone can take a photo, but the photo result is not saved. The content displayed in interface 902 is the cached photo result. If the user clicks the return control in interface 902 and does not save the photo result, the mobile phone will not store the photo result. Alternatively, if the user continues to edit the content of interface 902 and chooses to save, the mobile phone can also save the corresponding result. Interface 902 uses circles, animal icons, and other symbols to mark various object entities. For example, dog 9021 is marked with animal icon 90211, building 9022 is marked with circle 90221, and plant 9023 is marked with circle 90231.

[0166] In the recognition results, the mobile phone may highlight the item entity with the largest area and / or the most accurate recognition as the focus item entity (herein, a bold line is used to indicate the highlighting effect, but this is not a limitation), and mark the focus item entity with a category icon. For example, in interface 902, the recognition result for dog 9021 is the most accurate, so dog 9021 is highlighted in interface 902; and an animal icon 90211 is marked in the upper left corner of dog 9021, clearly indicating that the current focus item entity is an animal, i.e., animal icon 90211 is a category icon.

[0167] Other object entities in the recognition results are not highlighted and are marked with common symbols. For example, building 9022 and plant 9023 in interface 902 are not highlighted and are marked with circles, with circle 90221 marking building 9022 and circle 90231 marking plant 9023.

[0168] After displaying the recognition result, in response to a focus switching operation (which may be referred to as a fourth triggering operation), the mobile phone may switch the focused item entity. That is, the focus is switched from the current focused item entity (which may be referred to as the first item entity) to another item entity (which may be referred to as the third item entity). Similarly, the switched focused item entity is highlighted and marked with a category icon.

[0169] In some embodiments, the focus switching operation may be a triggering operation, such as a click operation, on a mark corresponding to a target item entity other than the current focus item entity (such as the first item mark of the third item entity).

[0170] For example, if the current focused item entity is dog 9021 in interface 902 shown in FIG. 9A , and the target item entity is building 9022 in interface 902 shown in FIG. 9A , in response to a user clicking circle 90221 in interface 902 , the mobile phone may display interface 903 shown in FIG. This interface 903 differs from interface 902 in that building 9022 is highlighted, and a building icon 9031 is marked in the upper left corner of building 9022, clearly indicating that the focused item entity after the switch is a building. That is, building icon 9031 is a category icon. Furthermore, in interface 903 , dog 9021 is no longer highlighted, and is still marked with a circle, such as circle 9032 . In other words, the focused item entity has switched from dog 9021 to building 9022 .

[0171] In other embodiments, the operation of switching focus includes a click operation on a mark corresponding to a target item entity other than the current focus item entity, and a click operation on the current focus item entity.

[0172] For example, if the current focused item entity is dog 9021 in interface 902 shown in FIG. 9B , and the target item entity is building 9022 in interface 902 shown in FIG. 9B , in response to a user clicking circle 90221 in interface 902 , the mobile phone may display interface 905 shown in FIG. 9B . Unlike interface 902 , interface 905 highlights not only dog ​​9021 but also building 9022 . Subsequently, in response to a user clicking dog 9021 in interface 905 , the mobile phone may display interface 903 shown in FIG. Unlike interface 905 , interface 903 no longer highlights dog 9021, but only highlights building 9022. This also allows the focus to be switched from dog 9021 to building 9022.

[0173] Furthermore, in this embodiment, in response to a user clicking on building 9022 in interface 903 shown in FIG9B , the mobile phone may display interface 906 shown in FIG9B , further removing the highlighting of building 9022. In other words, in response to a user clicking on a focused item entity, the highlighting of the focused item entity may be removed, i.e., the focused item entity may be switched to a non-focused state.

[0174] In response to a user triggering an action, such as a click, on the category icon of a focused item, the mobile phone may display introductory information about the focused item to assist the user in understanding the focused item. For example, in response to a user clicking building icon 9031 in interface 903 shown in FIG. 9A , the mobile phone may display interface 904 shown in FIG. This interface differs from interface 903 in that interface 904 includes a pop-up window 9041 containing introductory information about building 9022.

[0175] The mobile phone can also display quick entrances around the focus item entity to implement quick operations on the focus item entity. For ease of explanation, the quick entrance to the first item entity can be called the first quick entrance, and the quick entrance to the third item entity can be called the second quick entrance.

[0176] Among them, the quick entrance can be fixed, such as a search entrance, a purchase entrance, a copy entrance, a save entrance, a share entrance, etc.

[0177] Alternatively, the quick access points can be different for different focused items. For example, for buildings, the phone can provide a search entry; for commodities, the phone can provide a purchase entry and a price comparison entry. This way, the phone can provide targeted quick access points to precisely meet user needs.

[0178] Taking the focused item entity being the building 9022 in the interface 903 shown in FIG. 9A as an example, the following quick access entries are displayed at the bottom edge of the building 9022 : “Search” 9033 , “Save” 9034 , and “Share” 9035 .

[0179] In response to a user triggering a quick entry, such as a click, the phone can perform a corresponding quick action for the focused item. Specifically, in response to a user clicking a search entry, the phone can search the network for information related to the focused item and display it. For example, in response to a user clicking "Search" 9033 in interface 903, the phone can search for information related to building 9022. For example, after the search is complete, the phone can display interface 904 shown in FIG. The introductory information in pop-up window 9041 of interface 904 is the result of the search.

[0180] Furthermore, in response to a user's movement operation on the focused item entity, such as a long press followed by a drag operation, the mobile phone can move the display position of the focused item entity. It should be noted that after the display position of the focused item entity is moved, the mobile phone can still display the focused item entity at its initial position. Of course, the mobile phone can also display the focused item entity at a different position than the initial position, and this embodiment of the present application is not specifically limited to this.

[0181] Taking the dog 9021 in the interface 902 shown in FIG9A as an example, in response to the user long pressing and dragging the dog 9021, the mobile phone may display the interface 1001 shown in FIG10. The difference from the interface 902 is that the position of the dog 9021 in the interface 1001 has changed.

[0182] In response to the focus item entity's display position reaching the target area in the user interface after movement, the phone displays shortcut icons (also referred to as associated portals) for multiple associated applications / functions / services, facilitating quick execution of associated application / function-related processing for the focus item entity. For ease of explanation, the interface displaying shortcut icons may be referred to as the fourth interface. The following description primarily uses associated applications as an example, and accordingly, the shortcut icons are application icons.

[0183] The target area may be an edge area of ​​the user interface, such as a left edge or a right edge.

[0184] Among them, the associated applications can be fixed, such as chat applications, search applications, shopping applications, price comparison applications, sharing applications, collection applications, printing applications, recipe applications, cute pet applications, etc. Alternatively, corresponding to different focus item entities, the associated applications can be different, that is, they can include different shortcut icons. For example, for goods, the associated applications may include purchase applications and price comparison applications; for food, the associated applications may include recipe applications; for animals, the associated applications may include cute pet applications. Associated functions or services can also be displayed in a similar manner and will not be repeated here.

[0185] Continuing with FIG10 , as the user continues to move dog 9021 in interface 1001, dog 9021 can be moved to area 10021 (indicated by a dotted box in the figure, but not actually present) in interface 1002 shown in FIG10 , with area 10021 being the target area. Interface 1002 also includes shortcut icons for related applications, such as a favorites app 10022, a search app 10023, a pet app 10024, a sharing service 10025, and a chat app 10026.

[0186] In a specific implementation, the mobile phone may also display a dynamic effect when the focus object entity is moved. For example, the mobile phone may display a dynamic effect of the image in interface 1001 as shown in FIG10 occupying the entire user interface, and the image in interface 1002 forming a "door".

[0187] In a specific implementation, after the focused object entity is moved into the target area, the mobile phone can shrink the focused object entity. For example, the dog 9021 in the interface 1002 is much smaller than the dog 9021 in the interface 1001.

[0188] After displaying the shortcut icon of the associated application, in response to the focus item entity being moved to the position of the target shortcut icon (also called the first associated entry), the mobile phone can display interface 1 (also called the fifth interface) of the target associated application (also called the first application). The target associated application is an associated application corresponding to the target shortcut icon among multiple associated applications, and interface 1 includes information about the focus item entity.

[0189] For example, if the target shortcut icon is shortcut icon 10024 in interface 1002 shown in FIG10 , that is, the target associated application is the Cute Pet application, and interface 1 is the application icon of the Cute Pet application, as the user continues to move dog 9021 in interface 1001, the mobile phone may display interface 1003 shown in FIG10 , in which dog 9021 is moved to the position of shortcut icon 10024. In response to dog 9021 being moved to the position of shortcut icon 10024, the mobile phone may display interface 1004 shown in FIG10 , which includes a floating window 10041 displaying the application interface of the Cute Pet application, which includes information related to dog 9021.

[0190] At this point, it should be noted that in the specific implementation of the recommended shortcut icon described above, the user must first drag the focused item entity to the target area before the shortcut icon is displayed. In practice, this is not a limitation. For example, in response to the user moving the focused item entity and the moving distance exceeds a distance threshold, the phone can display the shortcut icon. For another example, in response to the user moving the focused item entity, the phone can display the shortcut icon.

[0191] The above description of the intelligent recognition function primarily addresses the recognition of object entities, such as dog 9021, building 9022, and plant 9023, and the subsequent processing based on the recognition results. However, as previously explained, the intelligent recognition function can also be used for text recognition. For details on the intelligent recognition function's features for text recognition, please refer to the description of the text extraction function below. We will not elaborate on this here.

[0192] Second, text extraction function.

[0193] The mobile phone uses a text extraction function that can recognize and extract text and can also mark text entities.

[0194] Taking the example of triggering the text extraction function through the aforementioned method 1, after selecting the text extraction function option, the mobile phone may display interface 1101 shown in Figure 11. In response to the user clicking the capture button 11011 in interface 1101, the mobile phone may display interface 1102 shown in Figure 11, which is the recognition results page. Interface 1102 shows address 1 with a location icon 11021, website 1 with a network icon 11022, and phone 2 with a phone icon 11023.

[0195] In the recognition results, the phone can mark all text entities. Alternatively, the phone can mark only some text entities to avoid labeling confusion.

[0196] In a specific implementation, for text entities of the same category, the mobile phone may mark only the first occurrence of text of that category. The mobile phone marks text entities in the order from top to bottom and from left to right on the interface, so the first occurrence refers to the first occurrence from top to bottom and from left to right.

[0197] For example, after the mobile phone executes the text extraction function and recognizes the current interface, it may display interface 1201 shown in FIG12 . Interface 1201 is the recognition results page. In interface 1201, the first occurrence of an address, i.e., Address 1, is marked with a location icon 12011, the first occurrence of a phone number, i.e., Phone 1, is marked with a phone icon 12012, and the first occurrence of a website link, i.e., Website 1, is marked with a network icon 12013.

[0198] That is, if two text entities (referred to as the first text entity and the fifth text entity) have the same entity category, only one of the text entities (such as the first text entity) can be marked. If the entity categories of the two text entities are different, the two text entities can be marked separately. For ease of explanation, the mark of the first text entity can be referred to as the first text mark, and the mark of the fifth text entity can be referred to as the third text mark.

[0199] In another specific implementation, if there is an obstruction between the entity icons of two text entities (which may be referred to as the first text entity and the sixth text entity), the mobile phone may omit the entity icon of one of the text entities, such as omitting the entity icon of the sixth text entity (which may be referred to as the fourth text marker), to avoid the entity icon being obstructed. For example, in interface 1102, between address 1 marked by location icon 11021 and website 1 marked by network icon 11022, phone 1 is also included. If a phone icon (as shown by the dotted line in interface 1102, which is not actually displayed) is used to mark phone number 1, obstruction between the marks will result. Therefore, the mobile phone may not mark phone 1. This can avoid obstruction between the marks.

[0200] Specifically, the mobile phone may mark text entities in order from top to bottom and from left to right. If there is an obstruction between the current entity icon and the marked entity icon, the mobile phone may cancel the mark of the current entity icon.

[0201] In another specific implementation, the mobile phone may mark only the number 1 of text entity categories whose user interest levels are ranked from high to low.

[0202] Specifically, a mobile phone or other device can analyze the interest of a large number of users (or only local users) in multiple types of text entities. The user's interest in a text entity is positively correlated with the number of times the user triggers the text entity. The more times a user triggers a certain type of text entity (such as clicking or long pressing) while using the phone, the higher the user's interest in that type of text entity.

[0203] In another specific implementation, when using the intelligent recognition function to recognize text, the mobile phone can also determine and mark text entities that match the object entities based on the object entities included in the current interface. Different categories of the object entities will result in different categories of marked text entities.

[0204] Specifically, if the first interface includes multiple text entities (including the first text entity and the second text entity), then the current interface includes item entity 1 (recorded as the first item entity), then the first text entity that matches item entity 1 is marked from the multiple text entities, and the unmatched second text entity is not marked; if the second interface includes multiple text entities (including the third text entity and the fourth text entity), then the current interface includes item entity 2 (recorded as the second item entity), then the third text entity that matches item entity 1 is marked from the multiple text entities, and the unmatched fourth text entity is not marked. The mark of the third text entity can be called the second text mark.

[0205] That is, the electronic device can mark text entities of the first category (the entity category of the first text entity and the fourth text entity) in an interface with a certain category (the entity category of the first item entity), but not mark text entities of the second category (the entity category of the second text entity and the third text entity); and the electronic device can mark text entities of the second category in an interface with another category (the entity category of the second item entity), but not mark text entities of the first category. It can be seen from this that the electronic device can mark matching text entities based on the item entities included in the user interface, thereby providing users with marks for quickly obtaining information from the user interface, which is conducive to improving the efficiency of human-computer interaction.

[0206] Taking the aforementioned method 3 as an example of recommending the translation function, after recommending the smart recognition function, the mobile phone can display the interface 1301 shown in Figure 13A (which can be regarded as a specific first interface), and the interface 1301 includes a picture 13011 and an entrance 13012 of the smart recognition function. In response to a click operation on the entrance 13012 of the smart recognition function, the mobile phone uses the smart recognition function to recognize the picture 13011 and can recognize the product 13013 (which can be regarded as a specific first item entity). Based on this, the mobile phone can predict that the user may want to view the merchant address and purchase goods, and can determine that the text entities matching the picture 13011 are the address and the website link (which can be regarded as two specific first text entities). Therefore, after the mobile phone uses the smart recognition function to recognize the picture 13011, it can display the interface 1302 shown in Figure 13A. Interface 1302 is the recognition result page. In interface 1302, the mobile phone marks the item entity, such as marking the product 13013 with the item tag 13021; ​​and, in interface 1302, the mobile phone also marks the address with the location icon 13022 (which can be regarded as a specific first text tag) so that the user can view the merchant address, and marks the QR code with the scan code icon 13023 (which can be regarded as another specific first text tag) so that the user can scan the code to purchase the product.

[0207] It should be noted that graphic codes, such as QR codes, essentially redirect pages, similar in function to URL links. Therefore, mobile phones can also treat graphic codes as special text entities, using features such as text extraction or intelligent recognition to identify them.

[0208] Furthermore, the mobile phone can mark the text entity that matches the current focus item entity. Moreover, as the focus item entity switches, the marked text entity will also change accordingly. For example, after switching from the first item entity to the third item entity, the mark will also switch from the first text entity to the second text entity, wherein the mark of the second text entity can be called the third text mark. For example, the current focus item entity is item entity 3, and the mobile phone can mark the text entity that matches item entity 3 from multiple text entities; after the focus item entity switches to item entity 4, the mobile phone can switch to marking the text entity that matches item entity 4. In this way, as the focus item entity switches, the mobile phone can dynamically adjust the marked text entity, so that the marked text entity always matches the focus item entity. Among them, regarding the switching of the focus item entity, please refer to the previous introduction on "First, intelligent recognition function", which will not be repeated here.

[0209] In this application, there is no specific limitation on the content of the mark of each type of text entity (ie, the first text mark or the second text mark).

[0210] In some embodiments, the mobile phone can mark text entities with icons corresponding to their categories. For example, a phone number is marked with a phone icon, a website is marked with a network icon, and an address is marked with a location icon. This allows the text entity's category to be clearly indicated through marking.

[0211] In other embodiments, the mobile phone can mark the text entity with the service icon of the service with the highest user interest (which can be called the first service) among the services associated with the text entity. Each type of text entity can be associated with one or more services. A phone number can be associated with multiple services such as making a call, adding a contact, and copying a number. A URL can be associated with multiple services such as visiting the URL, adding the URL to favorites, and sharing the URL. An address can be associated with multiple services such as opening in a map, navigating, adding the address to favorites, and sharing the address. A graphic code can be associated with multiple services such as identifying a graphic code, adding a graphic code to favorites, and sharing a graphic code.

[0212] Specifically, a mobile phone or other device can analyze the user's interest in various services. User interest in a service is positively correlated with the frequency of service use. The more times a user selects a service while using their mobile phone, the higher their interest in that service.

[0213] For example, the services associated with an address include opening it on a map, navigating to it, adding it to favorites, and sharing it:

[0214] If the user is most interested in the navigation service, the mobile phone may display the interface 1311 shown in Figure 13B after recognizing the interface 1301 shown in Figure 13A above. In the interface 1311, the address is marked with a service icon 13111 of the navigation service.

[0215] If the user is most interested in opening the service in the map, the mobile phone can display the interface 1312 shown in Figure 13B after identifying the interface 1301 shown in Figure 13A above. In the interface 1312, the address is marked with a service icon 13121 for opening the service in the map.

[0216] If the user is most interested in the favorite address, the mobile phone may display the interface 1313 shown in Figure 13B after identifying the interface 1301 shown in Figure 13A above. In the interface 1313, the address is marked with a service icon 13131 of the favorite service.

[0217] If the user is most interested in sharing the address, the mobile phone may display the interface 1314 shown in Figure 13B after identifying the interface 1301 shown in Figure 13A above. In the interface 1314, the address is marked with a service icon 13141 of the sharing service.

[0218] In the recognition results, the phone will also highlight the text entity. The phone can highlight the text entity through highlighting, underlining, projection, etc. For example, in interface 1102 and interface 1302, the address, phone number, and website are all underlined and projected.

[0219] After obtaining the recognition result, in response to a user triggering operation (also referred to as a third triggering operation) on the text entity or a mark of the text entity, such as a click operation, the mobile phone can expand the service options associated with the text entity. For ease of explanation, the interface displaying the service options of the text entity can be referred to as the third interface.

[0220] Taking the recognition result page as interface 1401 shown in Figure 14 (the same as interface 1102 above, no further description will be given here) as an example, in response to the user clicking on address 1 in interface 1401, the mobile phone can display interface 1402 shown in Figure 14. Interface 1402 includes pop-up window 14021 and pop-up window 14022. Among them, pop-up window 14021 includes a service option "Open in Map" 140212 for opening the service in the map, a service option "Navigate to" 140213 for the navigation service, a service option "Add to Notes" 140214 for the collection address service, and a service option "Share" 140215 for the sharing address service. In addition, pop-up window 14022 includes a route to address 1.

[0221] In this application, there is no specific limitation on the display order of service options.

[0222] In some embodiments, the order in which service options are displayed is always fixed. For example, among the service options of multiple services associated with an address, the order in which they are displayed from front to back is always: service options for opening the service in the map, service options for the navigation service, service options for the favorite address service, and service options for the share address service, as shown in interface 1402 in FIG. 14 .

[0223] In other embodiments, the display order of service options matches the user's interest in the services, so that service options for services that the user has high interest in can be displayed first, making it easier for the user to quickly view and operate them.

[0224] In a specific implementation, the mobile phone may display service options in descending order of the user's interest in the services.

[0225] In another specific implementation, the mobile phone may display the service option of the service that the user is most interested in first, and display the service options of other services in a fixed order.

[0226] Combining this implementation with the aforementioned embodiment of "marking a text entity with a service icon representing the service with the highest user interest," after marking the text entity with the service icon representing the service with the highest user interest, the mobile phone can display the service option for the service indicated by the service icon first in response to the user's triggering operation on the text entity. That is, the service option displayed first matches the service icon marked with the text entity in the recognition result. This allows the service option for the service with the highest user interest to be displayed first.

[0227] The above description of displaying service options is mainly for the marked address 14011. It should be noted that in practice, all text entities can be associated with one or more services, not just marked text entities.

[0228] For example, although address 2 in the above interface 1401 is not marked, it can also be associated with multiple services such as making a call, adding a contact, copying the number, etc. In response to the user clicking on address 2, the mobile phone can also provide service options such as opening in a map, navigating, adding the address to favorites, and sharing the address.

[0229] For unlabeled text entities, the display order of service options may also be fixed, or may be matched with the user's interest in the services.

[0230] In the recognition results, in response to a trigger operation on the non-entity text, such as a long press operation, the phone can select the text and provide a shortcut entry to perform a quick operation on the non-entity text. For details about the shortcut entry, please refer to the previous description and will not be repeated here.

[0231] For example, in response to a user long-pressing the text "Merchant List" 14011 in interface 1401 shown in FIG14 , the mobile phone may display interface 1403 shown in FIG14 . Unlike interface 1401 , interface 1403 shows that the text "Merchant List" 14011 is selected, and interface 1403 includes a "Copy" entry 14031 , a "Select All" entry 14032 , a "Translate" entry 14033 , a "Share" entry 14034 , and a "Search" entry 14035 .

[0232] At this point, it should be noted that during operations on text entities and item entities, the phone can hide entity markers in the interface. This avoids interference from markers. For example, in the above-mentioned interface 904, interface 1001-interface 1004, and interface 1402-interface 1403, the entity markers are all hidden.

[0233] Third, translation function.

[0234] The phone uses a translation function that can translate the text in the current interface.

[0235] Taking the recommendation of the translation function using the aforementioned method three or method four as an example, after recommending the translation function, the mobile phone can display the interface 1501 shown in Figure 15 (the same as the previous interface 602, which will not be introduced here). Interface 1501 includes "Translate" 15011. "Translate" 15011 is the entrance to the translation function. In response to the user clicking on "Translate" 15011, the mobile phone can display the interface 1502 shown in Figure 15. Interface 1502 is the translation result page. Different from interface 1501, the text in interface 1502 is all in Chinese, that is, all foreign languages ​​are translated into Chinese.

[0236] Furthermore, the translation result page also includes a language selection item for selecting the language of the original text and the translation. Taking the translation result page shown in interface 1502 in Figure 15 as an example, interface 1502 includes a language selection item 15021. In language selection item 15021, the left side is the original text, and the right side is the translation.

[0237] After obtaining the translation result, the mobile phone can also display a scrolling translation control on the result page. In response to the user's triggering operation on the scrolling translation control, such as a click operation, the mobile phone can scroll the interface content and take a screenshot. That is, a scrolling screenshot. Subsequently, in response to the event of ending the scrolling screenshot, the mobile phone can display the translation result of the scrolling screenshot. Among them, the event of ending the scrolling screenshot includes the event of the scrolling time reaching the duration 2, the event of scrolling to the bottom, or the event of receiving the user's click operation. The following is an example of the user's click operation. In this way, after the mobile phone translates the interface content currently displayed on the current interface, it can continue to conveniently translate the scrolling interface content.

[0238] Taking the translation result page shown in interface 1502 in Figure 15 as an example, interface 1502 also includes "scrolling translation" 15022. "Scrolling translation" 15022 is a control for scrolling translation. In response to the user's click operation on "scrolling translation" 15022 in the interface, the mobile phone can display interface 1503 shown in Figure 15. The screenshot is being scrolled in interface 1503. For example, the interface content is scrolling from bottom to top as indicated by the arrow in interface 1503. In response to the user's click operation in interface 1503, the mobile phone can display interface 1504 shown in Figure 15, which is a translation result page of the screenshot result of the scrolling screenshot.

[0239] After obtaining the translation result of the content of the scrolling screenshot, the mobile phone can also continue to provide a scrolling translation control so that the scrolling screenshot can be continued and translated. For example, the interface 1504 includes "Continue Scrolling" 15041, which is a scrolling translation control.

[0240] Furthermore, after obtaining the translation results of the scrolling screenshot, the mobile phone can return to the recommended translation function interface in response to the user's return operation. For example, in response to the user clicking the return control 15042 in the interface 1504 shown in Figure 15, the mobile phone can return to the interface 1501 shown in Figure 15. In this way, the mobile phone can quickly return to the initial interface before translation, allowing the user to operate on the initial interface.

[0241] In some embodiments, the mobile phone can identify whether the current interface is a scrollable interface. If it is, the mobile phone provides scrolling translation controls on the translation results page; if it is not, the mobile phone does not provide scrolling translation controls on the translation results page. The desktop, lock screen, and image viewing interfaces are generally not scrollable interfaces. In this way, the mobile phone can dynamically provide scrolling translation controls for different interfaces after obtaining translation results, thereby ensuring the effectiveness of the displayed controls.

[0242] After obtaining the translation result, in response to the original translation switching operation, such as a click operation on the current interface, the mobile phone can switch to the original text, thereby realizing a quick switch from the translation result to the original text. For example, in response to the user's click operation in the interface 1502 shown in Figure 15, the mobile phone can switch the Chinese in the interface 1502 (except the language selection item) to English. For another example, in response to the user's click operation in the interface 1504 shown in Figure 15, the mobile phone switches the Chinese in the interface 1504 (except the language selection item) to English.

[0243] The above introduction to the translation function mainly explains the implementation of the mobile phone's unified translation of the text in the current interface. In practice, after providing access to the translation function, the mobile phone can also translate part of the text in the current interface.

[0244] Specifically, after providing the translation function, in response to a user long-pressing the original text or the translation text, the phone can select the text and provide a translation entry for the selected text. In response to a triggering operation on the translation entry for the selected text, such as a click, the phone can display the translation result of the selected text. In this way, after providing the translation function entry, the phone can not only translate the entire current interface, but also translate the selected text.

[0245] Taking the translation result page shown in interface 1601 in Figure 16 as an example, in response to the user's long press operation on the text "have to" in interface 1601 shown in Figure 16, the mobile phone can display interface 1602 shown in Figure 16. Different from interface 1601, the text "have to" in interface 1602 is selected (i.e., "have to" is selected text), and interface 1602 includes "Translate" 16021. "Translate" 16021 is the translation entry for "have to". In response to the user's click operation on "Translate" 16021, the mobile phone can display interface 1603 shown in Figure 16. Interface 1603 includes pop-up window 16031. Pop-up window 16031 includes the translation result of "have to".

[0246] In addition, while providing a translation entry for the selected text, the mobile phone can also provide quick entries for other text operations, such as "Copy" 16022, "Select All" 16023, "Share" 16024 and "Search" 16025 in interface 1601 as shown in Figure 16, so as to perform other quick processing on the selected text.

[0247] Fourth, privacy protection function.

[0248] The phone uses a privacy protection function that can block private information in the current interface.

[0249] Taking the above-mentioned method 3 or method 4 as an example to recommend the privacy protection function, after recommending the privacy protection function, the mobile phone can display the interface 1701 shown in Figure 17 (the same as the previous interface 702, which will not be introduced here). Interface 1701 includes "Privacy Coding" 17011. "Privacy Coding" 17011 is the entrance to the privacy protection function. In response to the user's click operation on "Privacy Coding" 17011, the mobile phone can display the interface 1702 shown in Figure 17. The difference from interface 1701 is that the email address, ID number, ID card picture, address and avatar in interface 1702 are all coded. That is, the private information is blocked.

[0250] After blocking private information, the phone can unblock it if the user clicks on any of the blocked private information. If the user clicks on the same private information again, the phone can block it again. This allows the phone to flexibly block or unblock private information.

[0251] Taking interface 1702 shown in FIG17 as an example, ID card number 17021 in interface 1702 is obscured. In response to a user clicking on ID card number 17021 in interface 1702, the mobile phone may display interface 1703 shown in FIG17 . Unlike interface 1702, ID card number 17021 in interface 1703 is not obscured. Subsequently, in response to a user clicking on ID card number 17021 in interface 1703, the mobile phone may display interface 1702 shown in FIG17 again, i.e., unobstructing ID card number 17021.

[0252] After the privacy mask is completed, in response to the user's save operation on the mask effect, the mobile phone can also save the masked picture to the gallery. For example, in response to the user clicking the "Save" 17031 in the interface 1703 shown in Figure 17, the mobile phone can save the picture 17032 displayed in the interface 1703.

[0253] Furthermore, after the saving is completed, the mobile phone can provide a viewing entrance for the picture. In response to the user's triggering operation on the viewing entrance, such as a click operation, the mobile phone can display the saved picture in the gallery. Exemplarily, after the saving is completed, the mobile phone displays the interface 1704 shown in Figure 17, and the interface 1704 includes the prompt "The picture has been saved to the gallery" 17041 and "View" 17042. "View" 17042 is the viewing entrance for the picture. In response to the user's click operation on "View" 17042, the mobile phone can display the interface 1705 shown in Figure 17, which is the viewing interface for pictures in the gallery application. Interface 1705 includes a saved picture 17051, and the privacy information in picture 17051 is blocked.

[0254] In other words, with the privacy protection function, users only need to trigger the privacy protection function entrance and save the operation once to achieve the shielding of private information in the current interface and save the screenshot, thereby improving the efficiency of human-computer interaction.

[0255] Fifth, select the function.

[0256] The mobile phone adopts the selection function, which can easily process the text in the current interface.

[0257] Taking the recommendation and selection function using the aforementioned method 3 or method 4 as an example, after recommending the selection function, the mobile phone may display interface 1801 shown in Figure 18 (the same as interface 802 above, and will not be described here). Interface 1801 includes "Select All" 18011. "Select All" 18011 is a control for selecting the function. In response to the user clicking "Select All" 18011, the mobile phone may select all text in the current interface, such as displaying interface 1802 shown in Figure 18. The difference from interface 1802 is that all text in interface 1802 is selected.

[0258] Furthermore, after selecting all the text, in response to the user adjusting the selection box, the mobile phone can adjust the range of the selected text. This allows for flexible text selection. In addition, after selecting all the text or adjusting the range of the selected text, the mobile phone can also provide quick access to various text operations, such as "Copy" 18021, "Favorite" 18022, "Translate" 18023, "Share" 18024, and "Search" 18025 in interface 1802 as shown in Figure 18, so as to perform quick processing on the selected text.

[0259] An embodiment of the present application further provides an electronic device, which may include: a display screen, a memory, and one or more processors (such as a CPU, GPU, NPU, etc.). The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the various functions or steps performed by the device in the above method embodiment.

[0260] An embodiment of the present application further provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0261] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0262] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0263] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the image processing method in the above-mentioned method embodiments.

[0264] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0265] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0266] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0267] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0268] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0269] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0270] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A screen recognition method, characterized in that: Applied to electronic equipment, the method comprises: Displaying a first interface, wherein the first interface includes a first item entity, a first text entity, and a second text entity; In response to a first trigger operation on the first interface, adding a first text mark to the first text entity, and not adding a mark to the second text entity; Displaying a second interface, wherein the second interface includes a second item entity, a third text entity, and a fourth text entity; In response to a second trigger operation on the second interface, adding a second text mark to the third text entity, and not adding a mark to the fourth text entity; Among them, the entity categories of the first item entity and the second item entity are different, the entity categories of the first text entity and the fourth text entity are the same, the entity categories of the second text entity and the third text entity are the same, and the entity categories of the first text entity and the second text entity are different.

2. The method according to claim 1, characterized in that: The entity categories of text entities include at least two of the following: address, telephone number, flight information, express delivery number, email address, website link, certificate number for identity identification, and graphic code; The entity categories of an item entity include at least two of the following: animals, plants, buildings, and food.

3. The method according to claim 1 or 2, characterized in that: The first text tag indicates an entity category of the first text entity; or, The first text entity is associated with multiple services, including a first service, the first text tag indicates the first service, the first service is the service most selected by users under a first category of text entities, and the first category is an entity category of the first text entity.

4. The method according to claim 3, characterized in that After adding a first text mark to the first text entity in response to the first trigger operation on the first interface, the method further includes: In response to a third trigger operation on the first text entity or the first text mark, a third interface is displayed, wherein the third interface includes multiple service options, and the multiple service options correspond one-to-one to the multiple services; among the multiple service options, the service option for the first service is displayed first.

5. The method according to any one of claims 1 to 4, characterized in that The first interface also includes a fifth text entity; The method further comprises: In response to the first trigger operation on the first interface, and the entity category of the fifth text entity is different from that of the first text entity, adding a third text tag to the fifth text entity; If the fifth text entity has the same entity category as the first text entity, no mark is added to the fifth text entity.

6. The method according to any one of claims 1 to 5, characterized in that The first interface also includes a sixth text entity; The method further comprises: In response to the first trigger operation on the first interface, and the fourth entity mark of the sixth text entity and the first text mark are not blocked, adding the fourth text mark to the sixth text entity; If the fourth text mark and the first text mark are blocked, no mark is added to the sixth text.

7. The method according to any one of claims 1 to 6, characterized in that The first interface also includes a third item entity; The method further comprises: In response to the first trigger operation on the first interface, highlighting the first item entity and displaying a first shortcut entrance around the first item entity; In response to a fourth trigger operation on the third item entity, the third item entity is highlighted and a second shortcut entrance is displayed around the first item entity.

8. The method according to claim 7, characterized in that In response to the first trigger operation on the first interface, highlighting the first item entity and displaying a first shortcut entrance around the first item entity includes: In response to the first trigger operation on the first interface, and the first item entity meets a first condition, highlighting the first item entity and displaying a first shortcut entrance around the first item entity; The first condition includes at least one of the following: the area of ​​the first item entity is larger than the area of ​​the third item entity, the area blocked by the first item entity is smaller than the area blocked by the third item entity, and the clarity of the edge line of the first item entity is higher than the clarity of the edge line of the third item entity.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: In response to the first trigger operation on the first interface, displaying a first item mark on the third item entity; The fourth trigger operation includes a trigger operation on the first item mark.

10. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: In response to the fourth triggering operation on the third item entity, a third text tag is added to the second text entity.

11. The method according to any one of claims 7 to 10, characterized in that: The method further comprises: In response to a move operation on the highlighted item entity, a fourth interface is displayed, wherein the fourth interface includes a plurality of associated entries, each associated entry corresponds to an application or a service, and the plurality of associated entries include a first associated entry, and the first associated entry corresponds to a first application or a second service; In response to moving the highlighted item entity to the first associated entry, the electronic device displays a fifth interface, where the fifth interface is an interface of the first application or the second service, and the fifth interface includes associated information of the highlighted item entity.

12. The method according to claim 11, characterized in that The step of displaying a fourth interface in response to a move operation on the highlighted item entity comprises: In response to a move operation on the highlighted item entity, move the position of the highlighted item entity in the first interface; In response to the position of the highlighted item entity moving to the target area in the first interface, the fourth interface is displayed.

13. The method according to claim 11 or 12, characterized in that: The highlighted item entity is the first item entity, and the plurality of associated entries include a second associated entry; The highlighted item entity is the third item entity, and the plurality of associated entries include a third associated entry; The second associated entry is different from the third associated entry.

14. The method according to any one of claims 1 to 13, characterized in that The first interface is a camera's viewfinder interface, and the first trigger operation includes a shooting operation.

15. The method according to any one of claims 1 to 13, characterized in that Before adding a first text mark to the first text entity in response to a first trigger operation on the first interface, the method further includes: When the first interface satisfies the second condition, displaying the identification control in the first interface; Among them, the second condition includes: the first interface is a non-blank interface, the first interface includes text entities and / or object entities, and the first trigger operation includes a trigger operation on the identification control.

16. The method according to claim 15, characterized in that When the first interface satisfies the second condition, displaying an identification control in the first interface includes: When the first interface satisfies the second condition, in response to a fifth trigger operation of the user on the first interface, the identification control is displayed in the first interface.

17. The method according to claim 16, characterized in that The method further comprises: In response to a fifth trigger operation of the user on the first interface: When the number of foreign languages ​​in the first interface exceeds a first number, a translation control is displayed in the first interface; wherein, when the number of foreign languages ​​in the first interface does not exceed the first number, the translation control is not displayed, and the translation control is used to trigger the electronic device to translate the foreign language in the first interface; In the case where the first interface includes privacy information, a privacy protection control is displayed in the first interface; wherein, in the case where the first interface does not include privacy information, the privacy protection control is not displayed, and the privacy protection control is used to trigger the electronic device to shield the privacy information in the first interface; When the first interface includes text but does not include an entity, a selection control is displayed in the first interface; wherein, when the first interface does not include text or includes an entity, the selection control is not displayed, and the selection control is used to trigger the electronic device to select the text in the first interface.

18. An electronic device, characterized in that: include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in any one of claims 1-17.

19. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 17.

Images