Display method and device and electronic equipment
By receiving input from the program interface in electronic devices and displaying a physical service component on the screen, the problem of users having to perform multiple selection operations is solved, improving operational efficiency and the accuracy of recommended services.
Patent Information
- Application Number
- CN202511390953.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-26
- Publication Date
- 2026-01-13
AI Technical Summary
In electronic devices, users need to select multiple screen information to trigger the device to perform corresponding operations, which makes the operation process cumbersome and time-consuming.
By receiving the first input from the program interface, the screen entity service component corresponding to each screen entity is displayed, and the corresponding recommended service is determined based on the entity information, reducing the user's input for selecting each screen entity.
It improves the efficiency of electronic devices in handling screen information, reduces user interaction, and enables a more accurate understanding of user needs, providing recommendation services that match user intent.
Smart Images

Figure CN121326174A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a display method, device, and electronic device. Background Technology
[0002] With the development of electronic devices, most electronic devices support screen information recognition and provide corresponding services based on screen information, such as "circle search," which allows users to select the information they want to recognize by circling, and the electronic device automatically provides the corresponding operation.
[0003] However, in the above method, if the user needs to select multiple screen information on an interface, the user needs to trigger the electronic device to perform multiple selection operations before the electronic device can operate on the screen information selected by the user, which makes the process of the electronic device operating on the screen information cumbersome and time-consuming. Summary of the Invention
[0004] The purpose of this application is to provide a display method, device, electronic device, and storage medium that can improve the efficiency of electronic devices in operating screen information under the circle search function.
[0005] In a first aspect, embodiments of this application provide a display method, the display method comprising: receiving a first input on a program interface; and in response to the first input, displaying a screen entity service component corresponding to each screen entity in the program interface; wherein the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
[0006] In some embodiments of this application, the display method provided by the embodiments of this application further includes: identifying at least one screen entity in the program interface; and determining at least one service key step information corresponding to each screen entity based on the entity information of each screen entity in the at least one screen entity; one service key step information corresponding to one screen entity service component; and determining the recommended service corresponding to each screen entity service component based on the service key step information corresponding to each screen entity; wherein, the entity information includes: entity type and entity content.
[0007] In some embodiments of this application, when the screen entity is a session message, the entity information includes: session message content; when the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, link navigation information; when the screen entity is an image, the entity information includes: image content; when the screen entity is a video, the entity information includes: video content, video subtitles; when the screen entity is a component, the entity information includes: component name, component function, component type.
[0008] In some embodiments of this application, after displaying the screen entity service component corresponding to each screen entity in the program interface, the display method provided in this application embodiment further includes: receiving a second input to a first screen entity service component, wherein the first screen entity service component is any one of all screen entity service components displayed in response to the first input; and displaying the service interface of the recommended service corresponding to the first screen entity service component in response to the second input.
[0009] In some embodiments of this application, the above-mentioned program interface further includes at least one global service component, which is generated based on the interface content of the program interface.
[0010] In some embodiments of this application, when the program interface is a chat interface, at least one global service component includes at least one of the following: a group chat service component, an add contact service component, and a text editing service component; when the program interface is a document interface, at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component; when the program interface is a shopping interface, at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object rating service component.
[0011] In some embodiments of this application, after displaying the screen entity service component corresponding to each screen entity in the program interface, the display method provided in this application further includes: receiving a third input to a first screen entity in the program interface, wherein the first screen entity is any one of at least one screen entity; in response to the third input, displaying at least one entity sub-service component corresponding to the first screen entity in the program interface; wherein the recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
[0012] In some embodiments of this application, when the first screen entity is an image, the recommendation service corresponding to at least one entity sub-service component is determined based on the image content of the image; when the first screen entity is a video, the recommendation service corresponding to at least one entity sub-service component is determined based on the video content and video subtitles of the video; when the first screen entity is a session message, the recommendation service corresponding to at least one entity sub-service component is determined based on the message content of the session message; when the first screen entity is a link, the recommendation service corresponding to at least one entity sub-service component is determined based on the link content of the link; and when the first screen entity is a component, the recommendation service corresponding to at least one entity sub-service component is determined based on the component content contained in the component.
[0013] In some embodiments of this application, after receiving a third input to a first screen entity in the program interface, the display method provided in this application further includes: in response to the third input, canceling the display of all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity; wherein, the third screen entity service component includes at least one entity sub-service component and a second screen entity service component.
[0014] In some embodiments of this application, the program interface further includes an information input control; in the display method provided in the embodiments of this application, when a fourth input of search information is received on the information input control, the search information is displayed on the information input control, and result information is displayed in the program interface, wherein the result information is obtained by searching based on the search information and the interface content of the program interface; or, when a fifth input of search information is received on a second screen entity in at least one screen entity and a fourth input of search information is received on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
[0015] In some embodiments of this application, the display method provided by the embodiments of this application further includes: receiving a sixth input to an information input control; in response to the sixth input, displaying at least one search candidate word, wherein the at least one search candidate word is predicted based on the interface content of the program interface; or, the search candidate word is predicted based on the entity information of a second screen entity; receiving a fourth input of search information entered into the information input control, including: receiving a fourth input of one of the at least one search candidate word; displaying search information in the information input control, including: displaying the search candidate word selected by the fourth input in the information input control.
[0016] Secondly, embodiments of this application provide a display device, which includes a receiving module and a display module. The receiving module is used to receive a first input on a program interface. The display module is used to display a screen entity service component corresponding to each screen entity in the program interface in response to the first input received by the receiving module; wherein the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
[0017] In one possible implementation, the display device further includes a processing module. The processing module is configured to identify at least one screen entity in the program interface; and based on the entity information of each screen entity, determine at least one service key step information corresponding to each screen entity; one service key step information corresponding to one screen entity service component; and based on the service key step information corresponding to each screen entity, determine a recommended service corresponding to each screen entity service component; wherein the entity information includes: entity type and entity content.
[0018] In one possible implementation, when the screen entity is a session message, the entity information includes: session message content; when the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, link navigation information; when the screen entity is an image, the entity information includes: image content; when the screen entity is a video, the entity information includes: video content, video subtitles; when the screen entity is a component, the entity information includes: component name, component function, component type.
[0019] In one possible implementation, the receiving module is further configured to receive a second input to the first screen entity service component after the display module displays the screen entity service component corresponding to each screen entity in the program interface. The first screen entity service component is any one of all screen entity service components displayed in response to the first input. The display module is further configured to display the service interface of the recommended service corresponding to the first screen entity service component in response to the second input received by the receiving module.
[0020] In one possible implementation, the program interface further includes at least one global service component, which is generated based on the interface content of the program interface.
[0021] In one possible implementation, when the program interface is a conversational interface, at least one global service component includes at least one of the following: a group chat service component, an add contact service component, and a text editing service component; when the program interface is a document interface, at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component; when the program interface is a shopping interface, at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object evaluation service component.
[0022] In one possible implementation, the receiving module is further configured to, after displaying the screen entity service component corresponding to each screen entity in the program interface, receive a third input to the first screen entity in the program interface, wherein the first screen entity is any one of at least one screen entity. The display module is further configured to, in response to the third input received by the receiving module, display at least one entity sub-service component corresponding to the first screen entity in the program interface; wherein the recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
[0023] In one possible implementation, when the first screen entity is an image, the recommendation service corresponding to at least one entity sub-service component is determined based on the image content of the image; when the first screen entity is a video, the recommendation service corresponding to at least one entity sub-service component is determined based on the video content and video subtitles of the video; when the first screen entity is a session message, the recommendation service corresponding to at least one entity sub-service component is determined based on the message content of the session message; when the first screen entity is a link, the recommendation service corresponding to at least one entity sub-service component is determined based on the link content of the link; and when the first screen entity is a component, the recommendation service corresponding to at least one entity sub-service component is determined based on the component content contained in the component.
[0024] In one possible implementation, the display module is further configured to, after the receiving module receives a third input to the first screen entity in the program interface, respond to the third input by canceling the display of all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity; wherein the third screen entity service component includes at least one entity sub-service component and a second screen entity service component.
[0025] In one possible implementation, the program interface further includes an information input control; the display module is also configured to, upon receiving a fourth input of search information entered on the information input control, display the search information on the information input control and display result information in the program interface, wherein the result information is obtained by searching based on the search information and the interface content of the program interface; or, upon receiving a fifth input to a second screen entity in at least one screen entity and a fourth input of search information entered on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
[0026] In one possible implementation, the receiving module is further configured to receive a sixth input to the information input control. The display module is further configured to, in response to the sixth input, display at least one search candidate term, wherein the at least one search candidate term is predicted based on the interface content of the program interface; or, the search candidate term is predicted based on entity information of entities on the second screen. Specifically, the receiving module is configured to receive a fourth input to one of the at least one search candidate term. Specifically, the display module is configured to display the search candidate term selected by the fourth input in the information input control.
[0027] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.
[0028] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0029] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.
[0030] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.
[0031] In this embodiment, a first input is received from the program interface; in response to the first input, a screen entity service component corresponding to each screen entity in the program interface is displayed; wherein, the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity. In this solution, since the screen entity service components corresponding to all screen entities can be displayed through the first input in the program interface, the user does not need to select each screen entity, reducing the user interaction operations when using the service; moreover, by recognizing entities in the program interface, the user's current needs can be understood more accurately, thereby providing recommendation services that better match the user's intentions. Attached Figure Description
[0032] Figure 1 This is a flowchart illustrating a display method provided in some embodiments of this application;
[0033] Figure 2 This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0034] Figure 3 These are schematic diagrams illustrating an example of a browser interface provided by some embodiments of this application;
[0035] Figure 4 This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0036] Figure 5 This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0037] Figure 6 These are schematic diagrams illustrating an example of a browser interface provided by some embodiments of this application;
[0038] Figure 7 These are schematic diagrams illustrating an example of a browser interface provided by some embodiments of this application;
[0039] Figure 8A This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0040] Figure 8B This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0041] Figure 9 This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0042] Figure 10 These are schematic diagrams illustrating an example of a browser interface provided by some embodiments of this application;
[0043] Figure 11A This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0044] Figure 11B This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0045] Figure 12 This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0046] Figure 13A This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0047] Figure 13B This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0048] Figure 13C This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0049] Figure 14A This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0050] Figure 14B This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0051] Figure 14C This is a schematic diagram illustrating an example of a chat interface provided by some embodiments of this application;
[0052] Figure 15 This is a flowchart illustrating a display method provided in some embodiments of this application;
[0053] Figure 16 This is a schematic diagram of the structure of a display device provided in some embodiments of this application;
[0054] Figure 17 This is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of this application;
[0055] Figure 18 This is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of this application. Detailed Implementation
[0056] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0057] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0058] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."
[0059] The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. The terminology involved in the embodiments of this application is explained below.
[0060] Controls: A control is a graphical user interface (GUI) element. It is a software component contained within an application that controls all the data processed by the application and the interactive operations related to that data. Users can interact with controls through direct manipulation to read or edit information about the application. Generally, controls can include visual interface elements such as icons (e.g., application icons on electronic devices, folder icons), buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.
[0061] On-screen entities refer to the entities or information displayed on the screen when a user interacts with the device. These entities can include various interface elements, icons, text, buttons, etc., and are the objects through which the user interacts with the device. In the fields of human-computer interaction and artificial intelligence, understanding on-screen entities is crucial for achieving more intelligent interactive experiences. For example, when a user uses a voice assistant to perform a task, the assistant needs to be able to recognize and understand the relevant entities displayed on the screen in order to accurately execute the user's instructions.
[0062] An application (APP) is a computer program designed to perform one or more specific tasks. It runs in user mode, can interact with the user, and has a visual user interface.
[0063] An interface (interface) refers to the visual operational layer through which users interact with electronic devices, such as mobile phones. It serves as the medium for information presentation and input / output within a hardware or software system. Electronic devices can display an interface on a screen, allowing users to interact with the device through touch, clicks, or gestures. The interface can include elements such as icons, buttons, menus, text, and labels.
[0064] Circle Search Mode: Circle search mode is a function that allows users to search by circling content on the screen. It allows users to directly circle the content they want to search for using simple gestures, and the system automatically recognizes the circled content and returns the search results. This mode typically combines technologies such as high-precision image segmentation, OCR recognition, and large model understanding capabilities to more accurately identify and search the circled content. The advantages of circle search mode lie in its convenience and intuitiveness. Users do not need to leave the current page or think of suitable keywords; they can obtain the information they need simply through gesture operations. For example, a user can circle an image containing clothing on the screen, causing the electronic device to jump to the search interface for that clothing, which contains search results for that clothing.
[0065] Artificial Intelligence (AI) Assistants: AI assistants are intelligent agents capable of performing tasks. They possess autonomous planning, decision-making, and execution capabilities, and can independently complete complex tasks (such as writing reports or booking flights). The goal of AI assistants is to provide users with end-to-end problem solutions, rather than simply providing information. They support multi-turn dialogues, progressively clarifying user intent, and can remember context, allowing each query to be processed based on previous interactions. The development of AI assistants is primarily due to the continuous advancements in speech recognition and natural language processing technologies. These breakthroughs enable AI assistants to more accurately understand user intent and provide a more natural and fluid interactive experience.
[0066] Recommendation services: Recommendation services are services that provide personalized content to users based on their behavior and preferences. These services analyze user data to predict user interests and needs, and then recommend relevant products, information, or other services. Recommendation services are widely used in e-commerce, social media, news media, and other fields, helping users discover content they might be interested in, while also helping service providers improve user experience and increase user engagement. In algorithmic recommendation services, service providers use sophisticated algorithms to analyze user behavior and preferences and customize content for users based on this information.
[0067] A component is a graphical user interface element, a basic building block of the user interface, such as a window or text box, displayed in the program interface of any application. Controls can be buttons, text boxes, labels, etc., used to control all data processed by each application and interactive operations related to that data. Components can receive user input to perform corresponding processing or display relevant data. Components can include, but are not limited to, virtual buttons, sliders, progress bars, and checkboxes.
[0068] Screen Entity Service Component: This is a component used to carry the recommendation service corresponding to the screen entity. The screen entity service component can include the name of the recommendation service.
[0069] Global service component: This is a component used to carry the recommendation service corresponding to the program interface. The global service component can include the name of the recommendation service.
[0070] The display method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0071] On mobile devices, most manufacturers support screen recognition and provide corresponding services based on screen information. The industry generally adopts the "circle search" method, where users select the area they want to recognize by circling, and the system provides the corresponding search capability.
[0072] The display method provided in this application embodiment can be applied to search scenarios, such as navigation route search scenarios or item search scenarios.
[0073] For example, in the navigation route search scenario mentioned above, while the user is chatting with a contact, the contact sends the user a link containing restaurant location information. At this time, the user needs to use a navigation application to find the navigation route to the restaurant contained in the link.
[0074] In the above-mentioned item search scenario, when a user sees an image containing a dress they like in a browser application, the user needs to find that dress in the item search interface of the shopping application.
[0075] The execution subject of the display method provided in this application embodiment can be a display device, which can be an electronic device or a functional module in an electronic device. The following uses an electronic device as an example to illustrate the technical solution provided in this application embodiment.
[0076] This application provides a display method. Figure 1 A flowchart illustrating a display method provided in an embodiment of this application is shown. Figure 1 As shown, the display method provided in this application embodiment may include the following steps 201 and 202.
[0077] Step 201: The electronic device receives the first input on the program interface.
[0078] In some embodiments of this application, the above-described program interface includes at least one screen entity.
[0079] In some embodiments of this application, the first input is used to trigger the electronic device to display a screen entity service component corresponding to each of at least one screen entity in the program interface.
[0080] In some embodiments of this application, the first input can be a user's click input on the program interface using a touch device such as a finger or stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this application does not limit it.
[0081] Optionally, in the embodiments of this application, the aforementioned specific gesture can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure recognition gesture, a long-press gesture, an area change gesture, a double-press gesture, or a double-tap gesture.
[0082] Optionally, in the embodiments of this application, the above-mentioned click input can be single-click input, double-click input, or any number of click inputs, or it can be long-press input or short-press input.
[0083] For example, the first input mentioned above can be a double-click input by the user in the program interface.
[0084] In some embodiments of this application, the above-mentioned program interface can be any interface of any application installed in the electronic device. For example, the program interface can be the chat interface of an instant messaging program, the shopping interface of a shopping program, or the navigation interface of a navigation program, etc. The specific interface can be determined according to actual usage needs, and this application embodiment does not impose any restrictions.
[0085] In some embodiments of this application, the screen entities described above may include at least one of the following: images, videos, text, messages, links, and controls. The specific entity can be determined according to actual usage requirements, and this application does not impose any limitations.
[0086] In one example, considering the navigation route search scenario described above, and taking a mobile phone as an example, such as... Figure 2 As shown, when the chat interface 10 with contact 1 is displayed, the chat interface includes three chat messages. Chat message 101 is: "Want to have dinner together tonight?", chat message 102 is: "Sure, what do you want to eat?", and chat message 103 is: "[WWW.abc.com] Sihaiweijia Restaurant". Then, the user can perform a two-finger long press input in the chat interface, so that the phone can display the screen entity service component corresponding to each chat message in the chat interface 10.
[0087] In another example, combining the above-mentioned item search scenario, such as Figure 3 As shown, when the browser interface 11 is displayed, the browser interface 11 includes an image 12 containing a dress. Then, the user can long-press the desktop control 13 in the navigation bar in the chat interface to input, so that the mobile phone can display the screen entity service component corresponding to the image 12 in the browser interface 11.
[0088] Step 202: The electronic device responds to the first input and displays the screen entity service component corresponding to each screen entity in the program interface.
[0089] In some embodiments of this application, the screen entity service component corresponds to a recommendation service, which is generated based on the entity information of a screen entity.
[0090] In some embodiments of this application, the electronic device may display the screen entity service component corresponding to each screen entity at a preset display position of each screen entity in at least one screen entity.
[0091] In some embodiments of this application, each screen entity corresponding to the above-mentioned screen entity may be one or more screen entity service components.
[0092] For example, the user in Figure 2 After making the first input in the chat interface, such as Figure 4As shown, the mobile phone can display a "restaurant search" control 110 around the chat message "Want to have dinner together tonight?", a "join multi-person meeting" control 111 around the chat message "Sure, what do you want to eat?", and a "jump link" control 112 around the chat message "[WWW.abc.com] Sihaiweijia Restaurant".
[0093] For example, combined Figure 4 ,like Figure 5 As shown, in response to the first input, the mobile phone can display a "Restaurant Search" control 110 around the chat message "Want to have dinner together tonight?", a "Join a multi-person meeting" control 111 around the chat message "Sure, what do you want to eat?", and a "Jump Link" control 112 and a "Restaurant Location Search" control 113 around the chat message "[WWW.abc.com] Sihaiweijia Restaurant".
[0094] For example, combined Figure 3 ,like Figure 6 As shown, in response to the first input, the mobile phone can display an "item search" control 121 around the image 12 in the browser interface 11.
[0095] For example, combined Figure 6 ,like Figure 7 As shown, in response to the first input, the mobile phone can display an "item search" control 121 and a "find similar items" control 122 around the image 12 in the browser interface 11.
[0096] In the display method provided in this application embodiment, a first input is received on the program interface; in response to the first input, a screen entity service component corresponding to each screen entity in the program interface is displayed; wherein, the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity. In this solution, since the screen entity service components corresponding to all screen entities can be displayed through the first input on the program interface, the user does not need to select each screen entity individually, reducing user interaction operations when using the service; moreover, by recognizing entities on the program interface, the user's current needs can be understood more accurately, thereby providing recommendation services that better match the user's intent.
[0097] In some embodiments of this application, after step 201 above, the display method provided by the embodiments of this application further includes step 301 below, and step 202 above can be specifically implemented by step 202a below.
[0098] Step 301: The electronic device responds to the first input and starts the circle search mode.
[0099] In some embodiments of this application, after receiving the first input on the program interface, the electronic device can display light and shadow effects on the program interface to prompt the user that the search mode has been activated.
[0100] In some embodiments of this application, after the electronic device receives the first input in the program interface, it can display a circle search icon in the program interface to prompt the user that the circle search mode has been started.
[0101] For example, the above-mentioned search identifier can be any of the following: text identifier, emoticon identifier, symbol identifier, and image identifier, etc. The specific identifier can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions.
[0102] For example, electronic devices can display a search icon in a blank area of the program interface.
[0103] It should be noted that the aforementioned blank area refers to the area in the program interface that does not contain controls, links, files, or icons.
[0104] In some embodiments of this application, the aforementioned documents may include at least one of the following: text, messages, images, and videos.
[0105] Step 202a: In response to the first input, the electronic device displays the screen entity service component corresponding to each screen entity in the program interface in the circle search mode.
[0106] It should be noted that the specific implementation process of step 202a above can be found in the above embodiments, and will not be repeated here to avoid repetition.
[0107] In some embodiments of this application, the electronic device can display the screen entity service component corresponding to each screen entity in the program interface in the circle search mode. In this way, when the user needs to circle multiple screen entities in the program interface, the screen entity service component corresponding to each screen entity can be directly selected, without the need to circle multiple screen entities, thus reducing the circle selection operation of the electronic device.
[0108] In some embodiments of this application, the display method provided in this application further includes the following steps 401 to 403.
[0109] Step 401: Identify at least one screen entity in the electronic device's identification program interface.
[0110] In some embodiments of this application, the electronic device can take a screenshot of the program interface to obtain a first screenshot corresponding to the program interface, and then send the first screenshot to a first model to identify all screen entities in the program interface.
[0111] For example, the first model described above can be a neural network model or an artificial intelligence (AI) model, etc. The specific model can be determined according to actual usage requirements, and this application embodiment does not impose any limitations.
[0112] In some embodiments of this application, the electronic device can obtain all screen entities in the program interface through the application installation package of the application corresponding to the program interface.
[0113] Step 402: The electronic device determines at least one service key step information corresponding to each screen entity based on the entity information of each screen entity in at least one screen entity.
[0114] In some embodiments of this application, one of the screen entity service components in the above-mentioned at least one screen entity corresponds to a service key step information.
[0115] In some embodiments of this application, the aforementioned entity information includes: entity type and entity content.
[0116] For example, the above entity type can be any of the following: text type, image type, video type, component type, or link type.
[0117] In some embodiments of this application, when the screen entity is a session message, the aforementioned entity information includes: session message content.
[0118] For example, combined Figure 2 The aforementioned conversation message can be chat message 101 in chat interface 10, and the aforementioned entity information can be "Want to have dinner together tonight?".
[0119] For example, the content of the aforementioned conversation messages includes at least one of the following: text content, emoticon content, and special symbol content. The specific content can be determined according to actual usage needs, and this application embodiment does not impose any limitations.
[0120] In some embodiments of this application, when the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, and link navigation information.
[0121] For example, combined Figure 2 The aforementioned screen entity can be the link "
WWW.abc.com
WWW.abc.com
[0122] In some embodiments of this application, when the screen entity is an image, the entity information includes: image content.
[0123] For example, combined Figure 3 The aforementioned screen entity can be an image 12 of a dress contained in the browser interface 11, and the aforementioned entity information can be the color and style of the dress.
[0124] In some embodiments of this application, when the screen entity is a video, the entity information includes: video content and video subtitles.
[0125] For example, the screen entity mentioned above could be a video of a person running, and the entity information could be: text describing the video, such as "a person is running on the beach," and subtitles included in the video, such as "the sun is setting, the time is just right."
[0126] In some embodiments of this application, when the screen entity is a component, the entity information includes: component name, component function, and component type.
[0127] For example, the screen entity mentioned above can be a music playback component, and the entity information can be: the name of the music playback component, for example, the playback component for music A.
[0128] In some embodiments of this application, the electronic device may pre-determine the corresponding association between the entity information of each screen entity and the service key step information in at least one screen entity. Thus, after determining the entity information of each screen entity, the electronic device can automatically determine the entity information and service key step information of each screen entity based on the above association.
[0129] For example, the above-mentioned key service steps information includes the necessary steps required to perform the recommendation service.
[0130] For example, the key steps of the above service could be: Step 1, open the shopping application; Step 2, enter the item name into the item search bar in the shopping application; Step 3, click the search control in the item search bar.
[0131] Step 403: The electronic device determines the recommended service corresponding to each screen entity service component in at least one screen entity based on the service key step information corresponding to each screen entity in at least one screen entity.
[0132] In some embodiments of this application, an electronic device can generate a corresponding recommended service name based on the service key step information corresponding to each screen entity in at least one screen entity, and establish a mapping relationship between the control and the service key step information to obtain a screen entity service component.
[0133] It should be noted that each step from 401 to 403 above, and the related descriptions, can be implemented by an AI assistant in an electronic device.
[0134] In some embodiments of this application, electronic devices can more accurately understand the user's current needs by recognizing screen entities of screen content, thereby providing recommendation services that better match the user's intent and improving the accuracy of the electronic device's recommendation services.
[0135] In some embodiments of this application, after step 202 described above, the display method provided in this application further includes steps 501 and 502 as described below.
[0136] Step 501: The electronic device receives a second input to the first screen entity service component.
[0137] In this embodiment of the application, the first screen entity service component is any one of all screen entity service components that respond to the first input display.
[0138] In some embodiments of this application, the second input is used to trigger the electronic device to execute the service corresponding to the first screen entity service component.
[0139] In some embodiments of this application, the second input can be a user's click input on the first screen physical service component via a touch device such as a finger or stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this application embodiment does not limit it.
[0140] For example, the second input described above can be a user's click input on a first screen entity service component.
[0141] Step 502: The electronic device responds to the second input and displays the service interface of the recommended service corresponding to the first screen entity service component.
[0142] In some embodiments of this application, the electronic device can jump to the service interface of the recommended service corresponding to the first screen entity service component and execute the recommended service corresponding to the first screen entity service component.
[0143] In some embodiments of this application, the electronic device can display the service interface of the recommended service corresponding to the first screen entity service component in the program interface through a pop-up window.
[0144] Exemplarily, in combination with Figure 4 , such as Figure 8A shown, if the user wants to view the detailed information of the chat message "[WWW.abc.com] Sihai Weijia Restaurant", the user can click on the "Jump Link" control 112, as Figure 8B shown, the mobile phone can display the details interface 21 of the Sihai Weijia Restaurant in Application A through a pop-up window 20 in the chat interface. The details interface 21 includes: the address 210 of the Sihai Weijia Restaurant, the per capita price 211, the nearby parking lot information 212, and the navigation information 213.
[0145] In some embodiments of the present application, the user can directly input to the screen entity service component corresponding to the screen entity, so that the electronic device can directly execute the function corresponding to the screen entity service component, without performing multiple selection operations, improving the operation efficiency of the electronic device.
[0146] In some embodiments of the present application, the above program interface further includes at least one global service component, and the at least one global service component is generated based on the interface content of the program interface.
[0147] In some embodiments of the present application, the electronic device can display at least one global service component in the blank area of the program interface; or, the electronic device can display at least one global service component in the program interface through a pop-up window.
[0148] In some embodiments of the present application, the electronic device can input the interface content of the program interface into a first model to determine at least one recommended service corresponding to the interface content of the program interface through the first model, and obtain at least one global service component through the at least one recommended service.
[0149] It should be noted that the specific process of the electronic device obtaining at least one global service component through the at least one recommended service can be found in the above embodiments. To avoid repetition, it will not be elaborated here.
[0150] In some embodiments of the present application, when the above program interface is a session interface, the at least one global service component includes at least one of the following: a group chat service component, a contact adding service component, and a text editing service component.
[0151] Exemplarily, in combination with Figure 4 , such as Figure 9 shown, the mobile phone can display three global service components at the bottom of the chat interface. The three global service components include: a join group chat control 30, a copy control 31, and a reference control 32.
[0152] In some embodiments of this application, when the program interface is a document interface, the above-mentioned at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component.
[0153] In some embodiments of this application, when the program interface is a shopping interface, the above-mentioned at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object evaluation service component.
[0154] For example, combined Figure 7 ,like Figure 10 As shown, the mobile phone can display three global service components at the bottom of the browser 11. These three global service components include: image editing control 33, search control 34, and rating control 35.
[0155] It should be noted that the above-mentioned at least one global service component corresponds to the interface content of the program interface, that is, the function of the above-mentioned at least one global service component is different in different interfaces.
[0156] In some embodiments of this application, a user can input input to any one of the at least one global service components so that the electronic device can perform the function corresponding to that global service component.
[0157] It should be noted that the specific process by which an electronic device executes the function corresponding to any global service component can be found in the above embodiments, and will not be repeated here to avoid repetition.
[0158] In some embodiments of this application, the electronic device can display at least one global service component in the program interface through the interface content of the program interface, so as to more accurately understand the user's current needs, thereby providing services and recommended content that are more in line with the user's intentions, and improving the flexibility and accuracy of the electronic device in recommending content.
[0159] In some embodiments of this application, after step 202 described above, the display method provided in this application embodiment further includes steps 601 and 602 as described below.
[0160] Step 601: The electronic device receives a third input to the first screen entity in the program interface.
[0161] In some embodiments of this application, the first screen entity described above is any one of at least one screen entity.
[0162] In some embodiments of this application, the third input is used to trigger the electronic device to display at least one entity sub-service component corresponding to the first screen entity.
[0163] In some embodiments of this application, the aforementioned third input may be a user's click input on the first screen entity via a touch device such as a finger or stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this application embodiment does not limit it.
[0164] For example, the third input mentioned above can be a user's click input on a first screen entity.
[0165] Step 602: In response to the third input, the electronic device displays at least one entity sub-service component corresponding to the first screen entity on the program interface.
[0166] In some embodiments of this application, the recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
[0167] In some embodiments of this application, the electronic device may display at least one entity sub-service component at a preset display position of the first screen entity.
[0168] For example, an electronic device may display at least one entity sub-service component below the screen entity service component corresponding to the first screen entity.
[0169] In some embodiments of this application, when the first screen entity is an image, the recommendation service corresponding to the at least one entity sub-service component is determined based on the image content of the image.
[0170] In some embodiments of this application, when the first screen entity is a video, the recommendation service corresponding to the above-mentioned at least one entity sub-service component is determined based on the video content and video subtitles.
[0171] In some embodiments of this application, when the first screen entity is a session message, the recommendation service corresponding to the above-mentioned at least one entity sub-service component is determined based on the message content of the session message.
[0172] In some embodiments of this application, when the first screen entity is a link, the recommendation service corresponding to the at least one entity sub-service component is determined based on the link content.
[0173] In some embodiments of this application, when the first screen entity is a component, the recommendation service corresponding to the above-mentioned at least one entity sub-service component is determined based on the component content contained in the component.
[0174] Exemplarily, in combination with Figure 4 , such as Figure 11A shown, the user can click and input the chat message "[WWW.abc.com] Sihaiweijia Restaurant", such as Figure 11B shown, so that the mobile phone can display two entity sub-service components around the chat message 1, and the two entity sub-service components are respectively the "Restaurant Rating" control 36 and the "Restaurant Navigation" control 37.
[0175] In some embodiments of the present application, the user can click and input any one of the at least one entity sub-service component, so that the electronic device can execute the function corresponding to any one of the entity sub-service components.
[0176] In some embodiments of the present application, the electronic device allows the user to manually adjust the entity recognition range and perform secondary recognition, providing the user with greater flexibility and a sense of control, and enabling the user to easily obtain more relevant services. For example, according to the user's input on the screen entity, the electronic device can obtain the recommended services corresponding to the entity content corresponding to the screen entity again, so as to identify the potential needs of the user, and then meet the user's deeper needs, and perform service recommendations in a progressive manner to help the user quickly solve more complex problems.
[0177] In some embodiments of the present application, after the above step 601, the display method provided by the embodiments of the present application further includes the following step 701.
[0178] Step 701: In response to the third input, the electronic device cancels the display of all screen entity service components in the program interface except the third screen entity service component corresponding to the first screen entity in the program interface.
[0179] In some embodiments of the present application, the above third screen entity service component includes at least one entity sub-service component and a second screen entity service component.
[0180] In some embodiments of the present application, the above canceling the display of all screen entity service components in the program interface except the third screen entity service component corresponding to the first screen entity in the program interface can be understood as the electronic device performing a screen clearing process on the program interface according to the user's third input.
[0181] Exemplarily, in combination with Figure 11B , such as Figure 12As shown, in response to the third input, the mobile phone can cancel the display of the "Restaurant Search" control 110 corresponding to the chat message "Want to have dinner together tonight?" in the chat interface, cancel the display of the "Join Multi-person Meeting" control 111 corresponding to the chat message "Sure, what do you want to eat?" in the chat interface 10, and cancel the "Join Group Chat" control 30, "Copy" control 31, and "Quote" control 32 in the chat interface.
[0182] In some embodiments of this application, the user can input information into the program interface so that the electronic device can redisplay all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity.
[0183] In some embodiments of this application, the electronic device may hide all screen entity service components except for the third screen entity service component corresponding to the first screen entity, in order to enhance the readability of the first screen entity.
[0184] In some embodiments of this application, the above-mentioned program interface further includes an information input control; the display method provided in the embodiments of this application further includes the following steps 801 or 802.
[0185] Step 801: Upon receiving a fourth input of search information on the information input control, the electronic device displays the search information on the information input control and displays the result information in the program interface.
[0186] In some embodiments of this application, the above-mentioned results are obtained based on search information and the interface content of the program interface.
[0187] In some embodiments of this application, the fourth input is used to trigger the electronic device to display result information.
[0188] In some embodiments of this application, the above-mentioned information input control can be a text input box or a language input box.
[0189] For example, when the above information input control is a text input box, the fourth input can be text input in the text input box; or, when the above information input control is a language input box, the fourth input can be voice input in the voice input box.
[0190] In some embodiments of this application, the electronic device can input search information and the interface content of the program interface into the first model to obtain result information.
[0191] Step 802: When the electronic device receives a fifth input to a second screen entity in at least one screen entity and a fourth input to search information on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
[0192] In some embodiments of this application, when the user selects a second screen entity and does not enter search information, the above result information is generated based on the entity information of the second screen entity.
[0193] In some embodiments of this application, the electronic device can display the result information in the program interface through a pop-up window; or, the electronic device can create a new page and display the result information on that page.
[0194] In some embodiments of this application, the electronic device can input search information and the content of the program interface into the first model together to obtain search result information.
[0195] For example, the first model described above can be a large language model or a neural network model.
[0196] For example, combined Figure 4 ,like Figure 13A As shown, the phone can display a text input box 40 at the bottom of the chat interface. If the user does not select a second screen entity and taps the text input box 41 to input information, as shown... Figure 13B As shown, the mobile phone can display a text input method in the chat interface. Then, the user can use this text input method to enter "What are the signature dishes of Sihaiweijia Restaurant" in the text input box 41, such as... Figure 13C As shown, this allows the phone to display "Sihaiweijia Restaurant's signature dishes include: braised pork, char siu, and siu mai. For two people, we recommend braised pork and rice" in the search results interface 42.
[0197] Furthermore, by way of example, in combination Figure 4 ,like Figure 14A As shown, the mobile phone can display a text input box 41 at the bottom of the chat interface 10. When the user selects a link and clicks on the text input box to enter information, such as... Figure 14B As shown, the mobile phone can display a text input method in the chat interface. Then, the user can use this text input method to enter "What are the signature dishes of Sihaiweijia Restaurant" in the text input box 41, such as... Figure 14C As shown, this allows the mobile phone to display "Sihaiweijia Restaurant's signature dishes include: braised pork, char siu, and siu mai" in the search results interface 43.
[0198] For example, the mobile phone can display a text input box at the bottom of the chat interface. When the user selects a link and clicks to enter information in the text input box, the mobile phone can display a text input method in the chat interface. Then, if the user does not enter information in the text input box but directly clicks to enter information in the search control in the text input box, the search results interface can display "Sihaiweijia is located in District B of City A, 110 / person".
[0199] In some embodiments of this application, the electronic device can generate result information by combining the search information entered by the user in the information input control with the interface content of the program interface or the entity information of the second screen entity, thereby improving the flexibility of the electronic device in obtaining result information.
[0200] In some embodiments of this application, the display method provided in this application embodiment further includes the following steps 901 and 902, and the above step 802 can be specifically implemented by the following steps 802a and 802b.
[0201] Step 91: The electronic device receives the sixth input to the information input control.
[0202] In this embodiment of the application, the sixth input is used to trigger the electronic device to display search candidate words.
[0203] In some embodiments of this application, the sixth input can be a user clicking on the information input control using a touch device such as a finger or stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this application does not limit it.
[0204] For example, the sixth input mentioned above can be a user's click input on an information input control.
[0205] Step 902: The electronic device responds to the sixth input by displaying at least one search candidate word.
[0206] In some embodiments of this application, the above-mentioned at least one search candidate word is predicted based on the interface content of the program interface; or, the search candidate word is predicted based on the entity information of the second screen entity.
[0207] In some embodiments of this application, the search candidate terms can be one or more. The specific terminology can be determined based on actual usage requirements, and this application does not impose any limitations.
[0208] In some embodiments of this application, when the user does not select a second screen entity, the search candidate terms are generated based on the interface content of the program interface; when the user selects a second screen entity, the search candidate terms are generated based on the entity information of the second screen entity.
[0209] Step 802a: The electronic device receives a fourth input for one of the at least one search candidate words.
[0210] In some embodiments of this application, the fourth input is used to trigger the electronic device to fill the information input control with search candidate words.
[0211] In some embodiments of this application, the aforementioned fourth input may be a user's click input of a search candidate word through a touch device such as a finger or stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this application embodiment does not limit it.
[0212] For example, the fourth input mentioned above can be a user's click input for a search candidate word.
[0213] Step 802b: The electronic device displays the search candidate words selected by the fourth input in the information input control.
[0214] In some embodiments of this application, the electronic device may display search candidate words in the candidate word display area of the information input control.
[0215] For example, the candidate word display area mentioned above is preset by the electronic device; or it can be user-defined. The specific area can be determined according to actual usage needs, and this application embodiment does not impose any limitations.
[0216] In some embodiments of this application, after the electronic device receives a fourth input of search candidate words, the electronic device can automatically generate and display search results information based on the search candidate words and the interface content of the program interface.
[0217] In some embodiments of this application, the electronic device can automatically generate search candidate words based on the content of the program interface, without requiring manual input by the user, thereby improving the flexibility of the electronic device in performing input operations.
[0218] For example, such as Figure 15 As shown below, a specific example will be used to explain the display method provided in this application. Specifically, it can be implemented through steps 1 to 11 as described below.
[0219] Step 1: The electronic device activates the search function based on the user's first input on the program interface.
[0220] In some embodiments of this application, the electronic device can activate the circle search function by the user performing a two-finger long press input in the program interface; or, the electronic device can activate the circle search function by the user long pressing the bottom navigation key in the program interface; or, the electronic device can activate the circle search function through the circle search function component in the control center.
[0221] For example, a user can long-press the program interface with two fingers to make the electronic device display light and shadow animations, and then activate the circle search function; or, if there is a navigation bar at the bottom of the electronic device, the user can long-press the navigation bar to trigger the electronic device to activate the circle search function.
[0222] For example, the navigation bar mentioned above includes, but is not limited to, a small white bar and traditional three-button navigation.
[0223] Step 2: Identify at least one screen entity in the electronic device's program interface.
[0224] In some embodiments of this application, the electronic device can scan all screen entities in the program interface and then identify the type and boundaries of the executable screen entities.
[0225] For example, when an electronic device enables the search function, it can automatically identify executable screen entities in the program interface. This step is triggered automatically by the electronic device and does not require any user operation. The specific content that the electronic device can identify is as follows: identifying the type of executable screen entity, such as images, text, links, etc., and identifying the size, boundaries, and position of the executable screen entity.
[0226] Step 3: The electronic device displays the screen entity service component corresponding to each screen entity in the program interface.
[0227] For example, based on the identification in step 2, the electronic device can know which executable entities are in the program interface, as well as the size, boundaries, and positions of the executable screen entities. Therefore, the electronic device can generate corresponding recommendation services for the user based on the type of the executable screen entity, and the electronic device can display a screen entity service component above the executable screen entity based on the position of the executable screen entity.
[0228] For example, if an electronic device detects a question on the program interface, it can display a screen entity service component that says "Answer the question" on the question; as another example, if an electronic device detects an image of a piece of clothing on the program interface, it can display a screen entity service component that says "Search for similar items" on the image of the clothing.
[0229] Step 4: The electronic device receives a second input to the first screen entity service component.
[0230] In some embodiments of this application, the first screen entity service component described above is any one of all screen entity service components that respond to the first input display.
[0231] Step 5: The electronic device displays the service interface of the recommended service corresponding to the physical service component on the first screen.
[0232] For example, users can directly click on the recommendation service so that the electronic device can execute the recommendation service with one click; the execution method can be: opening the application corresponding to the recommendation service through a pop-up window, such as the "search for similar items" mentioned above. The electronic device can run the shopping application through a small window in the program interface and fill the clothing image in the program interface into the search for similar items function of the shopping application to help users complete all the necessary user interaction steps to search for a product.
[0233] In some embodiments of this application, steps 4 and 5 above can be referred to as concise interaction.
[0234] In some embodiments of this application, the above embodiments can enable users to see recommended services for multiple screen entities at once; however, for a single entity, the electronic device can only display one entity recommendation component to the user, which is insufficient for some complex entity types, and cannot meet the needs of users who want to perform in-depth service operations; therefore, the embodiments of this application design "focused interaction", as detailed in steps 6 and 7 below, recommending more service content for scenarios where users want to perform detailed operations on a certain screen entity.
[0235] Step 6: The electronic device can receive third input to the first screen entity in the program interface.
[0236] Step 7: In response to the third input, the electronic device displays at least one entity sub-service component corresponding to the first screen entity on the program interface.
[0237] For example, when a user clicks on an executable screen entity, the device enters a focused entity state. Based on the type of this executable screen entity, the electronic device can add some entity sub-service components. The intuitive interaction is that a single recommendation slot expands into multiple recommendation slots. For example, taking searching for clothes as an example, in the above embodiment, only one "A application search" may be recommended, while the expanded recommendations may simultaneously display "A application search", "B application search", "C application search", etc.
[0238] Step 8: The electronic device receives selection input for an entity subservice component.
[0239] Step 9: In response to the above selection input, the electronic device displays the service interface of the recommended service corresponding to an entity sub-service component.
[0240] Step 10: The electronic device receives the fourth input, in which the search information is entered on the information input control.
[0241] In some embodiments of this application, both of the above-mentioned interactions are based on the electronic device automatically recognizing screen entities in the program interface and automatically generating recommended content, which theoretically can meet the screen recognition needs of most users; however, considering that some users still have higher personalized requirements for program interface recognition and need to input text for searching, the electronic device also supports users to input information in the information input control; users can ask questions about the entire screen or a single entity, so that the electronic device can output corresponding search results according to the user's questions.
[0242] Step 11: The electronic device displays the search information in the program interface based on the search information.
[0243] For example, in the simplified interaction mode, users can directly click the information input control, enter the search information, and then click the send button. At this time, the electronic device will take a screenshot of the entire program interface, analyze it together with the user's search information, and finally display the search answer on the program interface.
[0244] For example, if a user wants to know how many people are in the current program interface, they can directly ask the question about the entire program interface through the simple interaction. After recognizing the entire program interface, the system will reply to the user in the top panel based on the user's question.
[0245] In focused interaction mode, since the user has previously selected the executable screen entity they want to focus on, the electronic device can also insert the selected executable screen entity into the input box, so that after entering the search information, the corresponding executable screen entity can be sent together for analysis.
[0246] Furthermore, this application also expands the text-based intelligent association recommendation interaction method: when the input box is pulled up, association recommendations based on entity content will be displayed above the keyboard to assist users in completing search input; for example, when the electronic device recognizes that the user has selected a news article, the electronic device can prompt the user whether they want to "summarize this article" or "find similar news reports" as association recommendations.
[0247] In some embodiments of this application, entity recognition based on screen content can more accurately understand the user's current needs, thereby providing services and recommended content that better match the user's intent. Furthermore, the recommended content is directly labeled on the recognized entities, allowing users to understand and use the recommended information in a more intuitive way, reducing learning costs and operational steps.
[0248] Furthermore, it allows users to manually adjust the entity recognition range and perform secondary recognition, providing them with greater flexibility and control, and enabling them to easily access more related services. It can meet deeper needs, identify users' potential requirements, and satisfy them through progressively deeper recommendations, helping users solve more complex problems.
[0249] The above-described method embodiments, or various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.
[0250] It should be noted that the display method provided in this application embodiment can be executed by a display device. This application embodiment uses a display device executing the display method as an example to illustrate the display device provided in this application embodiment.
[0251] Figure 16 A schematic diagram of a possible structure of the display device involved in an embodiment of this application is shown. For example... Figure 16 As shown, the display device 70 may include a receiving module 71 and a display module 72.
[0252] The receiving module 71 is used to receive a first input on the program interface. The display module 72 is used to respond to the first input received by the receiving module and display the screen entity service component corresponding to each screen entity in the program interface; wherein the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
[0253] In one possible implementation, the display device 70 further includes: a processing module, configured to identify at least one screen entity in the program interface; and based on the entity information of each screen entity, determine at least one service key step information corresponding to each screen entity; one service key step information corresponding to one screen entity service component; and based on the service key step information corresponding to each screen entity, determine the recommended service corresponding to each screen entity service component; wherein the entity information includes: entity type and entity content.
[0254] In one possible implementation, when the screen entity is a session message, the entity information includes: session message content; when the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, link navigation information; when the screen entity is an image, the entity information includes: image content; when the screen entity is a video, the entity information includes: video content, video subtitles; when the screen entity is a component, the entity information includes: component name, component function, component type.
[0255] In one possible implementation, the receiving module 71 is further configured to, after the display module 72 displays the screen entity service component corresponding to each screen entity in the program interface, receive a second input to the first screen entity service component, wherein the first screen entity service component is any one of all screen entity service components displayed in response to the first input. The display module 72 is further configured to, in response to the second input received by the receiving module 71, display the service interface of the recommended service corresponding to the first screen entity service component.
[0256] In one possible implementation, the above-mentioned program interface also includes at least one global service component, which is generated based on the interface content of the program interface.
[0257] In one possible implementation, when the program interface is a conversational interface, at least one global service component includes at least one of the following: a group chat service component, an add contact service component, and a text editing service component; when the program interface is a document interface, at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component; when the program interface is a shopping interface, at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object evaluation service component.
[0258] In one possible implementation, the receiving module 71 is further configured to, after displaying the screen entity service component corresponding to each screen entity in the program interface, receive a third input to the first screen entity in the program interface, wherein the first screen entity is any one of at least one screen entity. The display module is further configured to, in response to the third input received by the receiving module 71, display at least one entity sub-service component corresponding to the first screen entity in the program interface; wherein the recommendation service corresponding to at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
[0259] In one possible implementation, when the first screen entity is an image, the recommendation service corresponding to at least one entity sub-service component is determined based on the image content of the image; when the first screen entity is a video, the recommendation service corresponding to at least one entity sub-service component is determined based on the video content and video subtitles of the video; when the first screen entity is a session message, the recommendation service corresponding to at least one entity sub-service component is determined based on the message content of the session message; when the first screen entity is a link, the recommendation service corresponding to at least one entity sub-service component is determined based on the link content of the link; and when the first screen entity is a component, the recommendation service corresponding to at least one entity sub-service component is determined based on the component content contained in the component.
[0260] In one possible implementation, the display module 73 is further configured to, after the receiving module receives a third input to the first screen entity in the program interface, respond to the third input by canceling the display of all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity; wherein the third screen entity service component includes at least one entity sub-service component and a second screen entity service component.
[0261] In one possible implementation, the program interface further includes an information input control; the display module 72 is also configured to, upon receiving a fourth input of search information entered on the information input control, display the search information on the information input control and display result information in the program interface, wherein the result information is obtained by searching based on the search information and the interface content of the program interface; or, upon receiving a fifth input to a second screen entity in at least one screen entity and a fourth input of search information entered on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
[0262] In one possible implementation, the receiving module 71 is further configured to receive a sixth input to the information input control. The display module 72 is further configured to, in response to the sixth input, display at least one search candidate word, wherein the at least one search candidate word is predicted based on the interface content of the program interface; or, the search candidate word is predicted based on entity information of entities on the second screen. Specifically, the receiving module 71 is configured to receive a fourth input to one of the at least one search candidate words. Specifically, the display module 72 is configured to display the search candidate word selected by the fourth input in the information input control.
[0263] This application provides a display device that can display screen entity service components corresponding to all screen entities through a first input on the program interface. Therefore, users do not need to select and input each screen entity, reducing user interaction operations when using the service. Moreover, by recognizing entities on the program interface, the current needs of users can be understood more accurately, thereby providing recommendation services that are more in line with user intentions.
[0264] The display device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.
[0265] The display device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.
[0266] The display device provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.
[0267] Optionally, such as Figure 17 As shown, this application embodiment also provides an electronic device 90, including a processor 91 and a memory 92. The memory 92 stores a program or instructions that can run on the processor 91. When the program or instructions are executed by the processor 91, they implement the various steps of the above-described display method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0268] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0269] Figure 18 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.
[0270] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.
[0271] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 18 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0272] The user input unit 107 is used to receive a first input on the program interface. The display unit 106 is used to respond to the first input and display the screen entity service component corresponding to each screen entity in the program interface; wherein the program interface includes at least one screen entity, the screen entity service component corresponds to a recommendation service, and the recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
[0273] In some embodiments of this application, the processor 110 is further configured to identify at least one screen entity in the program interface; and based on the entity information of each screen entity, determine at least one service key step information corresponding to each screen entity; one service key step information corresponding to one screen entity service component; and based on the service key step information corresponding to each screen entity, determine the recommended service corresponding to each screen entity service component; wherein, the entity information includes: entity type and entity content.
[0274] In some embodiments of this application, the user input unit 107 is further configured to receive a second input to the first screen entity service component after displaying the screen entity service component corresponding to each screen entity in the program interface, wherein the first screen entity service component is any one of all screen entity service components displayed in response to the first input. The display unit 106 is further configured to display the service interface of the recommended service corresponding to the first screen entity service component in response to the second input.
[0275] In some embodiments of this application, the user input unit 107 is further configured to receive a third input on a first screen entity in the program interface after displaying the screen entity service component corresponding to each screen entity in the program interface, wherein the first screen entity is any one of at least one screen entity. The display unit 106 is further configured to, in response to the third input, display at least one entity sub-service component corresponding to the first screen entity in the program interface; wherein the recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
[0276] In some embodiments of this application, the display unit 106 is further configured to, after receiving a third input to the first screen entity in the program interface, respond to the third input by canceling the display of all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity; wherein, the third screen entity service component includes at least one entity sub-service component and a second screen entity service component.
[0277] In some embodiments of this application, the program interface further includes an information input control; the display unit 106 is also configured to, upon receiving a fourth input of search information entered on the information input control, display the search information on the information input control and display result information in the program interface, wherein the result information is obtained by searching based on the search information and the interface content of the program interface; or, upon receiving a fifth input of search information on a second screen entity in at least one screen entity and a fourth input of search information entered on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
[0278] In some embodiments of this application, the user input unit 107 is further configured to receive a sixth input to the information input control. The display unit 106 is further configured to, in response to the sixth input, display at least one search candidate word, wherein the at least one search candidate word is predicted based on the interface content of the program interface; or, the search candidate word is predicted based on entity information of a second screen entity. Specifically, the user input unit 107 is configured to receive a fourth input to one of the at least one search candidate words. Specifically, the display unit 106 is configured to display the search candidate word selected by the fourth input in the information input control.
[0279] This application provides an electronic device that can display screen entity service components corresponding to all screen entities through a first input on the program interface. Therefore, the user does not need to select and input each screen entity, reducing the user interaction operations when using the service. Moreover, by recognizing entities on the program interface, the user's current needs can be understood more accurately, thereby providing recommendation services that are more in line with the user's intentions.
[0280] The electronic device provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0281] For details on the beneficial effects of the various implementation methods in this embodiment, please refer to the beneficial effects of the corresponding implementation methods in the above method embodiments. To avoid repetition, these will not be repeated here.
[0282] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.
[0283] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.
[0284] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.
[0285] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0286] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0287] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0288] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0289] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes shown in the above-described method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be described again here.
[0290] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0291] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0292] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A display method, characterized in that, The method includes: Receive the first input from the program interface; In response to the first input, display the screen entity service component corresponding to each screen entity in the program interface; The program interface includes at least one screen entity, and the screen entity service component corresponds to a recommendation service. The recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
2. The method according to claim 1, characterized in that, The method further includes: Identify at least one screen entity in the program interface; Based on the entity information of each screen entity in the at least one screen entity, at least one key service step information corresponding to each screen entity is determined; one screen entity service component corresponds to one key service step information. Based on the service key step information corresponding to each screen entity, the recommended service corresponding to each screen entity service component is determined. The entity information includes: entity type and entity content.
3. The method according to claim 2, characterized in that, When the screen entity is a session message, the entity information includes: session message content; When the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, and link navigation information; When the screen entity is an image, the entity information includes: image content; When the screen entity is a video, the entity information includes: video content and video subtitles; When the screen entity is a component, the entity information includes: component name, component function, and component type.
4. The method according to claim 1, characterized in that, After displaying the screen entity service component corresponding to each screen entity in the program interface, the method further includes: Receive a second input to a first screen entity service component, wherein the first screen entity service component is any one of all screen entity service components that are displayed in response to the first input; In response to the second input, the service interface of the recommended service corresponding to the first screen entity service component is displayed.
5. The method according to claim 1, characterized in that, The program interface also includes at least one global service component, which is generated based on the interface content of the program interface.
6. The method according to claim 5, characterized in that, The method further includes: When the program interface is a conversational interface, the at least one global service component includes at least one of the following: a group chat service component, an add contact service component, and a text editing service component; When the program interface is a document interface, the at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component; When the program interface is a shopping interface, the at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object evaluation service component.
7. The method according to any one of claims 1 to 6, characterized in that, After displaying the screen entity service component corresponding to each screen entity in the program interface, the method further includes: Receive a third input to a first screen entity in the program interface, wherein the first screen entity is any one of the at least one screen entities; In response to the third input, at least one entity sub-service component corresponding to the first screen entity is displayed on the program interface; The recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
8. The method according to claim 7, characterized in that, When the first screen entity is an image, the recommendation service corresponding to the at least one entity sub-service component is determined based on the image content of the image; When the first screen entity is a video, the recommendation service corresponding to the at least one entity sub-service component is determined based on the video content and video subtitles of the video; When the first screen entity is a session message, the recommendation service corresponding to the at least one entity sub-service component is determined based on the message content of the session message; When the first screen entity is a link, the recommendation service corresponding to the at least one entity sub-service component is determined based on the link content; When the first screen entity is a component, the recommendation service corresponding to the at least one entity sub-service component is determined based on the component content contained in the component.
9. The method according to claim 7, characterized in that, After receiving the third input to the first screen entity in the program interface, the method further includes: In response to the third input, in the program interface, all screen entity service components except the third screen entity service component corresponding to the first screen entity are cancelled from the program interface. The third screen entity service component includes the at least one entity sub-service component and the second screen entity service component.
10. The method according to claim 1, characterized in that, The program interface also includes information input controls; the method further includes: Upon receiving a fourth input of search information on the information input control, the search information is displayed on the information input control, and result information is displayed in the program interface. The result information is obtained based on the search information and the interface content of the program interface. or, When a fifth input is received for a second screen entity among the at least one screen entities, and a fourth input is received for search information on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
11. The method according to claim 10, characterized in that, The method further includes: Receive a sixth input to the information input control; In response to the sixth input, at least one search candidate word is displayed, the at least one search candidate word being predicted based on the interface content of the program interface; or, the search candidate word being predicted based on the entity information of the second screen entity. The receiving of the fourth input, which involves entering search information into the information input control, includes: Receive a fourth input for one of the at least one search candidate words; Displaying the search information in the information input control includes: The information input control displays the search candidate words selected by the fourth input.
12. A display device, characterized in that, The display device includes: The receiving module is used to receive the first input from the program interface; The display module is used to display the screen entity service component corresponding to each screen entity in the program interface in response to the first input received by the receiving module. The program interface includes at least one screen entity, and the screen entity service component corresponds to a recommendation service. The recommendation service corresponding to the screen entity service component is generated based on the entity information of a screen entity.
13. The apparatus according to claim 12, characterized in that, The display device further includes: The processing module is configured to identify at least one screen entity in the program interface; and based on the entity information of each screen entity, determine at least one key service step information corresponding to each screen entity; one key service step information corresponding to one screen entity service component; and based on the key service step information corresponding to each screen entity, determine the recommended service corresponding to each screen entity service component; wherein, the entity information includes: entity type and entity content.
14. The apparatus according to claim 13, characterized in that, When the screen entity is a session message, the entity information includes: session message content; When the screen entity is a link, the entity information includes at least one of the following: link text information, link image information, link audio information, link video information, and link navigation information; When the screen entity is an image, the entity information includes: image content; When the screen entity is a video, the entity information includes: video content and video subtitles; When the screen entity is a component, the entity information includes: component name, component function, and component type.
15. The apparatus according to claim 12, characterized in that, The receiving module is further configured to receive a second input to the first screen entity service component after the display module displays the screen entity service component corresponding to each screen entity in the program interface, wherein the first screen entity service component is any one of all screen entity service components displayed in response to the first input; The display module is further configured to display the service interface of the recommended service corresponding to the first screen entity service component in response to the second input received by the receiving module.
16. The apparatus according to claim 12, characterized in that, The program interface also includes at least one global service component, which is generated based on the interface content of the program interface.
17. The apparatus according to claim 16, characterized in that, When the program interface is a conversational interface, the at least one global service component includes at least one of the following: a group chat service component, an add contact service component, and a text editing service component; When the program interface is a document interface, the at least one global service component includes at least one of the following: a full-text translation service component, a text extraction service component, and a summary extraction service component; When the program interface is a shopping interface, the at least one global service component includes at least one of the following: an image editing service component, an object search service component, and an object evaluation service component.
18. The apparatus according to any one of claims 12 to 17, characterized in that, The receiving module is further configured to receive a third input to a first screen entity in the program interface after displaying the screen entity service component corresponding to each screen entity in the program interface, wherein the first screen entity is any one of the at least one screen entity. The display module is further configured to, in response to the third input received by the receiving module, display at least one entity sub-service component corresponding to the first screen entity on the program interface; The recommendation service corresponding to the at least one entity sub-service component is generated based on the entity information of the first screen entity and the recommendation service corresponding to the second screen entity service component; the second screen entity service component is the screen entity service component corresponding to the first screen entity, and the recommendation service corresponding to the at least one entity sub-service component is a sub-service of the recommendation service corresponding to the second screen entity service component.
19. The apparatus according to claim 18, characterized in that, When the first screen entity is an image, the recommendation service corresponding to the at least one entity sub-service component is determined based on the image content of the image; When the first screen entity is a video, the recommendation service corresponding to the at least one entity sub-service component is determined based on the video content and video subtitles of the video; When the first screen entity is a session message, the recommendation service corresponding to the at least one entity sub-service component is determined based on the message content of the session message; When the first screen entity is a link, the recommendation service corresponding to the at least one entity sub-service component is determined based on the link content; When the first screen entity is a component, the recommendation service corresponding to the at least one entity sub-service component is determined based on the component content contained in the component.
20. The apparatus according to claim 18, characterized in that, The display module is further configured to, in response to the third input received by the receiving module for the first screen entity in the program interface, cancel the display of all screen entity service components in the program interface except for the third screen entity service component corresponding to the first screen entity; wherein, the third screen entity service component includes the at least one entity sub-service component and the second screen entity service component.
21. The apparatus according to claim 12, characterized in that, The program interface also includes an information input control; the display module is further configured to, upon receiving a fourth input of search information on the information input control, display the search information on the information input control and display result information in the program interface, wherein the result information is obtained based on the search information and the interface content of the program interface; or, When a fifth input is received for a second screen entity among the at least one screen entities, and a fourth input is received for search information on the information input control, the result information is obtained by searching based on the search information and the entity information of the second screen entity.
22. The apparatus according to claim 21, characterized in that, The receiving module is also used to receive a sixth input to the information input control; The display module is further configured to, in response to the sixth input, display at least one search candidate word, the at least one search candidate word being predicted based on the interface content of the program interface; or, the search candidate word being predicted based on the entity information of the second screen entity; The receiving module is specifically configured to receive a fourth input for one of the at least one search candidate words; The display module is specifically used to display the search candidate words selected by the fourth input in the information input control.
23. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the display method as described in any one of claims 1 to 11.
