Content search method and apparatus, and electronic device and storage medium

Abstract

A content search method and apparatus, and an electronic device and a storage medium. The method comprises: displaying a first interface; and in response to a detected search instruction, displaying a second interface, wherein the second interface comprises a search result obtained after performing, on a target referent, a search corresponding to the search instruction, and the target referent is content referred to by an entity mention in the search instruction that is determined from the first interface. The present method can make it convenient for a user to search for content in an interface.

Description

Content search method and device, electronic device, and storage medium

[0001] Cross-reference to Related Applications

[0002] This application claims priority to Chinese application No. 202411233651.5, filed on September 03, 2024, which is hereby incorporated by reference in its entirety for all purposes. TECHNICAL FIELD

[0003] The present application relates to the technical field of electronic devices, and more particularly, to a content search method and device, an electronic device, and a storage medium. BACKGROUND

[0004] With the rapid progress of technology and living standards, electronic devices (such as smart phones, tablet computers, etc.) have become one of the commonly used electronic products in people's lives. At present, when people use electronic devices, there are usually scenarios of searching for content in the display interface. SUMMARY

[0005] The present application provides a content search method and device, an electronic device, and a storage medium.

[0006] In a first aspect, an embodiment of the present application provides a content search method applied to an electronic device, the method comprising: displaying a first interface; in response to a detected search instruction, displaying a second interface, the second interface including search results obtained after performing a search corresponding to the search instruction on target referred content, the target referred content being content referred to by an entity reference in the search instruction determined from the first interface.

[0007] In a second aspect, an embodiment of the present application provides a content search device, the device comprising: a first interface display module and a second interface display module, wherein the first interface display module is configured to display a first interface; the second interface display module is configured to display a second interface in response to a detected search instruction, the second interface including search results obtained after performing a search corresponding to the search instruction on target referred content, the target referred content being content referred to by an entity reference in the search instruction determined from the first interface.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; and one or more application programs stored in the memory and configured to be executed by the one or more processors, the one or more application programs being configured to perform the content search method provided in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores program codes, and the program codes can be invoked by a processor to execute the content search method provided in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0011] FIG. 1 shows a flow diagram of a content search method according to an embodiment of the present application.

[0012] FIG. 2 shows an interface diagram provided by an embodiment of the present application.

[0013] FIG. 3 shows an interface diagram provided by an embodiment of the present application.

[0014] FIG. 4 shows a flow diagram of a content search method according to another embodiment of the present application.

[0015] FIG. 5 shows an interface diagram provided by an embodiment of the present application.

[0016] FIG. 6 shows a flow diagram of a content search method according to yet another embodiment of the present application.

[0017] FIG. 7 shows an interface diagram provided by an embodiment of the present application.

[0018] FIG. 8 shows an interface diagram provided by an embodiment of the present application.

[0019] FIG. 9 shows a flow diagram of a content search method according to still another embodiment of the present application.

[0020] FIG. 10 shows a flow diagram of a content search method according to yet another embodiment of the present application.

[0021] FIG. 11 shows an interface diagram provided by an embodiment of the present application.

[0022] FIG. 12 shows an interface diagram provided by an embodiment of the present application.

[0023] FIG. 13 shows an interface diagram provided by an embodiment of the present application.

[0024] FIG. 14 shows an interface diagram provided by an embodiment of the present application.

[0025] FIG. 15 shows a schematic diagram of an interface according to an embodiment of the present application.

[0026] FIG. 16 shows a schematic diagram of an interface according to an embodiment of the present application.

[0027] FIG. 17 shows a schematic diagram of an interface according to an embodiment of the present application.

[0028] FIG. 18 shows a flowchart of a content search method according to yet another embodiment of the present application.

[0029] FIG. 19 shows a flowchart of a content search method according to still another embodiment of the present application.

[0030] FIG. 20 shows a block diagram of a content search apparatus according to an embodiment of the present application.

[0031] FIG. 21 is a block diagram of an electronic device for performing a content search method according to an embodiment of the present application.

[0032] FIG. 22 is a storage unit for storing or carrying program codes for implementing a content search method according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make persons skilled in the art better understand the schemes of the present application, the technical schemes in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0034] At present, in some use scenarios of using electronic devices, there is a demand for searching or querying text, image content and the like in interfaces that are of interest to users. In the related art, when a user searches for content in an interface, the user usually needs to copy the content in the interface and then switch to a related application to search for the content. Moreover, when the content to be searched for is image content, the user also needs to take a screenshot of the interface and crop out the corresponding image region and then switch to a related application to search for the content. In such a manner, the user usually needs to perform multiple operations to view the search result that meets the demand, and thus the operation is relatively cumbersome.

[0035] At present, some electronic devices provide a smart assistant, and after invoking the smart assistant, the user can input a search instruction to the smart assistant to search for content. However, the user needs to explicitly specify the keyword to be searched for in the instruction. In the case where the instruction input by the user does not contain the keyword but contains an entity reference to the keyword, the smart assistant cannot understand the interface content that the user wants to search for. Moreover, in the case where the text content to be searched for by the user is long, involves foreign languages, or involves rare characters, the user also has inconvenience when inputting the instruction by typing or voice input.

[0036] To solve the above problems, the content search method, device, electronic device and storage medium provided in the embodiments of the present application can directly determine the content referred to by the entity reference in the search instruction from the interface based on the search instruction of the user, and perform corresponding search on the referred content, so that the user can search the corresponding content in the interface only by inputting the search instruction, thereby simplifying the operation of the user when searching the content in the interface and improving the usability of the electronic device. The specific content search method is described in detail in subsequent embodiments.

[0037] The embodiments of the present application provide a content search method applied to an electronic device, the method comprising: displaying a first interface; in response to a detected search instruction, displaying a second interface, wherein the second interface comprises search results obtained by performing a search corresponding to the search instruction on target referred content, and the target referred content is content referred to by an entity reference in the search instruction from the first interface.

[0038] According to the embodiments of the present application, in response to the detected search instruction, the second interface is displayed, comprising: determining the target referred content from the first interface based on the search instruction in response to the search instruction; displaying the second interface after performing the search corresponding to the search instruction on the target referred content.

[0039] According to the embodiments of the present application, the target referred content is determined from the first interface based on the search instruction, comprising: determining the entity reference in the search instruction and the intent information corresponding to the search instruction; determining the content referred to by the entity reference as the target referred content from the first interface based on the intent information.

[0040] According to the embodiments of the present application, the content referred to by the entity reference is determined from the first interface as the target referred content based on the intent information, comprising: obtaining an identification result obtained by performing content identification on an interface image of the first interface; determining the content matched with the intent information and the entity reference from the identification result as the target referred content.

[0041] According to the embodiments of the present application, the content referred to by the entity reference is determined from the first interface as the target referred content based on the intent information, comprising: determining the content referred to by the entity reference from the first interface based on the intent information; if the content referred to by the entity reference comprises multiple items of content, displaying candidate information corresponding to the multiple items of content in the interface; in response to a selection operation on target candidate information, determining the content corresponding to the target candidate information as the target referred content.

[0042] According to an embodiment of the present application, if the entity refers to multiple contents, the method further comprises: displaying a frame selection area for each of the multiple contents, and each frame selection area is a candidate information.

[0043] According to an embodiment of the present application, after the search corresponding to the search instruction is performed on the target referred content, the second interface is displayed, and the method further comprises: if the target referred content is the text content in the first interface, the target referred content is filled into the search instruction, and then the search corresponding to the search instruction is performed.

[0044] According to an embodiment of the present application, the search corresponding to the search instruction is performed on the target referred content, and the method further comprises: if the target referred content is the image content in the first interface, a region image of a region where the image content is located is obtained from an interface image of the first interface; and the search corresponding to the search instruction is performed on the region image.

[0045] According to an embodiment of the present application, before the second interface is displayed after the search corresponding to the search instruction is performed on the target referred content, the method further comprises: displaying first prompt information in the interface, and the first prompt information is used to prompt that the target referred content is being determined from the interface.

[0046] According to an embodiment of the present application, the first interface corresponds to a first application, and the second interface is displayed in response to the detected search instruction, and the method further comprises: in response to the search instruction, a second application is invoked, and after the search corresponding to the search instruction is performed on the target referred content by using the second application, a second interface corresponding to the second application is displayed, and the second interface is an interface displayed by the second application for the search result.

[0047] According to an embodiment of the present application, before the second application is invoked to perform the search corresponding to the search instruction on the target referred content, the method further comprises: determining the second application to be invoked based on the search instruction and a type of the target referred content.

[0048] According to an embodiment of the present application, the second application to be invoked is determined based on the search instruction and the type of the target referred content, and the method further comprises: determining a second application matched with the intent information corresponding to the search instruction and the type of the target referred content from a knowledge base based on the intent information and the type of the target referred content, and the knowledge base comprises application knowledge corresponding to applications installed in the electronic device.

[0049] According to an embodiment of the present application, the determining, from the knowledge base, the second application matching the intent information corresponding to the search instruction and the type of the target reference content comprises: determining, from the knowledge base, the application matching the intent information and the type of the target reference content based on the intent information corresponding to the search instruction and the type of the target reference content; and determining the second application from the multiple applications based on usage data of each of the multiple applications, the usage data comprising at least one of a usage frequency of the application, a usage times of the application, and a target search times, the target search times being a search times of the application used for a search task matching the intent information and the type of the target reference content.

[0050] According to an embodiment of the present application, after the invoking the second application, the method further comprises: displaying second prompt information in the interface, the second prompt information being used for prompting that the search corresponding to the search instruction is being performed by the second application.

[0051] According to an embodiment of the present application, after the invoking the second application, the method further comprises: displaying second prompt information in the interface, the second prompt information being used for prompting that the search corresponding to the search instruction is being performed by the second application.

[0052] According to an embodiment of the present application, the first application corresponds to the first application, and the displaying the second interface in response to the detected search instruction comprises: in response to the search instruction, invoking the second application and performing the search corresponding to the search instruction on the target reference content by the second application, and then displaying the second interface, the second interface comprising a first split-screen area and a second split-screen area, the first split-screen area comprising the first interface, and the second split-screen area comprising a third interface displayed by the second application for the search result, the search result being obtained by performing the search by the second application.

[0053] According to an embodiment of the present application, the second interface is an interface in which the result display area is superimposed on the first interface, and the displaying the second interface in response to the detected search instruction comprises: in response to the detected search instruction, superimposing the result display area on the first interface, the result display area comprising the search result.

[0054] The content search method provided by the embodiment of the present application will be described in detail below in combination with the drawings.

[0055] Please refer to FIG. 1, which shows a flowchart of the content search method provided by an embodiment of the present application. In a specific embodiment, the content search method is applied to the content search device 700 shown in FIG. 20 and the electronic device 100 (FIG. 21) configured with the content search device 700. In the following, the specific process of the embodiment will be described by taking the electronic device as an example. It can be understood that the electronic device to which the embodiment is applied can be a smart phone, a tablet computer, an e-book, a notebook computer, a smart watch, etc., which is not limited here. The flow shown in FIG. 1 will be described in detail below. The content search method can specifically include the following steps:

[0056] Step S110: Display a first interface.

[0057] The first interface can be any interface displayed by the electronic device, for example, a document interface, a browser interface, a video playing interface, a social application interface, etc., and the specific interface is not limited. It can be understood that in different interfaces displayed by the electronic device, the user can have the demand of searching the content in the interface, and therefore the content search method provided by the embodiment of the present application can be executed in the case that any interface is displayed by the electronic device.

[0058] In the embodiment of the present application, the electronic device can detect the instruction input by the user in the case that the first interface is displayed, so as to determine the corresponding content from the first interface based on the search instruction in the case that the search instruction is detected, and then perform the corresponding search for the determined content. In the case that the first interface is displayed, the electronic device can detect the voice instruction input by the user, and can also detect the instruction input in the first interface.

[0059] Step S120: In response to the detected search instruction, display a second interface, which includes the search result obtained by performing the search corresponding to the search instruction for the target referred content, which is the content referred by the entity mentioned in the search instruction and determined from the first interface.

[0060] In the embodiments of the present application, when the electronic device displays the first interface, after detecting the search instruction, the electronic device can perform the search corresponding to the search instruction on the content referred to by the entity reference in the search instruction in response to the search instruction, and display the second interface, so as to display the search result after performing the search corresponding to the search instruction in the second interface. In this way, without additional operations of the user, only by inputting the search instruction, the search task on the interface content required by the user can be performed according to the user's requirement, and the search result can be displayed. The entity reference is a name of a referred object, including pronouns (for example, "he", "they", "this", "this family"), nouns (for example, "capital", "provincial capital", "shoes", "clothes", "chair", "mobile phone"), and the like, and the content referred to by the entity reference is the object referred to by the entity reference.

[0061] In some embodiments, the electronic device can collect audio in the environment through the audio collection device, and identify whether the audio includes the search instruction according to the collected audio. Optionally, the electronic device can detect whether the collected audio includes speech; in the case where the collected audio includes speech, the speech in the collected audio is converted into text, and whether the text includes the search instruction is determined based on the obtained text; in the case where the collected audio does not include speech, it is indicated that the user does not input the instruction by voice, and therefore subsequent processing can not be performed. For example, after the electronic device converts the detected speech into text, the obtained text content is "search for the price of these shoes", and after performing semantic recognition on the text content, it is determined that the text content includes the search instruction.

[0062] In some embodiments, the electronic device can also display a virtual keyboard in the first interface, and detect the touch operation in the first interface; when detecting the touch operation through the virtual keyboard, whether the search instruction is input according to the input operation through the virtual keyboard is determined. For example, whether the text input through the virtual keyboard includes the search instruction can be determined according to the text input through the virtual keyboard.

[0063] In a possible implementation, the electronic device can be installed with a smart assistant, which can be an application based on artificial intelligence technology. The smart assistant is generally used to provide various services and assistance to the user. The smart assistant generally has the capabilities of speech recognition, natural language processing, and machine learning, and can have a conversation with the user and understand the user's needs, and then provide corresponding information, suggestions, or perform tasks. When the electronic device displays the above first interface, after detecting a calling operation for the smart assistant, the smart assistant can be controlled to be in an awake state in response to the calling operation, and an interaction area of the smart assistant can be displayed in the first interface, so that the user can input the above search instruction in a voice input manner through the smart assistant, or call a virtual keyboard in the above interaction area and input the above search instruction by using the virtual keyboard.

[0064] Exemplarily, referring to FIGS. 2 and 3, when the electronic device displays the first interface A1 as shown in FIG. 2, after detecting a calling operation for the smart assistant, the electronic device can display an interface as shown in FIG. 3, so that an interaction area A2 of the smart assistant is displayed in the interface, and the input content of the user input to the smart assistant can be displayed in the interaction area A2 after being detected.

[0065] In some embodiments, after detecting the above search instruction, the electronic device can perform semantic analysis on the search instruction when responding to the search instruction, and determine whether an entity reference is included in the search instruction according to the analysis result. If the search instruction includes an entity reference, the search instruction corresponding search can be performed on the content referred to by the entity reference in the first interface, and the second interface can be displayed.

[0066] In a possible implementation, after determining whether an entity reference is included in the above search instruction, if the search instruction does not include an entity reference and includes a keyword to be searched, it indicates that the search required by the user this time does not depend on the content in the display interface, and therefore the search instruction corresponding search can be directly performed on the keyword. For example, the search instruction is "search the price of A brand clothes", and the keyword to be searched included in the search instruction is "A brand clothes", so the price of "A brand clothes" can be directly searched to obtain the search result and display the search result. In addition, if the search instruction does not include an entity reference and does not include a keyword to be searched, the electronic device can output a prompt information to prompt the user to re-input the instruction.

[0067] In some embodiments, when the electronic device determines the content referred to by the above entity designation from the first interface, the electronic device can analyze and understand the content in the first interface, such as recognizing and parsing various elements such as text, images, icons, etc., and understand the intent of the above search instruction, and then determine the content referred to by the above entity designation (the content matching the user's intent and the content the user actually wants to select) according to the results of analyzing and understanding the first interface and the results of understanding the intent of the search instruction.

[0068] In some embodiments, when the electronic device performs the search corresponding to the search instruction on the target designation content, the application corresponding to the first interface can not be able to perform the search, so the electronic device can call other applications to perform the search and obtain the search results.

[0069] In a possible implementation, the application corresponding to the first interface can be a first application, and the application used to perform the search can be a second application. When the electronic device displays the search results obtained by performing the search corresponding to the search instruction on the target designation content by the second application, the electronic device can superimpose a result display area on the first interface, and the result display area includes the above search results. In other words, the second interface displayed by the electronic device can be the interface obtained by superimposing the result display area on the first interface.

[0070] Optionally, the electronic device can display a floating window in the interface, and the floating window is used to display the result display area. The electronic device can also detect an operation in the interface, and switch the search results displayed in the result display area according to the operation in the floating window, so as to display different search results in the result display area. The floating window is also called a suspended window. The floating window is a window displayed in the screen of the electronic device. The floating window is used to display the application interface of the running application program. The application interface displayed in the floating window is a scaled-down application interface, which can include all interface elements in the application interface. Alternatively, the floating window displays the core interface elements in the application interface, which are pre-set by the application.

[0071] Optionally, when the smart assistant of the electronic device is in an awake state and the first interface displays an interaction area of the smart assistant, the electronic device can display the interaction area as a conversation area of the smart assistant, and display the result display area in the conversation area, so as to display the search results in the conversation area of the smart assistant.

[0072] Optionally, after displaying the above result display area in the interface, the electronic device can switch the currently displayed interface from the second interface to the interface corresponding to the second application in response to detecting a touch operation on the search result in the result display area, for example, detecting a click operation on one of the results. The interface after switching can include the above search result, thereby realizing direct display of the search result in the application interface of the second application in which the search task is performed this time, so as to facilitate the user to perform relevant operations on the search result. For example, in the case where the second application is a shopping application, the user can conveniently purchase the searched goods in the interface after switching.

[0073] The content search method provided by the embodiments of the present application can determine the content referred to by the entity reference in the search instruction based on the search instruction of the user from the interface currently displayed by the electronic device, and perform corresponding search on the referred content, so that the user can search the corresponding content in the display interface only by inputting the search instruction, thereby simplifying the operation of the user when searching the content in the display interface and improving the usability of the electronic device.

[0074] Please refer to FIG. 4, which shows a flowchart of a content search method provided by another embodiment of the present application. The content search method is applied to the above-mentioned electronic device, and the flowchart shown in FIG. 4 will be described in detail as follows. Specifically, the content search method can include the following steps:

[0075] Step S210: display a first interface.

[0076] In the embodiments of the present application, step S210 can refer to the content of other embodiments, which will not be described here again.

[0077] Step S220: in response to the detected search instruction, determining target reference content from the first interface based on the search instruction.

[0078] The target reference content is the content referred to by the entity reference in the search instruction.

[0079] In the embodiments of the present application, in response to the above search instruction while displaying the first interface, the electronic device can determine the target reference content referred to by the entity reference in the search instruction, for example, the pronoun in the search instruction, from the first interface after determining the entity reference in the search instruction, so as to realize the search task of the user demand this time according to the determined target reference content.

[0080] In some embodiments, when the electronic device determines the content referred to by the entity designation in the search instruction from the first interface based on the above search instruction, the electronic device can determine the entity designation in the search instruction and the intent information corresponding to the search instruction; and then determine the content referred to by the entity designation from the first interface based on the intent information as the target referred content.

[0081] In the above embodiments, the entity designation can be obtained after entity recognition is performed on the above search instruction. When the entity designation in the above search instruction is determined, the search instruction can be subjected to word segmentation processing, the text corresponding to the search instruction is divided into individual independent lexical units, and part-of-speech tagging is performed to distinguish different types of words such as nouns, pronouns, verbs, adjectives, etc.; and then the entity designation existing in the word segmentation result is identified.

[0082] In the above embodiments, the intent information can be obtained after intent recognition is performed on the search instruction. Intent recognition is an important task in natural language processing (NLP) and speech processing, and its goal is to identify the true intent behind user input (whether text or speech), which can be to execute a specific command (such as “play music”), query certain information (such as “what’s the weather tomorrow”), express certain emotions or needs (such as “I need help”), conduct certain transactions (such as “buy this product”), etc. Alternatively, the above intent information can be obtained by performing intent recognition on the above search instruction through a pre-trained intent recognition model.

[0083] In one possible implementation, when the electronic device determines the content referred to by the entity designation from the first interface based on the intent information, the electronic device can obtain a recognition result obtained by performing content recognition on an interface image of the first interface; and then the electronic device can determine the content matching the intent information and the entity designation from the recognition result as the target referred content. The interface image of the first interface can be an image obtained by taking a screenshot of the first interface, and the content recognition on the interface image can be recognition of text, images or other elements in the interface image.

[0084] Exemplarily, the text corresponding to the above search instruction is "what is the price of this computer", the entity reference in the search instruction is "this computer", and the intent information is "search for the price", so as to search for the price of "this computer", the name or picture of the entity reference "this computer" needs to be known, therefore, the content of the interface image is identified, and after the identification result is obtained, it can be determined whether the name or picture of the computer exists in the identification result; if the name of the computer exists in the identification result, the name of the computer can be determined as the content referred to by "this computer"; if the picture of the computer exists in the identification result, the picture of the computer can be determined as the content referred to by "this computer".

[0085] Of course, in the embodiment of the application, the specific manner in which the electronic device determines the target reference content from the first interface based on the search instruction can not be limited, for example, the electronic device can also use a large language model (Large Language Model, LLM), and determine the target reference content based on the text corresponding to the above search instruction and the above interface image.

[0086] In some embodiments, in the process of determining the above target reference content in response to the above search instruction, the electronic device can also display first prompt information in the interface, the first prompt information being used to prompt that the above target reference content is being determined from the interface, thereby facilitating the user to know the processing stage of the electronic device for the search instruction, and avoiding the user from interrupting the processing flow for the search instruction by performing other operations. Exemplarily, referring to FIG. 5, the electronic device can display interface B1 as shown in FIG. 5, which can include prompt information A3 "understanding the screen", to prompt that the reference content to be searched is being determined from the displayed interface.

[0087] In a possible implementation, when the electronic device displays the above first prompt information in the interface, the electronic device can also highlight the first prompt information, so that the user can more easily perceive the first prompt information.

[0088] Optionally, the electronic device displays other regions in the interface except the above first prompt information with a first display brightness, and displays the above first prompt information with a second display brightness, the second display brightness being greater than the first display brightness, so that the first prompt information is more prominent than other regions.

[0089] Optionally, the above first prompt information is text prompt information, and the electronic device displays the first prompt information in a first color, which can be a colored font color different from other font colors in the screen.

[0090] Of course, the specific manner in which the electronic device highlights the above first prompt information can not be limited, for example, the electronic device can also adjust the transparency of other regions in the interface except the above first prompt information, so that the other regions are more transparent than the first prompt information, thereby making the first prompt information more prominent; for another example, the electronic device can also adjust the blur degree of other regions in the interface except the above first prompt information, so that the other regions become blurred, thereby making the first prompt information more prominent.

[0091] Step S230: After the search corresponding to the search instruction is performed on the target referred content, a second interface is displayed, and the second interface includes the search result obtained after the search corresponding to the search instruction is performed on the target referred content.

[0092] In the embodiments of the present application, after the electronic device determines the above target referred content, the second interface can be displayed after the search corresponding to the search instruction is performed on the target referred content, thereby realizing the search task required by the user and displaying the search result.

[0093] In some embodiments, after the electronic device determines the above target referred content, the script instruction of the corresponding automatic operation (i.e., automatic skill) can be generated based on the target referred content and the above search instruction according to a predefined rule or template, and then the script instruction is executed, thereby realizing the automatic execution of the search task required by the user.

[0094] In some embodiments, after the electronic device determines the above target referred content, when the search corresponding to the search instruction is performed on the target referred content, the electronic device can also output a prompt information to prompt that the search task corresponding to the search instruction is being performed, so as to avoid that the user interrupts the processing flow of the search instruction this time by performing other operations.

[0095] The content search method provided in the embodiments of the present application determines the target referred content from the display interface based on the search instruction of the user, determines the content referred by the entity reference in the search instruction from the interface currently displayed by the electronic device, and then displays an interface including the search result after performing the search corresponding to the search instruction on the target referred content, thereby completing the search task required by the user and displaying the search result to the user. Since the user only needs to input the search instruction in the entire operation process, the operation of searching the content in the display interface by the user can be reduced, the usability of the electronic device is improved, and the user experience of using the electronic device is provided.

[0096] Referring to FIG. 6, FIG. 6 shows a flowchart of a content searching method according to another embodiment of the present application. The content searching method is applied to the electronic device described above. The content searching method will be described in detail with reference to the flowchart shown in FIG. 6. The content searching method can include the following steps.

[0097] In step S310, the first interface is displayed.

[0098] In step S320, in response to the detected search instruction, the entity reference in the search instruction is determined, and the intent information corresponding to the search instruction is determined.

[0099] In step S330, based on the intent information, the content referred to by the entity reference is determined from the first interface.

[0100] In the embodiments of the present application, steps S310 to S330 can refer to the content of the foregoing embodiments, and will not be described here again.

[0101] In step S340, if the content referred to by the entity reference includes multiple items of content, candidate information corresponding to the multiple items of content is displayed in the interface.

[0102] In the embodiments of the present application, after the electronic device determines the content referred to by the entity reference from the first interface based on the intent information corresponding to the search instruction, it is considered that based on the intent information, the content referred to by the entity reference determined from the first interface can include multiple items of content. For example, the text of the search instruction input by the user is "search for the price of this piece of clothing", and the interface currently displayed by the electronic device includes pictures of multiple pieces of clothing. Therefore, when the electronic device identifies the content referred to by the entity reference, it can identify pictures of multiple pieces of clothing. However, the user actually wants to search for one item of content. Therefore, when it is determined that the content referred to by the entity reference can include multiple items of content, the electronic device can display candidate information corresponding to the multiple items of content in the interface, so that the user can select a corresponding item of content from the multiple items of content based on the displayed candidate information.

[0103] In some embodiments, when the electronic device displays the candidate information corresponding to the multiple items of content, it can display a frame selection area for each item of content in the interface, that is, each item of content referred to by the entity reference is in a frame selection area. Each frame selection area can be in a selectable state, and each frame selection area can be a candidate information. The electronic device can detect an operation in the interface to detect a selection operation on a corresponding frame selection area.

[0104] Exemplarily, please refer to Fig. 5 and Fig. 7 simultaneously, the search instruction is "what price is this hotel", after the electronic device determines the content referred by the entity designation in the first interface A1 shown in Fig. 5, the electronic device can display the interface B2 shown in Fig. 7, and the frame selection area A4 can be displayed for each item of content about "hotel" in the interface B2, so as to facilitate the user to select the content.

[0105] In a possible implementation, after the electronic device displays the frame selection area for each item of content in the above plurality of items of content, the electronic device can further highlight each frame selection area, so as to prompt the user to more easily see the frame selection areas and select the corresponding frame selection area.

[0106] Optionally, the electronic device displays other areas in the interface except the above frame selection areas with a third display brightness, and displays the above frame selection areas with a fourth display brightness, the fourth display brightness being greater than the third display brightness, so that the frame selection areas are more highlighted than the other areas.

[0107] Of course, the specific manner in which the electronic device highlights the above frame selection areas can not be limited, for example, the electronic device can also adjust the transparency of other areas in the interface except the above frame selection areas, so that the other areas are more transparent than the frame selection areas, thereby highlighting the frame selection areas; for another example, the electronic device can also adjust the blur degree of other areas in the interface except the above frame selection areas, so that the other areas become blurred, thereby highlighting the frame selection areas.

[0108] Optionally, the electronic device can also control the areas in the interface except the above frame selection areas to be in a non-selectable state, so as to avoid the user from mistakenly triggering the operation on other content in the interface, thereby avoiding the interruption of the selection of the frame selection areas.

[0109] In a possible implementation, when the electronic device displays the frame selection area for each item of content in the above plurality of items of content, the electronic device can also display prompt information in the interface, so as to prompt the user to select the frame selection area, thereby determining the target designation content to be searched this time based on the frame selection area selected by the user.

[0110] In some embodiments, when the electronic device displays the candidate information corresponding to the plurality of items of content, the electronic device can display a content list in the interface, the content list including options corresponding to each item of content in the above plurality of items of content, that is, each item of content corresponds to an option as a candidate information. The electronic device can detect an operation in the interface to detect a selection operation on the corresponding option.

[0111] Exemplarily, please refer to Fig. 5 and Fig. 8 simultaneously, the search instruction is "what price is this hotel", after the electronic device determines the content pointed by the entity designation in interface A1 shown in Fig. 5, the electronic device can display interface B3 shown in Fig. 8, and in interface B3, an option A5 can be displayed for each item of content about "hotel", thereby facilitating the user to select the content.

[0112] In a possible implementation, considering that the content pointed by the above target entity designation in the display interface can include text content or image content, therefore, in a case where any of the above content is text content, the electronic device can directly display the option corresponding to the text content when displaying the option corresponding to the content in the above content list; in a case where any of the above content is image content, the electronic device can acquire the image corresponding to the image content, and generate a thumbnail image for the image, and in the above content list, the electronic device can display the option corresponding to the thumbnail image.

[0113] In a possible implementation, when displaying the content selection list for the above content, the electronic device can also display prompt information in the interface to prompt the user to select the option, thereby determining the target designation content to be searched this time based on the option selected by the user.

[0114] Step S350: in response to the selection operation on the target candidate information, determining the content corresponding to the target candidate information as the target designation content.

[0115] In the embodiment of the present application, after the electronic device displays the candidate information for the content pointed by the above entity designation, the electronic device can detect the operation in the interface, and after detecting the selection operation on the target candidate information, the electronic device can determine the content corresponding to the target candidate information selected by the selection operation, and determine the content corresponding to the target candidate information as the target designation content to be searched this time.

[0116] In a possible implementation, the above selection operation can be a click operation on the target candidate information, that is, when the electronic device detects the click operation on the target candidate information, the electronic device can determine the content corresponding to the target candidate information as the target designation content to be searched this time.

[0117] Of course, the specific operation type of the above selection operation on the target candidate information is not limited, for example, the above selection operation can also be a long press operation on the target candidate information, and the long press operation can be a press operation with a press duration longer than a target press duration.

[0118] Step S360: after the search corresponding to the search instruction is performed on the target reference content, displaying the second interface, wherein the second interface includes the search result obtained after the search corresponding to the search instruction is performed on the target reference content.

[0119] In some embodiments, after the target reference content is determined, the electronic device determines the type of the target reference content, and performs the search task based on the type of the target reference content, because the target reference content can be text content or image content, and the operations for searching different types of reference content are different.

[0120] In a possible implementation, if the target reference content is text content in the first interface, the target reference content can be filled into the search instruction, and then the search corresponding to the search instruction is performed based on the search instruction filled with the target reference content. The target reference content can be filled into the search instruction by replacing the entity reference in the search instruction with the target reference content. For example, the text of the search instruction is "how about the price of this hotel", and the target reference content determined from the first interface is "A", then the entity reference "this" in "how about the price of this hotel" can be replaced with "A", so that the search instruction filled with the target reference content is "A hotel price how about".

[0121] In a possible implementation, if the target reference content is image content in the first interface, the region image of the region where the image content is located is obtained from the interface image of the first interface, and the search corresponding to the search instruction is performed on the region image. The interface image of the first interface can be a screenshot of the first interface. When the target reference content is image content in the first interface, the image of the region corresponding to the position coordinates of the image content in the first interface can be cropped from the interface image, so as to obtain the region image. The search corresponding to the search instruction can be performed on the region image by using an application with image search function.

[0122] The content search method provided by the embodiments of the present application can realize the search of the corresponding content in the display interface according to the search instruction input by the user, thereby reducing the operation of the user searching the content in the display interface, improving the usability of the electronic device, and providing the user experience of the electronic device. In addition, when the content to be searched is determined from the display interface based on the search instruction, the interactive function is provided for the user to select the content to be searched according to the user demand from the multiple contents, so as to better meet the search demand of the user.

[0123] Please refer to Fig. 9, Fig. 9 shows a flow diagram of a content search method provided by another embodiment of the present application. The content search method is applied to the electronic device, and the flow shown in Fig. 9 will be described in detail below. The content search method can specifically include the following steps:

[0124] Step S410: display a first interface, the first interface corresponding to a first application.

[0125] In the embodiment of the present application, the first interface displayed by the electronic device can be the application interface of the first application, that is, the scenario of content search in the embodiment of the present application can be the scenario that the user needs to search for the corresponding content in the application interface of the first application when the electronic device displays the application interface of the first application.

[0126] Step S420: in response to the search instruction, calling a second application, and after the target reference content is searched by the second application according to the search instruction, displaying a second interface corresponding to the second application, the second interface being an interface displayed by the second application for the search result.

[0127] The second interface includes the search result obtained after the target reference content is searched according to the search instruction, and the target reference content is the content referred to by the entity reference in the search instruction determined from the first interface.

[0128] In the embodiment of the present application, after the target reference content is obtained in response to the search instruction while the first interface is displayed, the second application can be called, and after the target reference content is searched by the second application according to the search instruction, the electronic device can switch the display interface to the second interface corresponding to the second application, and the second interface is the interface for displaying the search result of this search by the second application. In this way, the search result can be directly displayed in the application interface of the second application for this search task, so that the user can perform relevant operations on the search result.

[0129] In some embodiments, after the second application is called, the second application can also display second prompt information in the interface when the target reference content is searched according to the search instruction, the second prompt information being used to prompt that the search according to the search instruction is being performed by the second application, so as to facilitate the user to know the processing stage of the electronic device for the search instruction, and avoid the user to interrupt the processing flow for the search instruction by performing other operations.

[0130] In a possible implementation, when the electronic device displays the second prompt information in the interface, the electronic device can also highlight the second prompt information, so that the user can more easily perceive the first prompt information. The specific manner in which the electronic device highlights the second prompt information can be not limited.

[0131] In a possible implementation, when the electronic device invokes the second application and performs the search corresponding to the search instruction on the target-referenced content through the second application, the search can be processed in the background; after the search is completed through the second application, the second application is controlled to be in the foreground display, and then the second interface is displayed. Alternatively, the electronic device can start the second application, control the second application to run in the background, and perform the search corresponding to the search instruction on the target-referenced content by using the second application running in the background. In this way, the search process can be invisible to the user, and the user can directly see the second interface after inputting the search instruction, without seeing the process of starting the second application and operating the second application to perform the search.

[0132] The foreground running refers to a state in which the electronic device displays the application interface of the application program on the screen, and when the application program is in the foreground running, the user can see the application interface of the application program on the screen. The background running, opposite to the foreground running, refers to a state in which the electronic device does not display the application interface of the application program on the screen, and the user cannot interact with the application interface of the application program, but the application program occupies the system resource of the electronic device when running in the background.

[0133] In some embodiments, before performing the search corresponding to the search instruction on the target-referenced content through the second application, the electronic device can generate a script instruction for implementing the search task according to the skill template of the second application and based on the search instruction, and then execute the script instruction, so as to automatically perform the search task.

[0134] The content search method provided by the embodiments of the present application can achieve the search of the corresponding content in the display interface according to the user demand by inputting the search instruction, thereby reducing the operation of the user on the content in the display interface, improving the usability of the electronic device, and providing the user with a use experience of the electronic device. In addition, when the search result is displayed, the display interface is switched to the interface of the application performing the search task, so that the search result can be directly displayed in the application interface of the second application performing the search task, and the user can easily perform related operations on the search result.

[0135] Please refer to FIG. 10, which shows a flowchart of a content search method according to another embodiment of the present application. The content search method is applied to the electronic device described above. The content search method will be described in detail below with reference to the flowchart shown in FIG. 10. The content search method can specifically include the following steps:

[0136] Step S510: display a first interface, wherein the first interface corresponds to a first application.

[0137] In the embodiments of the present application, step S510 can refer to the content described in the foregoing embodiments, which will not be described here again.

[0138] Step S520: in response to the detected search instruction, determine a second application to be invoked based on the search instruction and the type of the target referred content.

[0139] In the embodiments of the present application, after the electronic device responds to the above search instruction and obtains the target referred content, the electronic device can determine the application used for the search task, i.e., determine the second application to be invoked. Since the application to be invoked needs to be able to complete the search task, and different applications can perform different search tasks, for example, a shopping application can be used to search for goods, a travel application can be used to search for tickets, and different applications can search for different types of content, for example, some applications can be used to search for text content and images, some applications can only be used to search for text content, and some applications can only search for images. Therefore, the electronic device can determine the second application to be invoked based on the search instruction and the type of the target referred content.

[0140] In some embodiments, the electronic device can perform intent recognition on the search instruction to obtain intent information corresponding to the search instruction, for example, the type of the task that needs to be performed on the target referred content (such as searching for price, searching for weather, searching for location, etc.), and the type of the target referred content (such as text type, image type, etc.). The electronic device can determine an application that matches the intent information and the type of the target referred content from a knowledge base based on the intent information and the type of the target referred content. The knowledge base can include application knowledge corresponding to the applications installed on the electronic device, and the application knowledge can include the search tasks that can be performed by each application and the types of content that can be searched by each application.

[0141] In a possible implementation, the electronic device can obtain the application knowledge of the above applications from a server in advance and save the obtained application knowledge in the knowledge base. Alternatively, the electronic device can obtain the application knowledge of each application from the server and save the application knowledge in the knowledge base when the electronic device installs the application.

[0142] In a possible implementation, considering that there are more applications installed in the electronic device, and more applications can implement the same search task and search content of the same content type, the electronic device can determine multiple applications based on the intent information and the type of the target reference content. Therefore, in a case where multiple applications are determined, the second application can be determined from the multiple applications based on the usage data of each application.

[0143] Optionally, the usage data can include at least one of a usage frequency of the application and a usage frequency of the application. The electronic device can determine a first usage score of each application in the multiple applications determined above based on the usage data of each application, and then determine the application with the highest first usage score from the multiple applications as the second application above, where the first usage score of the application is positively correlated with the usage frequency and the usage frequency of the application. Understandably, the usage frequency and the usage frequency of the application to some extent reflect the user's usage habits, so through the above method, the second application used to perform the search task this time can be accurately determined.

[0144] Optionally, the usage data can include a target search number, which is the number of searches in which the application is used to perform a search task matching the intent information and the type of the target reference content. The electronic device can determine a second usage score of each application based on the search number, and then determine the application with the highest second usage score from the multiple applications above as the second application above, where the second usage score is positively correlated with the search number. Understandably, the search number reflects the user's preference for using the application to perform a search task matching the intent information and the content type, so through the above method, the application that meets the user's preference can be determined.

[0145] In some embodiments, there can be a case where the search instruction contains a keyword corresponding to the application used for the search task this time, for example, the search instruction input by the user is "search the price of this chair in the B application". In this case, the electronic device can directly determine the second application used to perform the search task this time based on the keyword corresponding to the application in the search instruction.

[0146] Step S530: After invoking the second application and performing the search corresponding to the search instruction on the target reference content through the second application, a second interface corresponding to the second application is displayed, where the second interface is an interface displayed by the second application for the search result.

[0147] In the embodiments of the present application, step S530 can refer to the content of the foregoing embodiments, which will not be described here.

[0148] The content search method provided in the embodiments of the present application can realize the search of the corresponding content in the display interface according to the user demand by inputting the search instruction, thereby reducing the operation of the user when searching the content in the display interface, improving the usability of the electronic device, and providing the user with the use experience of the electronic device. When the search result is displayed, the display interface is switched to the interface of the application for which the search task is performed, so that the search result can be directly displayed in the application interface of the second application for which the search task is performed, and the user can conveniently perform the related operation on the search result. In addition, when the application for performing the search task is determined, the determination is based on the search instruction and the type of the content to be searched, so that the application for performing the search task can be accurately determined, thereby effectively ensuring that the search task according to the user demand can be successfully performed.

[0149] The content search method provided in the embodiments of the present application can realize the search of the corresponding content in the display interface according to the user demand by inputting the search instruction, thereby reducing the operation of the user when searching the content in the display interface, improving the usability of the electronic device, and providing the user with the use experience of the electronic device. When the search result is displayed, the display interface is switched to the interface of the application for which the search task is performed, so that the search result can be directly displayed in the application interface of the second application for which the search task is performed, and the user can conveniently perform the related operation on the search result. In addition, when the application for performing the search task is determined, the determination is based on the search instruction and the type of the content to be searched, so that the application for performing the search task can be accurately determined, thereby effectively ensuring that the search task according to the user demand can be successfully performed.

[0150] Exemplarily, referring to FIGS. 2, 3, 5, 11 and 12, when the electronic device displays the first interface A1 as shown in FIG. 2, after detecting the calling operation of the intelligent assistant, the electronic device can display the interface as shown in FIG. 3, so that the interactive region A2 of the intelligent assistant is displayed in the interface, and the input content can be displayed in the interactive region A2 after detecting the content input by the user for the intelligent assistant. After detecting the search instruction “how about the price of this hotel” input by the user, the interface as shown in FIG. 5 can be displayed, and the prompt information A3 “understanding the screen” can be displayed in the interface to prompt that the content to be searched is being determined from the displayed interface. After the target content “A hotel” in the interface is determined, the application A is called to search “A hotel”, and during the search process, the interface B4 as shown in FIG. 11 can be displayed, and the prompt information A6 “operating the application A” can be displayed in the interface B4 to prompt that “A hotel” is being searched through the application A. After the search is completed, the application interface A7 of the application A as shown in FIG. 12 can be displayed, and the search result for “A hotel” can be displayed in the application interface A7.

[0151] Exemplarily, please refer to FIG. 13, FIG. 14, FIG. 15, FIG. 16 and FIG. 17, when the electronic device displays the first interface A1 as shown in FIG. 13, after detecting the calling operation on the intelligent assistant, the electronic device can display the interface B5 as shown in FIG. 14, so that the interactive area A2 of the intelligent assistant is displayed in the interface B5, and the content input by the user for the intelligent assistant can be displayed in the interactive area A2 after detecting the content input by the user; after detecting the search instruction "does this chair have a sale in the application B" input by the user, the interface B6 as shown in FIG. 15 can be displayed, and the prompt information A3 "understanding the screen" can be displayed in the interface B6, to prompt that the reference content to be searched is determined from the displayed interface; after determining the target reference content "the picture of the chair" in the interface, the picture A8 can be intercepted from the interface, the application B is called to search "the picture A8", and during the searching process, the interface B7 as shown in FIG. 16 can be displayed, and the prompt information "operating the application B" can be displayed in the interface B7, to prompt that the picture A8 is searched through the application B; after completing the searching, the application interface A9 of the application B as shown in FIG. 17 can be displayed, and the search result for the picture A8 can be displayed in the application interface A9.

[0152] Please refer to FIG. 18, which shows a flowchart of a content searching method according to another embodiment of the present application. The content searching method is applied to the above-mentioned electronic device, and the following will be described in detail with respect to the flowchart shown in FIG. 18. Specifically, the content searching method can include the following steps:

[0153] Step S610: display a first interface, wherein the first interface corresponds to a first application.

[0154] In the embodiments of the present application, step S510 can refer to the content of the foregoing embodiments, which will not be described herein again.

[0155] Step S620: in response to the search instruction, calling a second application, and after the second application performs the search corresponding to the search instruction on the target reference content, displaying a second interface, wherein the second interface includes a first split-screen area and a second split-screen area, the first split-screen area includes the first interface, and the second split-screen area includes a third interface displayed by the second application for the search result, wherein the search result is obtained by searching through the second application.

[0156] In the embodiment of the present application, in the case that the electronic device displays the above first interface, in response to the above search instruction, after obtaining the above target reference content, the electronic device can call the second application, and after the second application performs the search corresponding to the search instruction, the electronic device can split the first application and the second application and display the second interface, which includes the first split-screen area and the second split-screen area. The first split-screen area includes the first interface, and the second split-screen area includes the third interface displayed by the second application for the search result. In this way, after the electronic device completes the search, the interface of the first application and the second application can be displayed simultaneously, thereby facilitating the user to view the search result and the first interface of the first application displayed previously.

[0157] It can be understood that after the user searches the corresponding interface content in the application interface, the user may return to the previous application for use after viewing the search result. The content search method provided in the embodiment of the present application can display the interface of the first application and the second application in the screen of the electronic device simultaneously, thereby reducing the number of times of application switching by the user and facilitating the user to operate the search result in the application interface displayed in the split-screen area.

[0158] In some embodiments, the above split-screen area displayed by the electronic device can be a first type of split-screen area or a second type of split-screen area. For the first type of split-screen area, the first type of split-screen area can be a traditional split-screen. When the electronic device displays the first type of split-screen area in the foreground, the application interface included in the split-screen area is completely displayed in the display interface. For the second type of split-screen area, the second type of split-screen area can be a canvas type split-screen. The canvas type split-screen refers to allowing at least part of the split-screen interface to be not displayed on the screen of the electronic device when splitting the screen. This can be understood as allowing part of the split-screen interface to be not completely displayed in the screen in the plurality of split-screen interfaces displayed by splitting the screen.

[0159] The content search method provided in the embodiment of the present application can realize that the user can search the corresponding content in the display interface by inputting the search instruction, thereby reducing the operation of the user when searching the content in the display interface, improving the usability of the electronic device, and providing the user experience of using the electronic device. In addition, when displaying the search result, the interface of the application displayed previously and the interface of the application performing the search task this time are split and displayed, thereby facilitating the user to view the search result and the interface displayed previously simultaneously, and also facilitating the user to perform related operations on the search result in the split-screen area.

[0160] The content search method provided in the foregoing embodiments will be further described below with reference to the accompanying drawings.

[0161] The interface interaction method related to the foregoing embodiment is introduced through FIG. 19.

[0162] As shown in FIG. 19, in a case where the smart assistant in the electronic device is in an awakened state, a search instruction input through voice or text can be detected; after detecting the search instruction, the electronic device can be triggered to take a screenshot of the current interface, and text recognition can be performed on the screenshot to obtain a recognition result; then the screenshot, the text recognition result, and the search instruction are input into a large language model, so as to complete intent recognition of the voice instruction and determine an interest point (i.e., the target referred content in the foregoing embodiment) in the display interface; if the type of the interest point is a text type, the text content can be filled into the search instruction; if the type of the interest point is an image type, a region image can be intercepted; then the large language model is used to match an automated skill for the above search instruction, to generate a corresponding script instruction, and then the script instruction is executed to complete the search task of the current user demand, and the search result can be displayed to enable the user to know the operation result.

[0163] Please refer to FIG. 20, which shows a structural block diagram of a content search apparatus 700 provided by an embodiment of the present application. The content search apparatus 700 applies the electronic device described above, and the content search apparatus 700 includes a first interface display module 710 and a second interface display module 720. The first interface display module 710 is configured to display a first interface; the second interface display module 720 is configured to display a second interface in response to a detected search instruction, the second interface including a search result obtained after performing a search corresponding to the search instruction on a target referred content, the target referred content being a content referred to by an entity designation in the search instruction and determined from the first interface.

[0164] In some embodiments, the second interface display module 720 can be specifically configured to: in response to the search instruction, determine the target referred content from the first interface based on the search instruction; and display the second interface after performing a search corresponding to the search instruction on the target referred content.

[0165] In a possible implementation, the second interface display module 720 can be further configured to determine an entity designation in the search instruction and intent information corresponding to the search instruction; and determine, as the target referred content, a content referred to by the entity designation from the first interface based on the intent information.

[0166] Optionally, the second interface display module 720 can be further configured to obtain a recognition result obtained by performing content recognition on an interface image of the first interface; and determine, as the target referred content, a content matched with the intent information and the entity designation from the recognition result.

[0167] Optionally, the second interface display module 720 can be further configured to determine the content referred to by the entity designation from the first interface based on the intention information; if the content referred to by the entity designation comprises a plurality of contents, display candidate information corresponding to the plurality of contents in the interface; and in response to a selection operation on target candidate information, determine the content corresponding to the target candidate information as the target designated content.

[0168] In a possible implementation, the second interface display module 720 can be further configured to, if the target designated content is text content in the first interface, fill the target designated content to the search instruction, and then perform the search corresponding to the search instruction.

[0169] In a possible implementation, the second interface display module 720 can be further configured to, if the target designated content is image content in the first interface, acquire a region image of a region where the image content is located from an interface image of the first interface, and perform the search corresponding to the search instruction on the region image.

[0170] In a possible implementation, the second interface display module 720 can be further configured to, after the search corresponding to the search instruction on the target designated content is performed, display first prompt information in the interface before the second interface is displayed, the first prompt information being used to prompt that the target designated content is being determined from the interface.

[0171] In some embodiments, the second interface display module 720 can be specifically configured to: in response to the search instruction, call a second application, and after the search corresponding to the search instruction on the target designated content is performed by the second application, display a second interface corresponding to the second application, the second interface being an interface displayed by the second application for the search result.

[0172] In a possible implementation, the second interface display module 720 can be further configured to, before the search corresponding to the search instruction on the target designated content is performed by the second application, determine the second application to be called based on the search instruction and a type of the target designated content.

[0173] In a possible implementation, the second interface display module 720 can be further configured to, after the second application is called, display second prompt information in the interface, the second prompt information being used to prompt that the search corresponding to the search instruction is being performed by the second application.

[0174] In some embodiments, the second interface display module 720 can be specifically configured to: in response to the search instruction, invoke a second application, and after the second application performs a search corresponding to the search instruction on the target referred content, display a second interface, the second interface including a first split-screen area and a second split-screen area, the first split-screen area including the first interface, and the second split-screen area including a third interface displayed by the second application for the search result, the search result being obtained after the search by the second application.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and modules can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0176] In several embodiments provided in the present application, the coupling between the modules can be electrical, mechanical or other forms of coupling.

[0177] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0178] To sum up, the scheme provided in the present application displays a first interface, and in response to a detected search instruction, displays a second interface, the second interface including a search result obtained after a search corresponding to the search instruction is performed on target referred content, the target referred content being content referred to by an entity designation in the search instruction determined from the first interface. Thus, the content referred to by the entity designation in the search instruction determined from the interface can be searched directly based on the user's search instruction, and the corresponding search is performed on the referred content, so that the user only needs to input the search instruction to search the corresponding content in the interface, thereby simplifying the operation of the user when searching the content in the interface, and improving the usability of the electronic device.

[0179] Please refer to FIG. 21, which shows a structural block diagram of an electronic device according to an embodiment of the present application. The electronic device 100 can be a smart phone, a tablet computer, an e-book, a notebook computer, a smart watch, or the like, which can run an application program. The electronic device 100 in the present application can include one or more of the following components: a processor 110, a memory 120, a display screen 130, an audio acquisition device 140, and one or more application programs, wherein the one or more application programs can be stored in the memory 120 and configured to be executed by the one or more processors 110, and the one or more application programs are configured to perform the method described in the foregoing method embodiments.

[0180] The processor 110 connects various parts within the entire electronic device 100 using various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and calling data stored in the memory 120.

[0181] The memory 120 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc.

[0182] The display screen 130 is a display component for image display, and is usually arranged on the front panel of the electronic device 100; the audio acquisition device 140 is used to acquire audio in the environment, and can be a microphone or the like.

[0183] Please refer to FIG. 22, which shows a structural block diagram of a computer readable storage medium provided by the embodiments of the present application. The computer readable medium 800 stores program codes therein, which can be invoked by a processor to execute the methods described in the above method embodiments.

[0184] The computer readable storage medium 800 can be an electronic storage such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer readable storage medium 800 includes a non-transitory computer readable medium. The computer readable storage medium 800 has a storage space for program codes 810 for executing any of the above methods. These program codes can be read from or written into one or more computer program products. The program codes 810 can be compressed in an appropriate form, for example.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features thereof; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A content search method, wherein, The method is applied to an electronic device, and the method comprises: displaying a first interface; in response to a detected search instruction, displaying a second interface, wherein the second interface comprises search results obtained by performing a search corresponding to the search instruction on target referred content, and the target referred content is content referred to by an entity reference in the search instruction and determined from the first interface.

2. The method of claim 1, wherein, The method further comprises: in response to the search instruction, determining the target referred content from the first interface based on the search instruction; after performing the search corresponding to the search instruction on the target referred content, displaying the second interface.

3. The method of claim 2, wherein, The method further comprises: determining the entity reference in the search instruction and intent information corresponding to the search instruction; based on the intent information, determining content referred to by the entity reference from the first interface as the target referred content.

4. The method of claim 3, wherein, The method further comprises: obtaining recognition results obtained by performing content recognition on an interface image of the first interface; from the recognition results, determining content matching the intent information and the entity reference as the target referred content.

5. The method of claim 3 or 4, wherein, The method further comprises: based on the intent information, determining content referred to by the entity reference from the first interface; if the content referred to by the entity reference comprises multiple items of content, displaying candidate information corresponding to the multiple items of content in an interface; in response to a selection operation on target candidate information, determining content corresponding to the target candidate information as the target referred content.

6. The method of claim 5, wherein, The method further comprises: if the content referred to by the entity reference comprises multiple items of content, displaying a box selection region for each item of content, and each box selection region is a candidate information.

7. The method according to any one of claims 2-6, wherein, The method further comprises: if the target referred content is text content in the first interface, filling the target referred content into the search instruction, and performing the search corresponding to the search instruction.

8. The method according to any one of claims 2-6, wherein, The method further comprises: if the target referred content is image content in the first interface, obtaining a region image of a region where the image content is located from an interface image of the first interface; performing the search corresponding to the search instruction on the region image.

9. The method according to any one of claims 2-8, wherein, The method further comprises: before displaying the second interface after performing the search corresponding to the search instruction on the target referred content, the method further comprises: Displaying a first prompt information in the interface, the first prompt information is used for prompting that the target reference content is being determined from the interface.

10. The method of any one of claims 1-9, wherein, The application corresponding to the first interface is a first application, and the second interface is displayed in response to the detected search instruction, including: In response to the search instruction, a second application is invoked, and after the second application performs a search corresponding to the search instruction on the target reference content, a second interface corresponding to the second application is displayed, which is an interface displayed by the second application for the search result.

11. The method of claim 10, wherein, Before invoking the second application to perform the search corresponding to the search instruction on the target reference content, the method further includes: Based on the search instruction and the type of the target reference content, determine the second application to be invoked.

12. The method of claim 11, wherein, The second application to be invoked is determined based on the search instruction and the type of the target reference content, including: Based on the intent information corresponding to the search instruction and the type of the target reference content, determine the second application that matches the intent information and the type of the target reference content from a knowledge base, the knowledge base including application knowledge corresponding to applications installed on the electronic device.

13. The method of claim 12, wherein, Based on the intent information corresponding to the search instruction and the type of the target reference content, determine the second application that matches the intent information and the type of the target reference content from a knowledge base, the knowledge base including application knowledge corresponding to applications installed on the electronic device. Based on the intent information corresponding to the search instruction and the type of the target reference content, determine the second application that matches the intent information and the type of the target reference content from a knowledge base, the knowledge base including application knowledge corresponding to applications installed on the electronic device. In the case where the determined application is a plurality of applications, the second application is determined from the plurality of applications based on usage data of each application in the plurality of applications, the usage data including at least one of the number of uses of the application, the frequency of use of the application, and the target search number, the target search number being the number of searches of the application used to perform a search task matching the intent information and the type of the target reference content.

14. The method according to any one of claims 10-13, wherein, After the second application is invoked, the method further includes: Displaying a second prompt information in the interface, the second prompt information is used for prompting that the search corresponding to the search instruction is being performed through the second application.

15. The method according to any one of claims 10-14, wherein, After the second application is invoked, the method further includes: In response to the search instruction, the second application is run in the background; Using the second application running in the background to perform the search corresponding to the search instruction on the target reference content; After completing the search corresponding to the search instruction through the second application, the second application is controlled to run in the foreground, and the second interface is displayed.

16. The method of any one of claims 1-15, wherein, The application corresponding to the first interface is a first application, and the second interface is displayed in response to the detected search instruction, including: In response to the search instruction, a second application is invoked, and after the target-referred content is searched by the second application according to the search instruction, a second interface is displayed, the second interface including a first split-screen area and a second split-screen area, the first split-screen area including the first interface, and the second split-screen area including a third interface displayed by the second application for the search result, the search result being obtained after the search by the second application.

17. The method of any one of claims 1-15, wherein, The second interface is an interface after a result display area is superimposed and displayed in the first interface, and the second interface is displayed in response to the detected search instruction, including: In response to the detected search instruction, the result display area is superimposed and displayed in the first interface, and the result display area includes the search result.

18. A content search apparatus, wherein, The apparatus includes a first interface display module and a second interface display module, wherein The first interface display module is configured to display a first interface. The second interface display module is configured to display a second interface in response to a detected search instruction, the second interface including a search result obtained after a target-referred content is searched according to the search instruction, the target-referred content being content referred to by an entity reference in the search instruction determined from the first interface.

19. An electronic device, comprising: comprise: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-17.

20. A computer readable storage medium, wherein, The computer-readable storage medium stores program code, and the program code can be invoked and executed by a processor to perform the method of any one of claims 1-17.

Images