Object searching method and device, electronic equipment, storage medium and product

Abstract

This disclosure relates to an object search method, apparatus, electronic device, storage medium, and product, and pertains to the fields of intelligent interaction and artificial intelligence technologies. The object search method includes: acquiring a search instruction related to a user intent, and determining a first interface image corresponding to a first interface displayed by the electronic device, the search instruction being used to search for a target object on the electronic device; determining an operation sequence, the operation sequence being determined based on the search instruction and the first interface image, and including multiple operation steps for searching the target object; invoking a target function corresponding to the multiple operation steps based on a functional interface, and executing the multiple operation steps included in the operation sequence through the target function to acquire and display the target object. This disclosure reduces the latency of the electronic device automatically searching for a target object based on a search instruction, achieving one-step access to user services.

Description

Technical Field

[0001] This disclosure relates to the fields of intelligent interaction and artificial intelligence technology, and in particular to object search methods, devices, electronic devices, storage media and products. Background Technology

[0002] With the development of artificial intelligence and speech recognition technologies, electronic devices are now able to recognize user voice commands through intelligent agents. These agents can then control the devices to execute functions intended by the user, allowing users to perform search functions via voice commands. For example, users can use voice commands to control their devices to search for videos, music, or applications.

[0003] Related technologies encounter problems in recognizing user intent through intelligent agents, such as repeated information extraction and confirmation, resulting in high latency and low recognition efficiency, leading to a poor user experience. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an object search method, apparatus, electronic device, storage medium, and product.

[0005] According to a first aspect of the present disclosure, an object search method is provided, comprising: acquiring a search instruction related to a user intent, and determining a first interface image corresponding to a first interface displayed by an electronic device, the search instruction being used to search for a target object on the electronic device; determining an operation sequence, the operation sequence being determined based on the search instruction and the first interface image, and including multiple operation steps for searching the target object; invoking a target function corresponding to the multiple operation steps based on a functional interface, and executing the multiple operation steps included in the operation sequence through the target function to acquire and display the target object.

[0006] According to some embodiments of this disclosure, determining the operation sequence includes: inputting the search instruction and the first interface image into an intelligent agent, and obtaining the operation sequence fed back by the intelligent agent, wherein the operation sequence is determined by the intelligent agent based on the search instruction and the first interface image.

[0007] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for the target object; the operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0008] According to some embodiments of this disclosure, the step of calling a target function corresponding to multiple operation steps based on a function interface, and executing multiple operation steps included in the operation sequence through the target function, includes: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the first interface through the target function.

[0009] According to some embodiments of this disclosure, the first interface image does not include a target search box image, the target search box being used to search for the target object; the method further includes: obtaining feedback information from the agent, the feedback information being determined by the agent according to the search instruction, and the feedback information being used to instruct the electronic device to display a second interface, the second interface displaying the target search box; wherein, the operation sequence is determined according to the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0010] According to some embodiments of this disclosure, the second interface image can be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0011] According to some embodiments of this disclosure, the second interface image cannot be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The method further includes: displaying the second interface based on the feedback information; obtaining the second interface image, the second interface image including the target search box image; and inputting the second interface image into the intelligent agent.

[0012] According to some embodiments of this disclosure, the step of calling a target function corresponding to multiple operation steps based on a function interface, and executing multiple operation steps included in the operation sequence through the target function, includes: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the second interface through the target function.

[0013] According to a second aspect of the present disclosure, an object search method is provided, the method comprising: acquiring a search instruction sent by an electronic device and a first interface image, the search instruction being used to search for a target object on the electronic device, the first interface image corresponding to a first interface displayed by the electronic device; determining an operation sequence based on the search instruction and the first interface image, the operation sequence including multiple operation steps for searching the target object; sending the operation sequence to the electronic device, the operation sequence being used to instruct the electronic device to call a target function based on a functional interface for the multiple operation steps, and to execute the multiple operation steps included in the operation sequence through the target function, so as to acquire and display the target object on the electronic device.

[0014] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for the target object; the step of determining the operation sequence according to the search instruction and the first interface image includes: determining the operation sequence according to the search instruction and the target search box image in the first interface image.

[0015] According to some embodiments of this disclosure, the first interface image does not include a target search box image, the target search box being used to search for the target object; the method further includes: sending feedback information to the electronic device, the feedback information being used to instruct the electronic device to display a second interface, the second interface displaying the target search box; wherein, the operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0016] According to some embodiments of this disclosure, the second interface image can be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0017] According to some embodiments of this disclosure, the second interface image cannot be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The method further includes: The device receives the second interface image sent by the electronic device, the second interface image including the target search box image.

[0018] According to a third aspect of the present disclosure, an object search apparatus is provided, comprising: a determining unit, configured to acquire a search instruction related to a user intent and determine a first interface image corresponding to a first interface displayed by an electronic device, the search instruction being used to search for a target object on the electronic device; a processing unit, configured to determine an operation sequence, the operation sequence being determined based on the search instruction and the first interface image, and including multiple operation steps for searching the target object; and an execution unit, configured to call a target function corresponding to the multiple operation steps based on a functional interface, and execute the multiple operation steps included in the operation sequence through the target function to acquire and display the target object.

[0019] According to some embodiments of this disclosure, the processing unit determines the operation sequence in the following manner: inputting the search instruction and the first interface image into the agent, and obtaining the operation sequence fed back by the agent, wherein the operation sequence is determined by the agent based on the search instruction and the first interface image.

[0020] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for the target object; the operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0021] According to some embodiments of this disclosure, the execution unit calls a target function corresponding to multiple operation steps based on a function interface in the following manner, and executes multiple operation steps included in the operation sequence through the target function: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the first interface through the target function.

[0022] According to some embodiments of this disclosure, the first interface image does not include a target search box image, the target search box being used to search for the target object; the processing unit is further configured to: obtain feedback information from the intelligent agent, the feedback information being determined by the intelligent agent according to the search instruction, and the feedback information being used to instruct the electronic device to display a second interface, the second interface displaying the target search box; wherein, the operation sequence is determined according to the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0023] According to some embodiments of this disclosure, the second interface image can be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0024] According to some embodiments of this disclosure, the second interface image cannot be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The processing unit is further configured to: display the second interface based on the feedback information; acquire the second interface image, which includes a target search box image; and input the second interface image into the intelligent agent.

[0025] According to some embodiments of this disclosure, the execution unit calls a target function corresponding to multiple operation steps based on a function interface in the following manner, and executes multiple operation steps included in the operation sequence through the target function: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the second interface through the target function.

[0026] According to a fourth aspect of the present disclosure, an object search apparatus is provided, the apparatus comprising: an acquisition unit, configured to acquire a search instruction sent by an electronic device and a first interface image, the search instruction being used to search for a target object on the electronic device, the first interface image corresponding to a first interface displayed by the electronic device; a processing unit, configured to determine an operation sequence based on the search instruction and the first interface image, the operation sequence including multiple operation steps for searching the target object; and a sending unit, configured to send the operation sequence to the electronic device, the operation sequence being used to instruct the electronic device to call a target function based on a functional interface to perform multiple operation steps through the target function, so as to acquire and display the target object on the electronic device.

[0027] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for the target object; the processing unit determines the operation sequence according to the search instruction and the first interface image in the following manner: the operation sequence is determined according to the search instruction and the target search box image in the first interface image.

[0028] According to some embodiments of this disclosure, the first interface image does not include a target search box image, the target search box being used to search for the target object; the processing unit is further configured to: send feedback information to the electronic device, the feedback information being used to instruct the electronic device to display a second interface, the second interface displaying the target search box; wherein, the operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0029] According to some embodiments of this disclosure, the second interface image can be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0030] According to some embodiments of this disclosure, the second interface image cannot be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The processing unit is further configured to: receive the second interface image sent by the electronic device, wherein the second interface image includes a target search box image.

[0031] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the object search method described in the first aspect or any embodiment of the first aspect, or execute the object search method described in the second aspect or any embodiment of the second aspect.

[0032] According to a sixth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the object search method described in the first aspect or any embodiment of the first aspect, or to perform the object search method described in the second aspect or any embodiment of the second aspect.

[0033] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the object search method as described in the first aspect or any embodiment of the first aspect, or implements the object search method as described in the second aspect or any embodiment of the second aspect.

[0034] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: After obtaining a search instruction related to the user's intent, a first interface image corresponding to the first interface displayed by the electronic device is determined. Based on the search instruction and the first interface image, an operation sequence including multiple operation steps is determined. A target function corresponding to the multiple operation steps is called based on a functional interface. Through the target function, the multiple operation steps included in the operation sequence are executed to obtain and display the target object. Through this disclosure, after obtaining a search instruction issued by the user, multiple operation steps are determined based on the search instruction and the interface screenshot corresponding to the electronic device's display interface. These operation steps are then executed by directly calling functions through a functional interface to search for the target object intended by the user. This eliminates the need to obtain the interface image again after each operation step to determine the next operation step. Furthermore, it saves time in obtaining the interface image, and the intelligent agent does not need to recognize the interface image every time before proceeding to the next step, reducing the delay in the electronic device automatically searching for the target object based on the search instruction and achieving one-step access to user services.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0037] Figure 1 This is a flowchart illustrating an object search method according to an exemplary embodiment.

[0038] Figure 2 This is a flowchart illustrating an object search method based on voice commands according to an exemplary embodiment of the present disclosure.

[0039] Figure 3 This is a flowchart illustrating a method for determining an operation sequence according to an exemplary embodiment.

[0040] Figure 4 This is a flowchart illustrating a method for determining a sequence of operations according to yet another exemplary embodiment.

[0041] Figure 5 This is a flowchart illustrating a method for obtaining and executing an operation sequence according to an exemplary embodiment.

[0042] Figure 6 This is a flowchart illustrating a method for obtaining and executing an operation sequence according to yet another exemplary embodiment.

[0043] Figure 7 This is a flowchart illustrating a method for obtaining and executing an operation sequence according to yet another exemplary embodiment.

[0044] Figure 8 This is a schematic diagram illustrating a method for executing a sequence of operations according to an exemplary embodiment of the present disclosure.

[0045] Figure 9 This is a flowchart illustrating an object search method according to yet another exemplary embodiment.

[0046] Figure 10 This is a flowchart illustrating a method for determining an operation sequence according to an exemplary embodiment.

[0047] Figure 11 This is a flowchart illustrating a method for sending feedback information to an electronic device according to yet another exemplary embodiment.

[0048] Figure 12 This is a flowchart illustrating a method for determining an operation sequence and sending the operation sequence to an electronic device according to an exemplary embodiment.

[0049] Figure 13 This is a flowchart illustrating a method for determining an operation sequence and sending the operation sequence to an electronic device according to yet another exemplary embodiment.

[0050] Figure 14 This is a block diagram illustrating an object search device according to an exemplary embodiment.

[0051] Figure 15 This is a block diagram illustrating an object search device according to yet another exemplary embodiment.

[0052] Figure 16 This is a block diagram illustrating an apparatus for object searching according to an exemplary embodiment.

[0053] Figure 17 This is a block diagram illustrating an apparatus for object searching according to an exemplary embodiment. Detailed Implementation

[0054] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0055] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0056] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0057] The object search method provided in some embodiments of this disclosure is applied to scenarios where target objects contained in control commands are automatically searched based on control commands issued by the user.

[0058] With the development of artificial intelligence and speech recognition technologies, intelligent voice interaction technology has become an important component of various electronic devices, enabling users to control electronic devices to perform various functions through voice. For example, drivers can use voice commands to control the vehicle's infotainment system to search for videos, music, and applications.

[0059] In view of this, some embodiments of this disclosure provide an object search method. When an electronic device receives a search instruction related to a user's intent, an interface image corresponding to the interface displayed by the electronic device is acquired, wherein the search instruction instructs the electronic device to search for a target object matching the user's intent. Based on the search instruction and the interface image, an operation sequence including multiple operation steps is determined. A target function corresponding to the multiple operation steps is called based on a functional interface. Through the target function, the multiple operation steps included in the operation sequence are executed to obtain and display the target object. Through this disclosure, after obtaining a search instruction issued by the user, multiple operation steps are determined based on the search instruction and the corresponding screenshot of the interface displayed by the electronic device, and the multiple operation steps are executed by calling the search box function through a functional interface to search for the target object intended by the user, reducing the delay in the electronic device automatically searching for the target object based on the search instruction.

[0060] In this embodiment, a user's search command is collected via an electronic device. An intelligent agent performs model inference based on the user command and corresponding interface images collected by the terminal, and generates an operation sequence. The electronic device then executes multiple operation steps included in the operation sequence. The intelligent agent can be deployed on other devices besides the electronic device, and the electronic device obtains the operation sequence for searching the target object by interacting with these other devices. Alternatively, the intelligent agent can be deployed locally on the electronic device, and the electronic device obtains the operation sequence for searching the target object by interacting with the local intelligent agent.

[0061] Figure 1 This is a flowchart illustrating an object search method according to an exemplary embodiment. For example... Figure 1 As shown, the method includes the following steps.

[0062] In step S101, a search instruction related to the user's intent is obtained, and the first interface image corresponding to the first interface displayed by the electronic device is determined. The search instruction is used to search for a target object on the electronic device.

[0063] In step S102, an operation sequence is determined. The operation sequence is determined based on the search instruction and the first interface image, and includes multiple operation steps for searching for the target object.

[0064] In step S103, the target function corresponding to multiple operation steps is called based on the function interface. Through the target function, the multiple operation steps included in the operation sequence are executed to obtain and display the target object.

[0065] In this embodiment of the disclosure, after the electronic device obtains a search instruction related to the user's intent, the electronic device performs a content search for the target object. A first interface image corresponding to the first interface displayed by the electronic device is obtained, and an operation sequence including multiple operation steps is determined based on the first interface image and the search instruction. Specifically, when obtaining the first interface image, a screenshot of the current display interface of the electronic device can be directly taken to determine the interface image corresponding to the current display interface as the first interface image. Alternatively, the electronic device can be woken up while the screen is off, and the interface displayed after waking up can be used as the first interface, thereby obtaining the first interface image corresponding to the first interface. The search instruction related to the user's intent can be a control instruction for searching a target object, such as: searching for specified knowledge, searching for a specified application. Voice commands issued by the user can also be control instructions containing a target object search process, such as: playing specified audio / video (including a process for searching audio / video), downloading specified application (including a process for searching application details). The search instruction related to the user's intent can be a voice instruction, a text instruction, or other types of instructions. The search instruction related to the user's intent can be understood as an instruction directly input by the user, or an instruction extracted by the agent in the electronic device based on other information issued by the user. In one example, the electronic device in this embodiment of the disclosure is an in-vehicle infotainment system in a vehicle.

[0066] In this embodiment of the disclosure, after obtaining the first interface image corresponding to the first interface displayed by the electronic device based on a search instruction related to the user's intent, analysis and reasoning are performed based on the search instruction and the first interface image to determine an operation sequence for searching for a target object. The operation sequence includes multiple operation steps for searching for the target object. Finally, the multiple operation steps included in the operation sequence are executed sequentially.

[0067] In this embodiment of the disclosure, after determining the operation sequence for searching the target object, the target function corresponding to the operation steps in the operation sequence can be called through a functional interface, and the multiple operation steps mentioned above can be executed through the target function. For the multiple operation steps included in the operation sequence, the function corresponding to each operation step is called through the functional interface. Through the function corresponding to each operation step, the multiple operation steps included in the operation sequence are executed sequentially to realize the user's intent to search for the target object. Since this embodiment of the disclosure directly calls the target function corresponding to the operation sequence through the system's underlying functional interface to complete the entire search process, this embodiment of the disclosure does not require simulating the user's touch operation of the search control during the search for the target object. This embodiment of the disclosure does not require further analysis of the updated display interface after a single operation step is performed. Instead, it directly determines the operation sequence including multiple operation steps and calls the search control function through the system's underlying function interface. Then, it directly executes multiple operation steps through the called search control function, reducing the frequency of executing the operation sequence of "acquiring the display interface image + inputting the display interface image into the agent + obtaining the agent's feedback" during image search. This reduces the time consumed by repeatedly executing the above-mentioned operation sequence of "acquiring the display interface image + inputting the display interface image into the agent + obtaining the agent's feedback", thereby reducing the time delay required for the electronic device to find the target object based on the user's search command and improving the user's interactive experience.

[0068] In this embodiment, after obtaining a search command related to the user's intent, a first interface image corresponding to the first interface displayed by the electronic device is acquired. Based on the search command and the acquired first interface image, an operation sequence including multiple operation steps is determined. Finally, the multiple operation steps in the operation sequence are executed sequentially to search for the target object intended by the user. This method can directly determine the operation sequence containing multiple operation steps based on the acquired search command and the first interface image corresponding to the first interface displayed by the electronic device. Finally, multiple operation steps are executed sequentially by calling the search box function through a functional interface to search for the target object. This eliminates the need to acquire the interface image again after each operation step to determine the next operation step. Furthermore, it saves time acquiring the interface image, and the intelligent agent does not need to recognize the interface image every time before proceeding to the next step, reducing the latency of the electronic device automatically searching for the target object based on the search command, and achieving one-step access to user services.

[0069] In one embodiment of this disclosure, Figure 2 This is a flowchart illustrating an object search method based on voice commands according to an exemplary embodiment of this disclosure. Figure 2As shown, this embodiment of the present disclosure can search for target objects based on user-issued voice search commands in the following manner: After the electronic device receives the user's voice search command, it acquires a screenshot of the currently displayed interface of the electronic device. Based on the screenshot and the text search command corresponding to the voice search command, an operation sequence including multiple operation steps is determined. These multiple operation steps may include clicking the search box, clearing the historical input text in the search box, inputting the search text corresponding to the voice command into the search box, and automatically pressing Enter on the search box. After determining the operation sequence, the electronic device automatically executes the multiple operation steps included in the operation sequence to search for the target object intended by the user. After obtaining the search results and determining that the search results match the user's intent, the electronic device announces a message indicating that the execution is complete. Through the search process executed by this embodiment of the present disclosure, only the "input screenshot + model understanding" operation step needs to be executed once in the entire search process to determine multiple operation steps for searching the target object, instead of executing the "input screenshot + model understanding" operation step for each operation, thus reducing the latency required for the entire search process.

[0070] In this embodiment of the present disclosure, after receiving a user's search instruction and obtaining the first interface image corresponding to the search instruction, the electronic device can input the search instruction and the first interface image into the intelligent agent. The intelligent agent performs model reasoning based on the search instruction and the first interface image to determine an operation sequence including multiple steps, and the intelligent agent feeds back the operation sequence to the electronic device.

[0071] Figure 3 This is a flowchart illustrating a method for determining a sequence of operations according to an exemplary embodiment. Figure 3 As shown, the method includes the following steps.

[0072] In step S201, a search instruction related to the user's intent is obtained, and the first interface image corresponding to the first interface displayed by the electronic device is determined.

[0073] In step S202, the search command and the first interface image are input into the agent, and the operation sequence fed back by the agent is obtained, wherein the operation sequence is determined by the agent according to the search command and the first interface image.

[0074] In this embodiment of the disclosure, after receiving a user's search command and acquiring a first interface image, the electronic device inputs the search command and the first interface image into an intelligent agent. Upon receiving the search command and the first interface image, the intelligent agent can perform model reasoning based on the search command and the first interface image to determine the target object the user intends to search for, and determine a target search box image in the first interface. The target search box corresponding to the target search box image is used to search for the target object. Finally, the agent directly determines the operation sequence used to control the search box to search for the target object, and then determines feedback information based on the operation sequence, sending the feedback information including the operation sequence to the electronic device. Alternatively, the intelligent agent can interact with the electronic device based on the search command and the first interface image, determine the operation sequence based on further information input by the electronic device, and then send feedback information including the operation sequence to the electronic device.

[0075] In this embodiment, the electronic device inputs a received search command and a determined first interface image into an intelligent agent. The intelligent agent then executes a process to determine an operation sequence based on the search command and the first interface image. After determining the operation sequence, the intelligent agent feeds it back to the electronic device. The electronic device then executes multiple operation steps included in the operation sequence based on the received feedback from the intelligent agent to search for the target object intended by the user. This avoids the electronic device frequently executing the operation step of "inputting an interface image to the intelligent agent and obtaining feedback from the intelligent agent," effectively reducing the delay in the electronic device automatically finding the target object based on the search command and improving the user's search experience.

[0076] In this embodiment of the disclosure, when the first interface displayed by the electronic device includes a target search box for searching for a target object, the intelligent agent can directly determine the corresponding operation sequence based on the target search box in the first interface image and the user's search command, and send the operation sequence to the electronic device. The method for determining the operation sequence is described below in this embodiment of the disclosure.

[0077] In some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for a target object; the operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0078] In this embodiment of the present disclosure, after the electronic device inputs a search command and a first interface image into the intelligent agent, if the first interface corresponding to the first interface image includes a target search box for searching for a target object, the intelligent agent, after determining the target object that the user intends to search for based on the received search command, can directly determine the target search based on the target search box image in the first interface image. Furthermore, based on the target search box and the target object pointed to by the search command, an operation sequence for controlling the search box to search for the target object is determined, and the operation sequence is fed back to the electronic device.

[0079] In this embodiment, after the electronic device inputs a first interface image and a search command into the intelligent agent, if the first interface displayed by the electronic device includes a target search box for searching for a target object, the intelligent agent can directly determine the operation sequence based on the target search box image and the search command in the first interface image, and send the operation sequence to the terminal. The electronic device only needs to perform a single operation step of "inputting an interface image to the intelligent agent and obtaining feedback from the intelligent agent" to obtain the complete operation sequence, enabling the electronic device to quickly obtain and execute the operation sequence to quickly search for the target object.

[0080] In this embodiment of the disclosure, when the first interface displayed by the electronic device includes a target search box, after obtaining the operation sequence fed back by the intelligent agent, multiple operation steps included in the operation sequence can be directly executed through the target search box on the first interface without performing page jump.

[0081] Figure 4 This is a flowchart illustrating a method for determining a sequence of operations according to yet another exemplary embodiment. For example... Figure 4 As shown, the method includes the following steps.

[0082] In step S301, the search command and the first interface image are input into the agent to obtain the operation sequence fed back by the agent.

[0083] In step S302, the target function is called based on the function interface corresponding to the target search box, and multiple operation steps including the operation sequence are executed on the first interface through the target function.

[0084] In this embodiment of the present disclosure, after the electronic device inputs the search command and the first interface image into the intelligent agent, the intelligent agent can directly determine the operation sequence based on the target search box image and the search command included in the first interface image, and send the determined operation sequence to the electronic device. The operation sequence includes multiple operation steps that need to be executed based on the target search box in the first interface.

[0085] In this embodiment of the disclosure, after receiving the operation sequence, the electronic device calls the target function of the target search box based on the function interface corresponding to the target search box. The called target function includes the function corresponding to each of the multiple operation steps. Through the target function, the multiple operation steps included in the operation sequence are executed on the first interface to search for and display the target object that matches the user's intent.

[0086] According to the embodiments of this disclosure, if the first interface image corresponding to the first interface displayed by the device includes a target search box image, the intelligent agent can directly determine the operation sequence based on the search command and the target search box, and feed the operation sequence back to the electronic device. The electronic device only needs to perform a single operation step of "inputting the interface image to the intelligent agent and obtaining feedback from the intelligent agent" to obtain the complete operation sequence. Then, it can execute the obtained operation sequence on the first interface without page redirection, thereby quickly searching for the target object that the user intends to search for after receiving the search command issued by the user.

[0087] In this embodiment of the disclosure, if the first interface image input by the electronic device does not include the target search box image, the intelligent agent can generate feedback information and an operation sequence based on the user's search instructions, and feed the feedback information and operation sequence back to the electronic device, so that the electronic device enters the first interface including the target search box, and executes the operation sequence on the second interface. The method for determining and executing the operation sequence is described below in this embodiment of the disclosure.

[0088] Figure 5 This is a flowchart illustrating a method for obtaining and executing an operation sequence according to an exemplary embodiment. Figure 5 As shown, the method includes the following steps.

[0089] In step S401, the search command and the first interface image are input into the intelligent agent.

[0090] In step S402, feedback information from the agent is obtained, and the operation sequence of the agent's feedback is obtained. The feedback information is determined by the agent according to the search instruction, and the feedback information is used to instruct the electronic device to display a second interface. The second interface displays a target search box, and the operation sequence is determined according to the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0091] In step S403, the target function is called based on the function interface corresponding to the target search box, and through the target function, multiple operation steps including the operation sequence are executed on the second interface.

[0092] In this embodiment, after the electronic device inputs a search command and a first interface image into the intelligent agent, if the first interface image does not include the target search box image, the intelligent agent cannot determine the operation sequence based on the received search command and the first interface image. The intelligent agent further acquires a second interface image including the target search box image, determines the operation sequence based on the search command and the target search box in the second interface image, generates feedback information to instruct the electronic device to display the second interface, and finally sends the feedback information and the operation sequence to the electronic device. After receiving the feedback information and operation sequence from the intelligent agent, the electronic device enters the second interface and calls the target function based on the function interface corresponding to the target search box. Through the target function, it executes multiple operation steps included in the operation sequence on the second interface to search for and display the target object that matches the user's intent.

[0093] In one example, if the user issues a search command to search for a specific piece of music, and the agent determines based on the first interface image that the current display interface of the electronic device does not include a music search box, the agent can obtain the interface image (second interface image) corresponding to the music search interface, determine the operation sequence for searching for the target music based on the music search command and the music search interface image, and send the operation sequence to the electronic device.

[0094] In this embodiment, when an electronic device inputs a search command and a first interface image into an intelligent agent, if the first interface image does not include a target search box image, the intelligent agent acquires a second interface image that includes the target search box and can generate feedback information to instruct the electronic device to enter the second interface. The intelligent agent can determine an operation sequence based on the target search box image in the second interface image and the search command, and can send the operation sequence and feedback information to the electronic device. The electronic device can switch to the second interface including the target search box based on the received feedback information, and perform multiple operation steps included in the operation sequence on the second interface based on the target search box. Even when the first interface displayed by the electronic device does not include a target search box for searching for a target object, the agent can instruct the electronic device to switch to the second interface including the target search box and perform multiple operation steps included in the operation sequence on the second interface, successfully finding the target object.

[0095] In this embodiment of the present disclosure, if the first interface image does not include a search box for searching for the target object, and the second interface including the target search box can be determined by the agent based on the search instruction, then the agent can directly determine the second interface image, determine the operation sequence according to the target search box image in the second interface image and the search instruction, and send the generated feedback information and operation sequence to the electronic device.

[0096] Figure 6This is a flowchart illustrating a method for obtaining and executing an operation sequence according to yet another exemplary embodiment. For example... Figure 6 As shown, the method includes the following steps.

[0097] In step S501, the search command and the first interface image are input into the intelligent agent.

[0098] In step S502, feedback information from the intelligent agent is obtained, and an operation sequence from the intelligent agent is obtained. The feedback information is used to instruct the electronic device to display a second interface, on which a target search box is displayed. The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface. The second interface image can be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0099] In step S503, the target function is called based on the function interface corresponding to the target search box, and through the target function, multiple operation steps including the operation sequence are executed on the second interface.

[0100] In this embodiment, after the electronic device inputs a search command and a first interface image into an intelligent agent, if the intelligent agent determines, based on the search command and the first interface image, that the current display interface of the electronic device does not include a search box for searching for a target object, the intelligent agent can further acquire a second interface image containing the target search box. The intelligent agent has a pre-set set of search tasks, which includes search tasks and corresponding interface images. If the search tasks included in the search task set include the target search task corresponding to the search command, the intelligent agent can directly determine the second interface image based on the search task pointed to by the search command and the search task set, determine the operation sequence based on the target search box image in the second interface image, generate feedback information, and finally send the feedback information and the second interface image together to the electronic device. The electronic device can then enter the second interface based on the received feedback information and call the target function based on the function interface corresponding to the target search box. Through the target function, it executes multiple operation steps included in the operation sequence on the second interface to search for and display a target object that matches the user's intent.

[0101] In this system, electronic devices can pre-store some search queries into the intelligent agent based on the applications they have installed and their search capabilities. For example, they can synchronously input search queries and corresponding search interface images into the intelligent agent, which then stores these queries and images in a search task set. Upon receiving a search command, the intelligent agent can directly determine the corresponding secondary interface image for any pre-registered search queries. It is understood that electronic devices can periodically send synchronization information to the intelligent agent based on application installation / uninstallation status, allowing the agent to add or remove search queries and corresponding search interface images from its task execution list.

[0102] In the embodiments of this disclosure, when the first interface image does not include the target search box image, the agent cannot identify the target search box in the first interface based on the user command and the first interface image. In this case, the agent can determine a second interface image including the target search box image based on the search command. The operation sequence is directly determined based on the target search box image in the second interface image and the search command. The operation sequence and feedback information are sent to the electronic device, which then enters the second interface and searches for the target object based on the target search box. Even in scenarios where the first interface image input by the electronic device does not include the target search box, the electronic device only needs to perform a single operation step of "inputting an interface image to the agent and obtaining feedback from the agent" to obtain the complete operation sequence, thus quickly finding the target object.

[0103] In this embodiment of the present disclosure, if the first interface corresponding to the first interface image input by the electronic device does not include a target search box, the intelligent agent can still generate feedback information and send the feedback information to the electronic device. Based on the feedback information, the electronic device enters a second interface including the target search box and inputs the second interface image corresponding to the second interface into the intelligent agent. The intelligent agent then determines an operation sequence based on the second interface image and user instructions, and sends the operation sequence to the electronic device. The method for determining the operation sequence is described below in this embodiment of the present disclosure.

[0104] Figure 7 This is a flowchart illustrating a method for obtaining and executing an operation sequence according to yet another exemplary embodiment. For example... Figure 7 As shown, the method includes the following steps.

[0105] In step S601, the search command and the first interface image are input into the intelligent agent.

[0106] In step S602, feedback information from the intelligent agent is obtained. This feedback information is used to instruct the electronic device to display a second interface, on which a target search box is displayed. The image of the second interface cannot be determined by the intelligent agent based on the search command and a preset set of search tasks. The set of search tasks includes search tasks and the interface images corresponding to the search tasks. The search tasks included in the set do not include the target search task corresponding to the search command.

[0107] In step S603, a second interface image is acquired and input into the intelligent agent. The second interface image includes a target search box image.

[0108] In step S604, the operation sequence fed back by the intelligent agent is obtained, the target function is called based on the function interface corresponding to the target search box, and multiple operation steps included in the operation sequence are executed on the second interface through the target function.

[0109] In this embodiment, after the electronic device inputs a search command and a first interface image into the intelligent agent, if the intelligent agent determines, based on the search command and the first interface image, that the currently displayed interface of the electronic device does not include a search box for searching for a target object, the intelligent agent needs to acquire a second interface image containing the target search box. If the pre-set set of search tasks in the intelligent agent does not include the target search task specified in the search command, the intelligent agent cannot directly determine the second interface image based on the search command. In this case, the intelligent agent generates feedback information to instruct the electronic device to display the second interface and sends the feedback information to the electronic device. The electronic device then enters the second interface based on the received feedback information, acquires the second interface image corresponding to the second interface, and inputs the second interface image into the intelligent agent. The intelligent agent determines an operation sequence based on the second interface image sent by the electronic device and the search command, and sends the operation sequence to the electronic device. After acquiring the operation sequence, the electronic device calls the target function based on the function interface corresponding to the target search box, and through the target function, executes multiple operation steps included in the operation sequence on the second interface to search for and display the target object that matches the user's intent.

[0110] For example, if the user issues a search command to search for a specific piece of music, and the agent determines, based on the first interface image, that the current display interface of the electronic device does not include a music search box, then the agent can determine the audio application corresponding to the music search command. After determining the target application corresponding to the search command, the agent sends feedback information containing the target application to the electronic device. The electronic device then opens the music application based on the feedback information and inputs the interface image corresponding to the music application interface into the agent.

[0111] In the embodiments of this disclosure, when the first interface image does not include the target search box image, the intelligent agent cannot identify the target search box in the first interface based on the user command and the first interface image. In this case, the intelligent agent can generate feedback information according to the search command and send the feedback information to the electronic device. The electronic device then enters the second interface based on the feedback information and sends the second interface image to the intelligent agent. The electronic device determines the operation sequence based on the received second interface image and the search command. The operation sequence and feedback information are sent to the electronic device, which then enters the second interface and searches for the target object based on the target search box. For scenarios where the first interface image input by the electronic device does not include the target search box, the electronic device only needs to perform two operation steps—"inputting the interface image to the intelligent agent and obtaining feedback from the intelligent agent"—to obtain the complete operation sequence, and then execute the obtained operation sequence. This eliminates the need to repeatedly perform "inputting the interface image to the intelligent agent and obtaining feedback from the intelligent agent" to determine and execute each operation step individually, thereby improving the versatility of this application and effectively reducing the time delay for the electronic device to search for the target object based on the user command.

[0112] In this embodiment of the disclosure, after the electronic device obtains the operation sequence fed back by the intelligent agent, it calls the function of the target search box through the function interface and searches for the target object through the called function.

[0113] In some embodiments of this disclosure, the sequentially executed multiple operation steps include: triggering the target search box; clearing the historical search text already entered into the target search box; entering the search text corresponding to the search command into the target search box; and triggering the target search box to start a search.

[0114] Figure 8 This is a schematic diagram illustrating a method for executing a sequence of operations according to an exemplary embodiment of this disclosure. Figure 8 As shown, after obtaining the operation sequence fed back by the intelligent agent, this embodiment of the disclosure can search for a target object that matches the user's intent in the following manner: Triggering the search box in the display interface. Deleteing the text already entered in the search box. Entering the search text "XXX" corresponding to the target object contained in the user's instruction into the search box. Executing the search function to search for content related to the target object "XXX". Displaying the search results in the display interface, the search results include search results that match the user's intent (such as search result 1: XXX), and also other search results related to the search text (such as search result 2: XXXBB, search result 3: AAXXX).

[0115] The following embodiments of this disclosure describe a method for determining feedback information in the cloud based on a search command sent by an electronic device and a first interface image, and sending the feedback information to the electronic device.

[0116] Figure 9 This is a flowchart illustrating an object search method according to yet another exemplary embodiment. For example... Figure 9 As shown, the method includes the following steps.

[0117] In step S701, the search command sent by the electronic device and the first interface image are obtained. The search command is used to search for the target object on the electronic device, and the first interface image corresponds to the first interface displayed by the electronic device.

[0118] In step S702, an operation sequence is determined based on the search instruction and the first interface image. The operation sequence includes multiple operation steps for searching for the target object.

[0119] In step S703, an operation sequence is sent to the electronic device. The operation sequence is used to instruct the electronic device to call a target function with multiple operation steps based on the function interface, and to execute the multiple operation steps included in the operation sequence through the target function in order to acquire and display the target object on the electronic device.

[0120] In this embodiment, after the electronic device obtains a search instruction related to the user's intent, it further obtains a first interface image corresponding to the first interface displayed by the electronic device, and sends the search instruction and the first interface image to the cloud. Upon receiving the search instruction and the first interface image, the cloud can use an agent deployed on the cloud side to perform model inference based on the search instruction and the first interface image to determine the target object of the user's intent search, determine a search box for searching the target object, and ultimately directly determine the operation sequence for controlling the search box to search for the target object. Then, based on the operation sequence, it determines feedback information and sends the feedback information, including the operation sequence, to the electronic device. Alternatively, the cloud can interact with the electronic device based on the search instruction and the first interface image, determine the operation sequence based on further information sent by the electronic device, and then send feedback information, including the operation sequence, to the electronic device.

[0121] In this embodiment, after the electronic device receives the operation sequence for searching the target object, it calls the function corresponding to each operation step in the operation sequence through a functional interface. The multiple operation steps are executed sequentially through the function corresponding to each operation step to search for the target object intended by the user. Since this embodiment directly calls the target function corresponding to the operation sequence through the system's underlying functional interface to complete the entire search process, it does not need to perform further analysis on the updated display interface after executing a single operation step and the display interface is updated. Instead, it directly determines the operation sequence including multiple operation steps and calls the search control function through the system's underlying functional interface. Then, it directly executes multiple operation steps through the called search control function, reducing the frequency of executing the "acquiring display interface image + inputting display interface image into the agent + obtaining agent feedback" operation sequence during image search. This reduces the time consumed by repeatedly executing the above "acquiring display interface image + inputting display interface image into the agent + obtaining agent feedback" operation sequence, reducing the time delay required for the electronic device to find the target object based on the user's search command, and improving the user's interactive experience.

[0122] In this embodiment, the electronic device sends the received search command and the determined first interface image to the cloud. The cloud then executes a process to determine an operation sequence based on the search command and the first interface image. After determining the operation sequence in the cloud, the cloud feeds back the operation sequence to the electronic device. Based on the operation sequence received from the cloud, the electronic device performs multiple operation steps by calling the target function through a functional interface to search for the target object intended by the user. This avoids the electronic device frequently performing the operation step of "sending the interface image to the cloud and obtaining cloud feedback," effectively reducing the delay in the electronic device automatically finding the target object based on the search command and improving the user's search experience.

[0123] In this embodiment of the disclosure, when the first interface displayed by the electronic device includes a target search box for searching for a target object, the cloud can directly determine the corresponding operation sequence based on the target search box in the first interface image and the user's search command, and send the operation sequence to the electronic device. The method for determining the operation sequence by the cloud is described below in this embodiment of the disclosure.

[0124] Figure 10 This is a flowchart illustrating a method for determining a sequence of operations according to an exemplary embodiment. Figure 10 As shown, the method includes the following steps.

[0125] In step S801, the search command and the first interface image sent by the electronic device are acquired.

[0126] In step S802, the operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0127] In this embodiment of the disclosure, after the electronic device sends a search command and a first interface image to the cloud, if the first interface corresponding to the first interface image includes a target search box for searching for a target object, the cloud, after determining the target object the user intends to search for based on the received search command, can directly determine the target search based on the target search box image in the first interface image. Then, based on the target search box and the target object pointed to by the search command, an operation sequence for controlling the search box to search for the target object is determined, and the operation sequence is fed back to the electronic device.

[0128] In this embodiment, after the electronic device sends a first interface image and a search command to the cloud, if the first interface displayed by the electronic device includes a target search box for searching for a target object, the cloud can directly determine the operation sequence based on the target search box image and the search command in the first interface image, and send the operation sequence to the terminal. For the electronic device, only one operation step of "sending the interface image to the cloud and obtaining cloud feedback" is required to obtain the complete operation sequence, enabling the electronic device to quickly obtain and execute the operation sequence to quickly search for the target object.

[0129] In this embodiment of the disclosure, if the first interface image sent by the electronic device does not include the target search box image, the cloud can generate feedback information and an operation sequence based on the user's search instructions, and feed the feedback information and operation sequence back to the electronic device, so that the electronic device enters the first interface including the target search box, and executes the operation sequence on the second interface. The following embodiments of this disclosure further illustrate the method of the cloud determining the operation sequence and sending the operation sequence to the electronic device.

[0130] Figure 11 This is a flowchart illustrating a method for sending feedback information to an electronic device according to yet another exemplary embodiment. For example... Figure 11 As shown, the method includes the following steps.

[0131] In step S901, the search command and the first interface image sent by the electronic device are acquired.

[0132] In step S902, in response to the fact that the first interface image does not include the target search box image, feedback information is sent to the electronic device. The feedback information is used to instruct the electronic device to display a second interface, on which the target search box is displayed. The target search box is used to search for the target object, and the operation sequence is determined according to the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0133] In this embodiment, when the electronic device sends a search command and a first interface image to the cloud, if the first interface image does not include a target search box image, the cloud obtains a second interface image including the target search box and can generate feedback information to instruct the electronic device to enter the second interface. The cloud can determine an operation sequence based on the target search box image in the second interface image and the search command, and can send the operation sequence and feedback information to the electronic device. The electronic device can switch to the second interface including the target search box based on the received feedback information, and perform multiple operation steps included in the operation sequence on the second interface based on the target search box. When the first interface displayed by the electronic device does not include a target search box for searching for a target object, the cloud can instruct the electronic device to switch to the second interface including the target search box and perform multiple operation steps included in the operation sequence on the second interface to successfully search for the target object. For example, for audio applications. After determining the target application corresponding to the search command, the cloud feeds back the target application to the electronic device.

[0134] In this embodiment of the disclosure, if the first interface image does not include a search box for searching for the target object, and the second interface including the target search box can be determined by the cloud based on the search command, then the cloud can directly determine the second interface image, determine the operation sequence based on the target search box image in the second interface image and the search command, and send the generated feedback information and operation sequence to the electronic device. The following embodiments of this disclosure further illustrate the method of determining the operation sequence by the cloud and sending the operation sequence to the electronic device.

[0135] Figure 12 This is a flowchart illustrating a method for determining an operation sequence and sending the operation sequence to an electronic device according to an exemplary embodiment. Figure 12 As shown, the method includes the following steps.

[0136] In step S1001, the search command and the first interface image sent by the electronic device are acquired.

[0137] In step S1002, in response to the fact that the first interface image does not include the target search box image, a second interface image is determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and the interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0138] In step S1003, the operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0139] In step S1004, feedback information is sent to the electronic device, and an operation sequence is sent to the electronic device. The feedback information is used to instruct the electronic device to display the second interface.

[0140] In this embodiment, if the first interface image does not include the target search box image, the cloud cannot identify the target search box in the first interface based on the user command and the first interface image. The cloud pre-sets a set of search tasks, which includes search tasks and their corresponding interface images. After receiving the search command and the first interface image sent by the electronic device, if the search tasks included in the search task set include the target search task corresponding to the search command, the cloud can determine a second interface image including the target search box image based on the target search task pointed to by the search command and the search task set. The operation sequence is directly determined based on the target search box image in the second interface image and the search command. The operation sequence and feedback information are sent to the electronic device, which then enters the second interface and searches for the target object based on the target search box. Even in scenarios where the first interface image sent by the electronic device does not include the target search box, the electronic device only needs to perform a single operation step of "sending the interface image to the cloud and receiving cloud feedback" to obtain the complete operation sequence, thus quickly finding the target object.

[0141] In this embodiment of the disclosure, if the first interface corresponding to the first interface image sent by the electronic device does not include a target search box, the cloud can still generate feedback information and send the feedback information to the electronic device. The electronic device, based on the feedback information, enters a second interface including the target search box and sends the second interface image corresponding to the second interface to the cloud. The cloud then determines an operation sequence based on the second interface image and user instructions, and sends the operation sequence to the electronic device. The following embodiments of this disclosure further illustrate the method of the cloud determining the operation sequence and sending the operation sequence to the electronic device.

[0142] Figure 13 This is a flowchart illustrating a method for determining an operation sequence and sending the operation sequence to an electronic device, according to yet another exemplary embodiment. For example... Figure 13 As shown, the method includes the following steps.

[0143] In step S1101, the search command and the first interface image sent by the electronic device are acquired.

[0144] In step S1102, in response to the first interface image not including the target search box image, feedback information is sent to the electronic device, and an operation sequence is sent to the electronic device. The feedback information is used to instruct the electronic device to display the second interface. The search task set includes the search tasks that do not include the target search task corresponding to the search instruction. The search task set includes the search tasks and the interface image corresponding to the search tasks.

[0145] In step S1103, the second interface image sent by the electronic device is acquired, and the operation sequence is determined according to the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0146] In step S1104, an operation sequence is sent to the electronic device.

[0147] In this embodiment, if the first interface image does not include the target search box image, the cloud cannot identify the target search box in the first interface based on the user command and the first interface image. After obtaining the search command and the first interface image sent by the electronic device, if the preset search task set in the cloud does not include the target search task corresponding to the search command, the cloud cannot determine the second interface image containing the target search box based on the search command. In this case, the cloud can generate feedback information based on the search command and send the feedback information to the electronic device. The electronic device then enters the second interface based on the feedback information and sends the second interface image to the cloud. The electronic device determines the operation sequence based on the received second interface image and the search command. The operation sequence and feedback information are sent to the electronic device, and the electronic device enters the second interface and searches for the target object based on the target search box in the second interface. For the scenario where the first interface image sent by the electronic device does not include the target search box, the electronic device only needs to perform two operation steps of "sending the interface image to the cloud and obtaining cloud feedback" to obtain the complete operation sequence, and then execute the obtained operation sequence. Instead of repeatedly executing the steps of "sending interface images to the cloud and obtaining cloud feedback" to determine and perform operations one by one, this application improves versatility and effectively reduces the time delay for electronic devices to search for target objects based on user commands.

[0148] In this embodiment, when an electronic device receives a search instruction related to a user's intent, it acquires a first interface image corresponding to the first interface displayed by the electronic device. The search instruction instructs the electronic device to search for a target object matching the user's intent. The search instruction and the first interface image are input into an intelligent agent, and an operation sequence fed back by the intelligent agent is obtained. Multiple operation steps are called based on a functional interface to a target function. Through the target function, multiple operation steps included in the operation sequence are executed to obtain and display the target object. This disclosure eliminates the need to acquire the interface image again after each operation step to determine the next operation step. Furthermore, it saves time acquiring the interface image, and the intelligent agent does not need to recognize the interface image every time before proceeding, reducing the delay in the electronic device automatically finding the target object based on the search instruction and achieving one-step access to user services.

[0149] Based on the same concept, this disclosure also provides an object search device 100.

[0150] It is understood that the object search device 100 provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0151] Figure 14 This is a block diagram illustrating an object search device 100 according to an exemplary embodiment. (Refer to...) Figure 14 The device includes a determining unit 101, a processing unit 102, and an execution unit 103.

[0152] The determining unit 101 is used to obtain a search instruction related to the user's intent and determine the first interface image corresponding to the first interface displayed by the electronic device. The search instruction is used to search for a target object on the electronic device.

[0153] The processing unit 102 is used to determine an operation sequence, which is determined based on the search instruction and the first interface image, and includes multiple operation steps for searching for the target object.

[0154] The execution unit 103 is used to call the target function corresponding to multiple operation steps based on the function interface, and execute multiple operation steps included in the operation sequence through the target function to obtain and display the target object.

[0155] According to some embodiments of this disclosure, the processing unit 102 determines the operation sequence in the following manner: inputting the search command and the first interface image into the agent, and obtaining the operation sequence fed back by the agent, wherein the operation sequence is determined by the agent based on the search command and the first interface image.

[0156] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for a target object. The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0157] According to some embodiments of this disclosure, the execution unit 103 executes multiple operation steps included in the operation sequence in the following manner: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the first interface through the target function.

[0158] According to some embodiments of this disclosure, the first interface image does not include a target search box image, which is used to search for a target object. The processing unit 102 is further configured to: acquire feedback information from the agent, the feedback information being determined by the agent according to a search instruction, and the feedback information being used to instruct the electronic device to display a second interface, on which the target search box is displayed. The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0159] According to some embodiments of this disclosure, the second interface image can be determined by an intelligent agent based on a search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0160] According to some embodiments of this disclosure, the second interface image cannot be determined by the intelligent agent based on a search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The processing unit 102 is further configured to: display the second interface based on feedback information; acquire the second interface image, which includes the target search box image; and input the second interface image into the intelligent agent.

[0161] According to some embodiments of this disclosure, the execution unit 103 executes multiple operation steps included in the operation sequence in the following manner: calling the target function based on the function interface corresponding to the target search box, and executing multiple operation steps included in the operation sequence on the second interface through the target function.

[0162] Figure 15 This is a block diagram illustrating an object search device 200 according to an exemplary embodiment. (Refer to...) Figure 15 The device includes an acquisition unit 201, a processing unit 202, and a transmission unit 203.

[0163] The acquisition unit 201 is used to acquire the search command and the first interface image sent by the electronic device. The search command is used to search for the target object on the electronic device, and the first interface image corresponds to the first interface displayed by the electronic device.

[0164] The processing unit 202 is used to determine an operation sequence based on the search instruction and the first interface image. The operation sequence includes multiple operation steps for searching for the target object.

[0165] The sending unit 203 is used to send an operation sequence to the electronic device. The operation sequence is used to instruct the electronic device to call a target function with multiple operation steps based on the function interface, and to execute the multiple operation steps included in the operation sequence through the target function in order to acquire and display the target object on the electronic device.

[0166] According to some embodiments of this disclosure, the first interface image includes a target search box image, which is used to search for a target object. The processing unit 202 determines the operation sequence based on the search instruction and the first interface image in the following manner: The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

[0167] According to some embodiments of this disclosure, the first interface image does not include a target search box image, which is used to search for a target object. The processing unit 202 is further configured to: send feedback information to the electronic device, the feedback information instructing the electronic device to display a second interface, on which the target search box is displayed. The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

[0168] According to some embodiments of this disclosure, the second interface image can be determined based on a search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

[0169] According to some embodiments of this disclosure, the second interface image cannot be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The processing unit 202 is further configured to: receive the second interface image sent by the electronic device, wherein the second interface image includes a target search box image.

[0170] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0171] Figure 16 This is a block diagram illustrating an apparatus 300 for object search according to an exemplary embodiment. The apparatus 300 can be provided as an electronic device, such as a terminal. For example, the apparatus 300 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, in-vehicle terminal, etc.

[0172] Reference Figure 16 The device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0173] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0174] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0175] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0176] Multimedia component 308 includes a screen that provides an output interface between device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0177] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0178] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0179] Sensor assembly 314 includes one or more sensors for providing state assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0180] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0181] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0182] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0183] Figure 17 This is a block diagram illustrating an apparatus 400 for object search according to an exemplary embodiment. For example, apparatus 400 may be provided as a server, such as in the cloud. (See also...) Figure 17The apparatus 400 includes a processing component 422, which further includes one or more processors, and memory resources represented by memory 432 for storing instructions, such as application programs, that can be executed by the processing component 422. The application programs stored in memory 432 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 422 is configured to execute instructions to perform the search method described above.

[0184] Device 400 may also include a power supply component 426 configured to perform power management of device 400, a wired or wireless network interface 450 configured to connect device 400 to a network, and an input / output (I / O) interface 458. Device 400 may operate on an operating system stored in memory 432, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0185] In some embodiments of this disclosure, a storage medium including instructions is also provided, such as a memory 432 including instructions, which can be executed by the processing component 422 of the device 400 to perform the above-described method. For example, the storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0186] In some embodiments of this disclosure, a storage medium is provided, which may be a non-transitory computer-readable storage medium.

[0187] In some embodiments of this disclosure, when instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the object search method described above.

[0188] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the object search method involved in any of the above embodiments.

[0189] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0190] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, a first component, part, region, layer, or section mentioned in the examples may also be referred to as a second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature.

[0191] In this description, "multiple" means at least two, referring to two or more, such as two, three, etc., unless otherwise explicitly specified. Other quantifiers are similar. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, unless otherwise specified or clearly indicated from the context, the articles "a" and "an" as used in this application and the appended claims are generally understood to mean "one or more."

[0192] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of two or more; "and / or" describes the association relationship between related objects, indicating that three relationships may exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Similarly, "at least one of..." includes any one of the related listed items and any combination of two or more.

[0193] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0194] Furthermore, the term "exemplary" is used herein to indicate that it serves as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term "exemplary" is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to indicate an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to indicate any of the natural inclusive permutations. That is, if X applies A; X applies B; or X applies both A and B, then applying A or B satisfies the condition under any of the foregoing instances.

[0195] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in this disclosure, such terms are intended to be inclusive in a manner similar to the term “including.”

[0196] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0197] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method of searching for an object, characterized by, include: Obtain search instructions related to user intent and determine the first interface image corresponding to the first interface displayed by the electronic device, wherein the search instructions are used to search for target objects on the electronic device; A sequence of operations is determined based on the search instruction and the first interface image, and includes multiple operation steps for searching the target object; Based on the function interface, multiple operation steps are called to the target function. Through the target function, multiple operation steps included in the operation sequence are executed to obtain and display the target object.

2. The method of claim 1, wherein, The determination of the operation sequence includes: The search command and the first interface image are input into the agent, and the operation sequence fed back by the agent is obtained, wherein the operation sequence is determined by the agent based on the search command and the first interface image.

3. The method of claim 2, wherein, The first interface image includes a target search box image, which is used to search for the target object; The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

4. The method of claim 3, wherein, The method of calling a target function corresponding to multiple operation steps based on a functional interface, and executing multiple operation steps included in the operation sequence through the target function, includes: The target function is invoked based on the function interface corresponding to the target search box, and through the target function, multiple operation steps included in the operation sequence are executed on the first interface.

5. The method of claim 2, wherein, The first interface image does not include the target search box image, which is used to search for the target object; The method further includes: The feedback information of the intelligent agent is obtained. The feedback information is determined by the intelligent agent according to the search instruction, and the feedback information is used to instruct the electronic device to display a second interface, on which the target search box is displayed. The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

6. The method of claim 5, wherein, The second interface image can be determined by the agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

7. The method according to claim 5, characterized in that, The second interface image cannot be determined by the agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The method further includes: Based on the feedback information, a second interface is displayed, and an image of the second interface is obtained, the second interface image including the image of the target search box; The second interface image is input into the intelligent agent.

8. The method according to any one of claims 5 to 7, characterized in that, The method of calling a target function corresponding to multiple operation steps based on a functional interface, and executing multiple operation steps included in the operation sequence through the target function, includes: The target function is invoked based on the function interface corresponding to the target search box, and through the target function, multiple operation steps included in the operation sequence are executed on the second interface.

9. An object search method, characterized in that, The method includes: The system acquires a search command and a first interface image sent by an electronic device. The search command is used to search for a target object on the electronic device, and the first interface image corresponds to a first interface displayed by the electronic device. Based on the search instruction and the first interface image, an operation sequence is determined, the operation sequence including multiple operation steps for searching the target object; The operation sequence is sent to the electronic device, which instructs the electronic device to call a target function with multiple operation steps based on a function interface, and to execute the multiple operation steps included in the operation sequence through the target function, so as to acquire and display the target object on the electronic device.

10. The method according to claim 9, characterized in that, The first interface image includes a target search box image, which is used to search for the target object; The step of determining the operation sequence based on the search instruction and the first interface image includes: The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

11. The method according to claim 9, characterized in that, The first interface image does not include the target search box image, which is used to search for the target object; The method further includes: Send feedback information to the electronic device, the feedback information being used to instruct the electronic device to display a second interface, on which the target search box is displayed; The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

12. The method according to claim 11, characterized in that, The second interface image can be determined based on the search command and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search command.

13. The method according to claim 11, characterized in that, The second interface image cannot be determined based on the search command and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search command. The method further includes: The device receives the second interface image sent by the electronic device, the second interface image including the target search box image.

14. An object search device, characterized in that, include: The determining unit is used to acquire a search instruction related to the user's intent and determine the first interface image corresponding to the first interface displayed by the electronic device, wherein the search instruction is used to search for a target object on the electronic device; A processing unit is configured to determine an operation sequence, the operation sequence being determined based on the search instruction and the first interface image, and including multiple operation steps for searching the target object; An execution unit is used to call a target function corresponding to multiple operation steps based on a functional interface, and to execute multiple operation steps included in the operation sequence through the target function in order to obtain and display the target object.

15. The apparatus according to claim 14, characterized in that, The processing unit determines the operation sequence in the following manner: The search command and the first interface image are input into the agent, and the operation sequence fed back by the agent is obtained, wherein the operation sequence is determined by the agent based on the search command and the first interface image.

16. The apparatus according to claim 15, characterized in that, The first interface image includes a target search box image, which is used to search for the target object; The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

17. The apparatus according to claim 16, characterized in that, The execution unit calls the target function corresponding to multiple operation steps based on the function interface in the following manner, and executes the multiple operation steps included in the operation sequence through the target function: The target function is invoked based on the function interface corresponding to the target search box, and through the target function, multiple operation steps included in the operation sequence are executed on the first interface.

18. The apparatus according to claim 15, characterized in that, The first interface image does not include the target search box image, which is used to search for the target object; The processing unit is also used for: The feedback information of the intelligent agent is obtained. The feedback information is determined by the intelligent agent according to the search instruction, and the feedback information is used to instruct the electronic device to display a second interface, on which the target search box is displayed. The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

19. The apparatus according to claim 18, characterized in that, The second interface image can be determined by the agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search instruction.

20. The apparatus according to claim 18, characterized in that, The second interface image cannot be determined by the intelligent agent based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks do not include the target search task corresponding to the search instruction. The processing unit is further configured to: Based on the feedback information, a second interface is displayed, and an image of the second interface is obtained, the second interface image including the image of the target search box; The second interface image is input into the intelligent agent.

21. The apparatus according to any one of claims 18 to 20, characterized in that, The execution unit calls the target function corresponding to multiple operation steps based on the function interface in the following manner, and executes the multiple operation steps included in the operation sequence through the target function: The target function is invoked based on the function interface corresponding to the target search box, and through the target function, multiple operation steps included in the operation sequence are executed on the second interface.

22. An object search device, characterized in that, The device includes: The acquisition unit is used to acquire a search instruction and a first interface image sent by an electronic device. The search instruction is used to search for a target object on the electronic device, and the first interface image corresponds to the first interface displayed by the electronic device. The processing unit is configured to determine an operation sequence based on the search instruction and the first interface image, the operation sequence including multiple operation steps for searching the target object; A sending unit is configured to send the operation sequence to the electronic device, the operation sequence being configured to instruct the electronic device to call a target function with multiple operation steps based on a function interface, and to execute the multiple operation steps included in the operation sequence through the target function, so as to acquire and display the target object on the electronic device.

23. The apparatus according to claim 22, characterized in that, The first interface image includes a target search box image, which is used to search for the target object; The processing unit determines the operation sequence based on the search instruction and the first interface image in the following manner: The operation sequence is determined based on the search instruction and the target search box image in the first interface image.

24. The apparatus according to claim 22, characterized in that, The first interface image does not include the target search box image, which is used to search for the target object; The processing unit is also used for: Send feedback information to the electronic device, the feedback information being used to instruct the electronic device to display a second interface, on which the target search box is displayed; The operation sequence is determined based on the search instruction and the target search box image in the second interface image corresponding to the second interface.

25. The apparatus according to claim 24, characterized in that, The second interface image can be determined based on the search command and a preset set of search tasks. The set of search tasks includes search tasks and interface images corresponding to the search tasks. The search tasks included in the set of search tasks include the target search task corresponding to the search command.

26. The apparatus according to claim 24, characterized in that, The second interface image cannot be determined based on the search instruction and a preset set of search tasks. The set of search tasks includes search tasks and corresponding interface images. The search tasks included in the set do not include the target search task corresponding to the search instruction. The processing unit is further configured to: The device receives the second interface image sent by the electronic device, the second interface image including the target search box image.

27. An electronic device, characterized in that, include: processor: Memory used to store computer programs or instructions that can be executed by a processor; The processor is configured to execute the computer program or instructions to implement the object search method of any one of claims 1 to 8, or to implement the object search method of any one of claims 9 to 13.

28. A storage medium, characterized in that, The storage medium stores a computer program or instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the object search method of any one of claims 1 to 8, or the object search method of any one of claims 9 to 13.

29. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the object search method as described in any one of claims 1 to 8, or implements the object search method as described in any one of claims 9 to 13.

Images