Picture searching method, operating system configuration method, electronic equipment, computer storage medium and computer program product

Abstract

The embodiment of the invention provides a picture searching method, an operating system configuration method, electronic equipment, a computer storage medium and a computer program product. The picture searching method is applied to the electronic equipment provided with a mobile operating system and comprises the steps that in a first browsing page, triggering operation for a suspension control of the mobile operating system is obtained, and the suspension control is suspended on the upper layer of the first browsing page; in response to the trigger operation, a target application program is called to execute screen capture on the first browsing page, and the target application program is a current desktop application or a second desktop application; and executing picture search on at least part of the screen capture results, and calling a small window or skipping to a second browsing page to display search results of the at least part of the screen capture results. Cross-application search between the target application program and the application program of the first browsing page is achieved through convenient operation, and user experience fluency is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of Internet technology, and in particular to an image search method, an operating system configuration method, an electronic device, a computer storage medium, and a computer program product. Background Art

[0002] Generally speaking, a mobile operating system is an operating system designed for mobile devices such as smartphones and tablets, which is used to manage and control the hardware resources, software applications, and user interactions of mobile devices, such as embedded operating systems. Mobile operating systems are the core software platform of mobile devices, providing users with an environment for running applications, managing data, and communicating.

[0003] Mobile operating systems such as IOS are limited in the openness of the system components they define due to security considerations, which has a certain impact on the development of desktop applications for mobile operating systems. For example, mobile operating systems do not support convenient cross-application search operations. In this case, users are required to manually switch between the current browsing application and the search application in order to query the content of the current browsing application through the search application. For example, when a user needs help with content in a question-solving application, they need to manually query it in the search application, resulting in a poor user experience in the search application. Summary of the invention

[0004] In view of this, embodiments of the present invention provide an image search method, an operating system configuration method, an electronic device, a computer storage medium, and a computer program product to solve the above problems.

[0005] According to a first aspect of an embodiment of the present invention, there is provided an image search method, which is applied to an electronic device installed with a mobile operating system, the method comprising: in a first browsing page, obtaining a trigger operation for a floating control of the mobile operating system, the floating control floating on an upper layer of the first browsing page; in response to the trigger operation, calling a target application to take a screenshot of the first browsing page, the target application being a current desktop application or a second desktop application; performing an image search on at least part of the screenshot results, and bringing up a small window or jumping to a second browsing page to display the search results of at least part of the screenshot results.

[0006] According to a second aspect of an embodiment of the present invention, there is provided an operating system configuration method, which is applied to an electronic device installed with a mobile operating system, the method comprising: obtaining a predefined combination instruction, the predefined combination instruction comprising a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction is configured to respond to a trigger operation of a floating control of the mobile operating system, call a target application to perform a screenshot of a current display page of the electronic device, the target application is a current desktop application or a second desktop application, wherein the search instruction is configured to perform an image search on at least part of the screenshot results, and bring up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results; and configure the combination instruction into the mobile operating system.

[0007] According to a third aspect of an embodiment of the present invention, there is provided an electronic device having a mobile operating system installed thereon, comprising: an acquisition module for acquiring, in a first browsing page, a trigger operation for a floating control of the mobile operating system, the floating control floating on an upper layer of the first browsing page; a calling module for responding to the trigger operation, calling a target application to take a screenshot of the first browsing page, the target application being a current desktop application or a second desktop application; a search module for performing an image search on at least part of the screenshot results, and bringing up a small window or jumping to a second browsing page to display search results of at least part of the screenshot results.

[0008] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device having a mobile operating system installed thereon, comprising: an acquisition module for acquiring predefined combination instructions, the predefined combination instructions comprising a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction being configured to respond to a trigger operation of a floating control of the mobile operating system, call a target application to perform a screenshot of a current display page of the electronic device, the target application being a current desktop application or a second desktop application, wherein the search instruction is configured to perform an image search of the target application of the mobile operating system on at least part of the screenshot results, and bring up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results; and a configuration module for configuring the combination instruction into the mobile operating system.

[0009] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the method described in the first aspect or the second aspect.

[0010] According to a sixth aspect of an embodiment of the present invention, there is provided a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0011] According to a seventh aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the method described in the first aspect or the second aspect.

[0012] In the scheme of the embodiment of the present invention, in response to the trigger operation of the floating control of the mobile operating system, the target application is called to perform a screenshot of the first browsing page, and the content of the first browsing page can be obtained without manually switching applications. Then, when an image search is performed on at least part of the screenshot results, search results of at least part of the screenshot results are obtained. Through convenient operations, cross-application search between the target application and the application of the first browsing page is implemented, thereby improving the smoothness of user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1 A schematic diagram of a search network architecture applicable to some embodiments of the present invention.

[0015] Figure 2 The present invention is a flowchart of the steps of the image search method according to some embodiments of the present invention.

[0016] Figure 3A for Figure 2 A flowchart of the steps of some example image search methods of an embodiment.

[0017] Figure 3B for Figure 3A A schematic diagram of page switching of an image search method according to an example.

[0018] Figure 4A for Figure 2 A flowchart of the steps of some other example image search methods of the embodiments.

[0019] Figure 4B for Figure 4A A schematic diagram of page switching of an image search method according to an example.

[0020] Figure 5A for Figure 2A flowchart of the steps of some other example image search methods of the embodiments.

[0021] Figure 5B for Figure 5A A schematic diagram of page switching of an image search method according to an example.

[0022] Fig. 6A for Figure 2 A flowchart of the steps of some other example image search methods of the embodiments.

[0023] Figure 6B for Fig. 6A A schematic diagram of page switching of an image search method according to an example.

[0024] Figure 7 A flowchart of the steps of the operating system configuration method according to some other embodiments of the present invention.

[0025] Figure 8 is a structural block diagram of an electronic device according to some other embodiments of the present invention.

[0026] Fig. 9 is a structural block diagram of an electronic device according to some other embodiments of the present invention.

[0027] Fig.10 Schematic diagram of the structure of electronic devices according to other embodiments of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0029] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0030] Figure 1 FIG. 1 is a schematic diagram of a search network architecture 100 applicable to some embodiments of the present invention. Figure 1As shown, the search network architecture 100 includes a client system 120 and a search server system 160 connected to each other through a network 110. Among them, the video resource server 132 bypasses the network 110 and the search server system 160. The reading resource server 142 is connected to the search server system 160 through the network 110. In addition, the video resource server 132 can be connected to the search server system 160 through the network 110. The reading resource server 142 can bypass the network 110 and connect to the search server system 160. In addition, although a specific number of client systems 120, search server systems 160, video resource servers 132, reading resource servers 142 and networks 110 are shown, embodiments of the present invention contemplate any suitable number of client systems 120, search server systems 160, video resource servers 132, reading resource servers 142 and networks 110. As an example and not a limitation, the search network 100 may include multiple client systems 120, search server systems 160, video resource servers 132, reading resource servers 142 and networks 110.

[0031] Embodiments of the present invention contemplate any suitable network 110. By way of example and not limitation, one or more portions of network 110 may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of a public switched telephone network (PSTN), a cellular telephone network, or a combination of two or more of these. Network 110 may include one or more networks 110.

[0032] The network communication link 150 may connect the client system 120, the search server system 160, and the video resource server 132, the reading resource server 142 to the communication network 110 or to each other. Embodiments of the present invention contemplate any suitable network communication link 150. In a specific implementation, the one or more network communication links 150 include one or more wired lines (e.g., digital subscriber line (DSL) or cable data service interface specification (DOCSIS), wireless (e.g., Wi-Fi or Worldwide Interoperability for Microwave Access (WiMAX)), or optical (e.g., synchronous optical network (SONET) or synchronous digital hierarchy (SDH)) links. In a specific implementation, the one or more network communication links 150 each include an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, a portion of the Internet, a portion of the PSTN, a network based on cellular technology, a network based on satellite communication technology, another network communication link 150, or a combination of two or more such network communication links 150. The network communication links 150 are not necessarily the same throughout the search network architecture 100. One or more first network communication links 150 may be different from one or more second network communication links 150 in one or more aspects.

[0033] In a specific implementation, the search server system 160 can be a network-addressable computing system that can host online searches. The search server system 160 can generate, store, receive, and send search data. The search server system 160 can be accessed directly by other components of the search network architecture 100 or via the network 110. As an example and not a limitation, the client system 120 can use a web browser / search application 122 or a local application associated with the search server system 160 (e.g., a mobile search application, a messaging application, another suitable application, or any combination thereof) to access the search server system 160 directly or via the network 110. In a specific implementation, the search server system 160 may include one or more servers 162. Each server 162 can be a single server or a distributed server across multiple computers or multiple data centers. The server 162 can be of various types, such as, but not limited to, a web server, a news server, a mail server, a message server, an advertising server, a file server, an application server, an exchange server, a database server, a proxy server, another server suitable for performing the functions or processing described herein, or any combination thereof. In a specific implementation, each server 162 may include a hardware, software or embedded logic component or a combination of two or more such components to perform the appropriate functions implemented or supported by the server 162. In a specific implementation, the search server system 160 may include one or more data storages 164. Data storage 164 can be used to store various types of information. In a specific implementation, the information stored in the data storage 164 can be organized according to a specific data structure. In a specific implementation, each data storage 164 can be a relational database, a column database, a related database or other suitable database. Although the embodiment of the present invention describes or shows a specific type of database, the embodiment of the present invention considers any suitable type of database. A specific implementation can provide a page that enables the client system 120, the search server system 160 or the video resource server 132, the reading resource server 142 to manage, retrieve, modify, add or delete the information stored in the data storage 164.

[0034] In one specific implementation, the client system 120 may be an electronic device including hardware, software, or embedded logic components, or a combination of two or more such components, and capable of performing appropriate functions implemented or supported by the client system 120. By way of example and not limitation, the client system 120 may include a computer system such as a desktop computer, a notebook or laptop computer, a netbook, a tablet computer, an e-book reader, a GPS device, a camera, a personal digital assistant (PDA), a handheld electronic device, a cellular phone, a smart phone, other suitable electronic devices, or any suitable combination thereof. Embodiments of the present invention contemplate any suitable client system 120. The client system 120 may enable network users at the client system 120 to access the network 110. The client system 120 may enable its users to communicate with other users at other client systems 120.

[0035] In one specific implementation, the search server system 160 may store multiple reading objects or multiple video objects in one or more data stores 164. In one specific implementation, the search server system 160 may provide users with the ability to take actions on various types of items or objects supported by the search server system 160.

[0036] In a specific implementation, the search server system 160 can link various entities. As an example and not a limitation, the search server system 160 can enable users to interact with each other and receive content from the video resource server 132, the reading resource server 142 or other entities, or allow users to interact with these entities through an application programming interface (API) or other communication channels.

[0037] In a specific implementation, the video resource server 132 and the reading resource server 142 may include one or more types of servers, one or more data stores, one or more interfaces (including but not limited to APIs), one or more network services, one or more content sources, one or more networks, or any other suitable components with which, for example, a server can communicate. The video resource server 132 and the reading resource server 142 may be operated by an entity different from the entity operating the search server system 160. However, in a specific implementation, the search server system 160 and the video resource server 132 and the reading resource server 142 may operate in conjunction with each other to provide search services to users of the search server system 160 or the video resource server 132 and the reading resource server 142. In this sense, the search server system 160 may provide a platform or backbone that other systems such as the video resource server 132 and the reading resource server 142 may use to provide search services and functions to users over the Internet.

[0038] Mobile operating systems such as IOS are limited in the openness of defined system components due to security and other considerations, which has a certain impact on the application development of mobile operating systems. For example, mobile operating systems do not support convenient cross-application search operations. In this case, the user is required to manually switch between the current browsing application and the search application in order to query the content of the current browsing application through the search application. For example, when a user needs help with content in a question-solving application, he or she needs to manually query in the search application, resulting in a poor user experience fluency of the search application. To this end, various embodiments of the present invention can improve the fluency of the user experience.

[0039] Figure 2 The image search method 200 of some embodiments of the present invention is shown. The image search method 200 can be applied to an electronic device installed with a mobile operating system, and a plurality of application programs are installed in the mobile operating system. Specifically, the image search method 200 includes: Step S210: In the first browsing page, a trigger operation for a floating control of the mobile operating system is obtained, and the floating control is suspended on an upper layer of the first browsing page.

[0040] It should be understood that the floating control, as a window floating object, can be a floating button in the IOS operating system. As a flexible and efficient interactive design element, it can be used for quick operation entry or auxiliary navigation functions of the mobile operating system. The floating control can be customized to implement the general desktop function in the mobile operating system, and the floating control does not belong to any desktop application, and the desktop application can be accessed through settings.

[0041] It should also be understood that, among various applications of the mobile operating system, the application of the first browsing page may be a browser application or a learning tool application, etc. The target application may be a search application with an image search function, and the search application may use data including images as input to perform AI intelligent search or intelligent question and answer based on search results to implement the image search function. The image search function includes but is not limited to image recognition, formula recognition, knowledge question and answer, search for text questions or graphic questions, etc.

[0042] Step S220: In response to the triggering operation, a target application is called to take a screenshot of the first browsing page, where the target application is the current desktop application or the second desktop application.

[0043] It should be understood that the floating control can be displayed in the upper layer of the page of any desktop application. For example, a floating button as a floating control can be displayed in the upper layer of the page of an application such as a browser application or a learning tool application to switch to other desktop applications such as a second desktop application or return to the system desktop.

[0044] It should also be understood that when the user triggers an operation in the floating control, the mobile operating system will respond to the operation and execute the screenshot function. For example, the screenshot function is usually implemented through the camera component of the mobile operating system, which can capture all visible content in the current screen. After the screenshot, the camera component can directly save the screenshot result in the camera management component, for example, in the camera screenshot directory in the camera management component.

[0045] Specifically, the user page module of the mobile operating system can monitor the triggering operation of the floating control, set an event listener (for example, a click event listener) in the floating control, and call the function of the camera component when the user triggers the operation. When the camera component captures the current screen content, the screenshot result is directly saved in the camera screenshot directory or other storage space of the camera management component.

[0046] It should also be understood that the camera management component can be set in the general settings of the mobile operating system to allow some desktop applications to access it, or it can be set to prohibit some desktop applications from accessing it.

[0047] Step S230: performing an image search on at least part of the screenshot results, and opening a small window or jumping to a second browsing page to display search results of at least part of the screenshot results.

[0048] It should be understood that the target application can be configured to access the screenshot results in the camera management component. When the camera management component is set to allow the target application to access, the mobile operating system can call the target application to obtain the screenshot results from the camera management component.

[0049] It should also be understood that the target application can be configured to respond to the screenshot page being saved to the camera management component when running in the background, switching from background running to foreground running. Alternatively, the target application can also be configured to respond to the screenshot page being saved to the camera management component to start running.

[0050] It should also be understood that when performing an image search on at least part of the screenshot results, a search term may be generated based on at least part of the screenshot results, and the search term may be used as a search input for the target application. When generating the search term, the screenshot results may use OCR (Optical Character Recognition) technology to convert text or graphics in the screenshot results into searchable text as the search term, or use image recognition technology to directly search for similar images as the search term.

[0051] In some examples, the above-mentioned image search method can be performed by executing a predefined combination instruction through a mobile operating system. For example, the predefined combination instruction is configured to include a call instruction and a search instruction, and then the predefined combination instruction is configured as a development component in the mobile operating system.

[0052] In the scheme of the embodiment of the present invention, in response to the trigger operation of the floating control of the mobile operating system, the target application is called to perform a screenshot of the first browsing page, and the content of the first browsing page can be obtained without manually switching applications. Then, when an image search is performed on at least part of the screenshot results, search results of at least part of the screenshot results are obtained. Through convenient operations, cross-application search between the target application and the application of the first browsing page is implemented, thereby improving the smoothness of user experience.

[0053] Specifically, the call instruction can be configured to respond to the trigger operation of the floating control of the mobile operating system and perform a screenshot of the current display page of the electronic device. The search instruction can be configured to perform an image search of the target application on at least part of the screenshot results, and bring up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results.

[0054] For example, the calling instruction can be associated with the floating control, so that the mobile operating system responds to the triggering operation of the floating control and calls the camera component to perform a window screenshot of the electronic device. The calling instruction can also monitor the state of the camera management component, and while the screenshot result is saved to the camera management component, call the already started second application and encapsulate the screenshot result into the search box input of the target application.

[0055] In some examples, when the second application is running in the background of the mobile operating system, at least part of the screenshot results are packaged into the search box input of the target application, and the second application is switched from the background to the foreground. In other examples, when the second application is not started in the mobile operating system, the screenshot results are directly displayed in the preview page of the camera management component, the second application is started, and at least part of the screenshot results are packaged into the search box input of the target application.

[0056] Without loss of generality, as an example of taking a screenshot of the first browsing page in response to the trigger operation, a calling instruction in a predefined combination instruction may be executed, wherein the calling instruction is specifically configured to call a target application in response to a trigger operation of a floating control of the mobile operating system, wherein when the target application is called, takes a screenshot of the currently displayed page of the electronic device through the camera component of the mobile operating system, and stores the screenshot result in a camera management component.

[0057] Further, as an example of performing an image search on at least part of the screenshot results, the search instruction in the predefined combination instruction can be triggered in response to the call instruction, and the search instruction is specifically configured to use a target application of the mobile operating system to obtain the screenshot results from the camera management component and use the screenshot results as a search item for the target application.

[0058] Specifically, in the relevant embodiments and related examples throughout the present invention, a process component such as NSXPCConnection can be associated with a floating control such as UITapGestureRecognizer and a camera component such as UIScreen to construct a call instruction of the target application, and the call instruction is used to call the camera component to perform a screenshot in response to a trigger operation of the floating control when the target application is running in the background. In addition, a page editing subcomponent such as UIViewController can be associated with a camera component such as UIScreen to construct a search instruction. Then, a predefined combined instruction is obtained by combining the call instruction and the search instruction.

[0059] Furthermore, the search application includes a search term component (eg, a search term generation function) and a search execution component (eg, a search execution function), wherein the search term component can call the search execution component when a search term is obtained.

[0060] That is, when the predefined combination instruction including the call instruction and the search instruction is installed in the mobile operating system, the predefined combination instruction is associated with the search item component, so that the search item component listens to the trigger operation of the floating control through the process component, responds to the trigger operation, and calls the screenshot subcomponent (e.g., snapshotView) in the camera component to perform the screenshot. Then the search item component is configured to be associated with the camera component, for responding to the screenshot result executed by the screenshot subcomponent, calling the page editing subcomponent (e.g., present subcomponent) in the camera component to display the screenshot result, and calling the editing subcomponent such as the editing page controller (ScreenshotEditViewController) in the camera component to obtain a content screenshot of at least part of the screenshot result indicated by the user's screenshot operation, and passing the content screenshot to the search item component as a search item. In some examples, the editing subcomponent can be implemented as an interactive button for confirming the completion of the edit, for the user's screenshot operation.

[0061] It should be understood that the search item component can be a pre-configured acquisition logic of the target application, and can obtain a content screenshot of at least part of the screenshot result by associating with the camera component, which is compatible with the pre-configured acquisition logic and improves development efficiency. It should also be understood that the search item component can be associated with the search execution component by referencing the identifier of the pre-configured search execution component so as to call the search execution component, thereby being compatible with the development logic of the search execution component in the target application and improving development efficiency.

[0062] In some embodiments, as some examples of taking a screenshot of the first browsing page in response to a trigger operation, the mobile operating system may be called to take a screenshot of the first browsing page in response to the trigger operation, and then the screenshot result may be saved in the camera management component of the mobile operating system. Specifically, the trigger operation may be obtained through a floating control such as UITapGestureRecognizer, and in response to the trigger operation, a screenshot subcomponent (e.g., snapshotView) in the camera component may be called to take a screenshot.

[0063] In other words, it is compatible with the camera management function of the mobile operating system and saves the screenshot results to the camera management component of the mobile operating system. There is no need to configure storage space for the screenshot results separately, which is conducive to obtaining the screenshot results from the camera management component.

[0064] Accordingly, as some examples of performing an image search on at least part of the screenshot result, the screenshot result can be obtained from the camera management component through the target application, and then, a search term of the target application is generated according to at least part of the screenshot result, and then the target application is called to perform an image search on the search term. Specifically, the screenshot result can be presented through a page editing subcomponent such as a present subcomponent, and an editing subcomponent such as an edit page controller (e.g., ScreenshotEditViewController) can be called to obtain a content screenshot of at least part of the screenshot result, and the content screenshot is passed to the search term component as the search term.

[0065] That is to say, the camera management component is a component of the mobile operating system, and each application can be easily configured to have access rights to the camera management component, and the screenshot results can be obtained from the camera management component through the target application, without adding too much additional software configuration to the target application. In addition, the search item of the target application is generated based on at least part of the screenshot results, which can be compatible with the image input function of the target application to perform image search for the search item, and also reduces the additional software configuration of the target application.

[0066] In other embodiments, Figure 3A Some example image search methods 300 are shown, including: Step S310: in the first browsing page, a trigger operation for a floating control of the mobile operating system is obtained, and the floating control is suspended on an upper layer of the first browsing page.

[0067] Step S320: In response to the triggering operation, the mobile operating system is called to take a screenshot of the first browsing page, so as to save the screenshot result into the camera management component of the mobile operating system.

[0068] Furthermore, the user can perform manual selection (e.g., manual selection) based on the screenshot results for the first browsing page to select a content screenshot indicating at least part of the screenshot results. In this case, the screenshot result page needs to be displayed to the user, and then a search term is generated based on the user's operation.

[0069] Specifically, the image search method 300 further includes: Step S330: Obtain and display the screenshot result from the camera management component through the target application.

[0070] Step S340: Acquire a screenshot operation for the screenshot result to obtain a content screenshot including at least part of the content.

[0071] Step S350: In response to the confirmation operation on the content screenshot, a search item for the target application is generated so as to call the target application to perform an image search for the search item.

[0072] That is, displaying the screenshot results in the preview page of the camera management component and obtaining the screenshot operation for the screenshot results are compatible with the functions of the camera management component, and also allow users to more flexibly select the content to be searched through the screenshot operation. In addition, generating the search item of the target application in response to the confirmation operation of the content screenshot actually reduces the additional software configuration of the target application by associating the confirmation operation of the camera management component with the search input operation of the target application.

[0073] Specifically, the image search method 300 further includes: Step S360: Display the search results of the content screenshots in the search browsing page of the target application.

[0074] Specifically, Figure 3B As shown, the page marked with X is the first browsing page, the page marked with Y is the page of the target application, and the page including the dotted selection box is the screenshot result page. It can be seen that as an example of calling up a small window or jumping to the second browsing page to display the search results of at least part of the screenshot results, in response to the confirmation operation of the content screenshot, the preview page can be switched to the search browsing page of the target application, and then the search results of the content screenshot can be displayed in the search browsing page.

[0075] That is to say, you can switch from the preview page of the camera management component to the search browsing page, which can display the search results to the user more comprehensively. For example, the user can further obtain extended content such as knowledge points or title information related to the content indicated by the content screenshot in the search browsing page (for example, by clicking an extended link).

[0076] In other embodiments, Figure 4A Some other example image search methods 400 are shown, including: Step S410: in the first browsing page, a triggering operation for a floating control of a mobile operating system is obtained, and the floating control is suspended on an upper layer of the first browsing page.

[0077] Step S420: In response to the triggering operation, the mobile operating system is called to take a screenshot of the first browsing page, so as to save the screenshot result into the camera management component of the mobile operating system.

[0078] Furthermore, the user can perform manual selection (e.g., manual selection) based on the screenshot results for the first browsing page to select a content screenshot indicating at least part of the screenshot results. In this case, the screenshot result page needs to be displayed to the user, and then a search term is generated based on the user's operation.

[0079] Specifically, the image search method 400 further includes: Step S430: Obtain and display the screenshot result from the camera management component through the target application.

[0080] Step S440: Acquire a screenshot operation for the screenshot result to obtain a content screenshot including at least part of the content.

[0081] Step S450: In response to the confirmation operation on the content screenshot, a search item for the target application is generated so as to call the target application to perform an image search for the search item.

[0082] Specifically, the image search method 400 further includes: Step S460: Display the search result of the content screenshot as a floating small window of the mobile operating system on the upper layer of the first browsing page.

[0083] Specifically, Figure 4B As shown, the page marked with X is the first browsing page, the page marked with Y is the page of the target application, and the page including the dotted selection box is the screenshot result page. It can be seen that as an example of calling up a small window or jumping to the second browsing page to display at least part of the search results of the screenshot results, in response to the confirmation operation of the content screenshot, the preview page can be switched to the first browsing page, and then the search results of the content screenshot can be displayed as a floating small window of the mobile operating system on the upper layer of the first browsing page.

[0084] That is, you can switch from the preview page of the camera management component to the first browsing page, and display the search results of the content screenshots on the upper layer of the first browsing page, which can ensure the user's experience consistency of the application on the first browsing page. In addition, by using the search results of the content screenshots as a floating window of the mobile operating system, you only need to associate the target application with the floating window of the mobile operating system, which reduces the additional software configuration of the target application.

[0085] Alternatively, in some other embodiments, the screenshot result may be subjected to content recognition, and a content screenshot indicating the screenshot result may be automatically generated. Figure 5A Some other example image search methods 500 are shown, including: Step S510: in the first browsing page, a trigger operation for a floating control of the mobile operating system is obtained, and the floating control is suspended on an upper layer of the first browsing page.

[0086] Step S520: In response to the triggering operation, the mobile operating system is called to take a screenshot of the first browsing page, so as to save the screenshot result into the camera management component of the mobile operating system.

[0087] Step S530: Obtain the screenshot result from the camera management component through the target application, and identify the content elements of the screenshot result.

[0088] That is, when generating the search item of the target application according to at least part of the screenshot results, the screenshot results can be identified as content elements, and at least part of the screenshot results including the content elements can be determined as the search item of the target application. In other words, at least part of the screenshot results can be obtained by performing content element identification on the screenshot results, without the user having to manually select in the screenshot results, which improves the efficiency of generating the search items and greatly improves the overall fluency of the user experience.

[0089] Specifically, the image search method 500 further includes: Step S540: Determine at least part of the screenshot result including the content element as a search item of the target application, so as to call the target application to perform an image search for the search item.

[0090] Step S550: Displaying search results of the content screenshots in the search browsing page of the target application.

[0091] Specifically, Figure 5BAs shown, the page marked with X is the first browsing page, the page marked with Y is the page of the target application, and the page including the dotted selection box is the screenshot result page. It can be seen that as an example of calling up a small window or jumping to the second browsing page to display the search results of at least part of the screenshot results, it is possible to switch from the first browsing page to the search browsing page of the target application, and then display the search results of at least part of the screenshot results in the search browsing page. In other words, switching from the first browsing page to the search browsing page can display the search results more comprehensively to the user. For example, the user can further obtain extended content such as knowledge points or title information related to the content indicated by the content screenshot in the search browsing page (for example, by clicking on the extension link).

[0092] In other embodiments, Fig. 6A The image search method 600 of some other examples is shown, including: Step S610: In the first browsing page, a trigger operation for a floating control of a mobile operating system is obtained, and the floating control is suspended on an upper layer of the first browsing page.

[0093] Step S620: In response to the triggering operation, the mobile operating system is called to take a screenshot of the first browsing page, so as to save the screenshot result into the camera management component of the mobile operating system.

[0094] Step S630: Obtain the screenshot result from the camera management component through the target application, and identify the content elements of the screenshot result.

[0095] That is, when generating the search item of the target application according to at least part of the screenshot results, the screenshot results can be identified as content elements, and at least part of the screenshot results including the content elements can be determined as the search item of the target application. In other words, at least part of the screenshot results can be obtained by performing content element identification on the screenshot results, without the user having to manually select in the screenshot results, which improves the efficiency of generating the search items and greatly improves the overall fluency of the user experience.

[0096] Specifically, the image search method 600 further includes: Step S640: Determine at least part of the screenshot result including the content element as a search item of the target application, so as to call the target application to perform an image search for the search item.

[0097] Step S650: Display the search result of the content screenshot as a floating small window of the mobile operating system on the upper layer of the first browsing page.

[0098] Specifically, Figure 6BAs shown, the page marked with X is the first browsing page, the page marked with Y is the page of the target application, and the page including the dotted selection box is the screenshot result page. It can be seen that as an example of calling up a small window or jumping to the second browsing page to display the search results of at least part of the screenshot results, the search results of at least part of the screenshot results can be displayed as a floating small window of the mobile operating system on the upper layer of the first browsing page.

[0099] That is, displaying the search results of the content screenshot on the upper layer of the first browsing page can ensure the user's experience consistency of the application on the first browsing page. In addition, using the search results of the content screenshot as a floating window of the mobile operating system only requires associating the target application with the floating window of the mobile operating system, reducing the additional software configuration of the target application.

[0100] The following will be combined Figure 7 Other operating system configuration methods of the present invention are described in detail. Figure 7 The operating system configuration method 700 is applied to an electronic device installed with a mobile operating system, and the operating system configuration method 700 includes: Step S710: Obtain a predefined combination instruction, the predefined combination instruction includes a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction is configured to respond to the trigger operation of the floating control of the mobile operating system, perform a screenshot of the current display page of the electronic device, and the search instruction is configured to perform an image search on at least part of the screenshot results, and call up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results.

[0101] Step S720: configuring the combined instruction into the mobile operating system.

[0102] In the scheme of the embodiment of the present invention, the calling instruction is configured to respond to the triggering operation of the floating control of the mobile operating system, perform a screenshot of the current display page of the electronic device, and the search instruction is configured to perform an image search on at least part of the screenshot results, and call up a small window or jump to the second browsing page to display the search results of at least part of the screenshot results. Therefore, by configuring the predefined combination instruction into the mobile operating system, cross-application search between the target application and the application on the first browsing page is achieved through convenient operation, thereby improving the smoothness of user experience.

[0103] In some embodiments, the calling instruction is specifically configured to call a target application in response to a triggering operation of a floating control of the mobile operating system, wherein when the target application is called, it takes a screenshot of the current display page of the electronic device through a camera component of the mobile operating system and stores the screenshot result in a camera management component.

[0104] In some other embodiments, the search instruction is specifically configured to use a target application of the mobile operating system to obtain the screenshot result from the camera management component, and use the screenshot result as a search item for the target application.

[0105] Specifically, in the relevant embodiments and related examples throughout the present invention, a process component such as NSXPCConnection can be associated with a floating control such as UITapGestureRecognizer and a camera component such as UIScreen to construct a call instruction of the target application, and the call instruction is used to call the camera component to perform a screenshot in response to a trigger operation of the floating control when the target application is running in the background. In addition, a page editing subcomponent such as UIViewController can be associated with a camera component such as UIScreen to construct a search instruction. Then, a predefined combined instruction is obtained by combining the call instruction and the search instruction.

[0106] Furthermore, the search application includes a search term component (eg, a search term generation function) and a search execution component (eg, a search execution function), wherein the search term component can call the search execution component when a search term is obtained.

[0107] That is, when the predefined combination instruction including the call instruction and the search instruction is installed in the mobile operating system, the predefined combination instruction is associated with the search item component, so that the search item component listens to the trigger operation of the floating control through the process component, responds to the trigger operation, and calls the screenshot subcomponent (e.g., snapshotView) in the camera component to perform the screenshot. Then the search item component is configured to be associated with the camera component, for responding to the screenshot result executed by the screenshot subcomponent, calling the page editing subcomponent (e.g., present subcomponent) in the camera component to display the screenshot result, and calling the editing subcomponent such as the editing page controller (ScreenshotEditViewController) in the camera component to obtain a content screenshot of at least part of the screenshot result indicated by the user's screenshot operation, and passing the content screenshot to the search item component as a search item. In some examples, the editing subcomponent can be implemented as an interactive button for confirming the completion of the edit, for the user's screenshot operation.

[0108] It should be understood that the search item component can be a pre-configured acquisition logic of the target application, and can obtain a content screenshot of at least part of the screenshot result by associating with the camera component, which is compatible with the pre-configured acquisition logic and improves development efficiency. It should also be understood that the search item component can be associated with the search execution component by referencing the identifier of the pre-configured search execution component so as to call the search execution component, thereby being compatible with the development logic of the search execution component in the target application and improving development efficiency.

[0109] Figure 8 Electronic devices according to further embodiments of the present invention are shown. Figure 8 The electronic device 800 is installed with a mobile operating system, including: The acquisition module 810 acquires, in a first browsing page, a triggering operation for a floating control of the mobile operating system, wherein the floating control is suspended on an upper layer of the first browsing page.

[0110] The calling module 820, in response to the triggering operation, performs a screenshot of the first browsing page.

[0111] The search module 830 performs an image search on at least part of the screenshot results, and opens a small window or jumps to a second browsing page to display search results of at least part of the screenshot results.

[0112] In the scheme of the embodiment of the present invention, in response to the trigger operation of the floating control of the mobile operating system, the target application is called to perform a screenshot of the first browsing page, and the content of the first browsing page can be obtained without manually switching applications. Then, when an image search of the target application is performed on at least part of the screenshot results, search results of at least part of the screenshot results are obtained. Cross-application search between the target application and the application of the first browsing page is implemented through convenient operations, thereby improving the smoothness of user experience.

[0113] In some other embodiments, the calling module is specifically used to: respond to the trigger operation, call the mobile operating system to perform a screenshot of the first browsing page; and save the screenshot result to a camera management component of the mobile operating system.

[0114] In other embodiments, the search module is specifically used to: obtain the screenshot result from the camera management component through a target application; generate a search item for the target application based on at least part of the screenshot result; and call the target application to perform an image search for the search item.

[0115] In other embodiments, the search module is specifically used to: display the screenshot result in the preview page of the camera management component; obtain a screenshot operation for the screenshot result to obtain a content screenshot including at least part of the content; and generate a search item for the target application in response to a confirmation operation on the content screenshot.

[0116] In other embodiments, the search module is specifically used to: in response to a confirmation operation on the content screenshot, switch from the preview page to a search browsing page of the target application; and display search results of the content screenshot on the search browsing page.

[0117] In other embodiments, the search module is specifically used to: respond to a confirmation operation on the content screenshot, switch from the preview page to the first browsing page; display the search results of the content screenshot as a floating window of the mobile operating system on the upper layer of the first browsing page.

[0118] In some other embodiments, the search module is specifically used to: identify content elements of the screenshot results; and determine at least part of the screenshot results including content elements as search items of the target application.

[0119] In some other embodiments, the search module is specifically used to: switch from the first browsing page to the search browsing page of the target application; and display the search results of the at least partial screenshot results in the search browsing page.

[0120] In some other embodiments, the search module is specifically used to: display the search results of the at least partial screenshot results as a floating small window of the mobile operating system on the upper layer of the first browsing page.

[0121] Further, Fig. 9 Electronic devices according to further embodiments of the present invention are shown. Fig. 9 The electronic device 900 is installed with a mobile operating system, including: Acquisition module 910 acquires predefined combination instructions, the predefined combination instructions including a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction is configured to respond to the triggering operation of the floating control of the mobile operating system, perform a screenshot of the current display page of the electronic device, and the search instruction is configured to perform an image search on at least part of the screenshot results, and call up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results.

[0122] The configuration module 920 configures the combined instruction into the mobile operating system.

[0123] In the scheme of the embodiment of the present invention, the calling instruction is configured to respond to the triggering operation of the floating control of the mobile operating system, perform a screenshot of the current display page of the electronic device, and the search instruction is configured to perform an image search on at least part of the screenshot results, and call up a small window or jump to the second browsing page to display the search results of at least part of the screenshot results. Therefore, by configuring the predefined combination instruction into the mobile operating system, cross-application search between the target application and the application on the first browsing page is achieved through convenient operation, thereby improving the smoothness of user experience.

[0124] In some embodiments, the calling instruction is specifically configured to call a target application in response to a triggering operation of a floating control of the mobile operating system, wherein when the target application is called, it takes a screenshot of the current display page of the electronic device through a camera component of the mobile operating system and stores the screenshot result in a camera management component.

[0125] In some other embodiments, the search instruction is specifically configured to use a target application of the mobile operating system to obtain the screenshot result from the camera management component, and use the screenshot result as a search item for the target application.

[0126] It should be understood that the specific implementation of each module in the electronic device can refer to the corresponding description of the corresponding steps in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above electronic device and module can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0127] Reference Fig.10 , shows a schematic structural diagram of an electronic device according to another embodiment of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0128] like Fig.10 As shown, the electronic device may include: a processor (processor) 1002 for executing a program 1010 , a communication interface (Communications Interface) 1004 , a memory (memory) 1006 , and a communication bus 1008 .

[0129] The processor, the communication interface, and the memory communicate with each other via a communication bus.

[0130] Communication interface, used to communicate with other electronic devices or servers.

[0131] The processor is used to execute the program, and specifically can execute the relevant steps in the above method embodiment.

[0132] Specifically, the program may include program codes including computer operation instructions.

[0133] The processor may be a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0134] Memory is used to store programs. The memory may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk storage.

[0135] The program may include multiple computer instructions. Specifically, the program may enable the processor to execute the image search method or operating system configuration method described in any of the aforementioned method embodiments through the multiple computer instructions.

[0136] The specific implementation of each step in the program can refer to the corresponding description of the corresponding steps, modules or units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described device, equipment or module can refer to the corresponding process description in the above method embodiments, which will not be repeated here.

[0137] The embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned multiple method embodiments is implemented. The computer storage medium includes, but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk or a magneto-optical disk, etc.

[0138] An embodiment of the present invention further provides a computer program product, including computer instructions, which instruct a computing device to execute the image search method or operating system configuration method in the above-mentioned multiple method embodiments.

[0139] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0140] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0141] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0142] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.

[0143] The above implementation methods are only used to illustrate the embodiments of the present invention, and are not limitations of the embodiments of the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A method for searching an image, applied to an electronic device equipped with a mobile operating system, the method comprising: In a first browsing page, a triggering operation for a floating control of the mobile operating system is obtained, wherein the floating control is suspended on an upper layer of the first browsing page; In response to the trigger operation, calling a target application to take a screenshot of the first browsing page, the target application being a current desktop application or a second desktop application; Perform an image search on at least part of the screenshot results, and bring up a small window or jump to a second browsing page to display the search results of at least part of the screenshot results.

2. The method according to claim 1, wherein: In response to the trigger operation, calling the target application to take a screenshot of the first browsing page includes: In response to the trigger operation, calling a target application, wherein when the target application is called, taking a screenshot of the first browsing page through the mobile operating system; The screenshot result is saved in the camera management component of the mobile operating system.

3. The method according to claim 2, wherein: Perform an image search on at least some of the screenshot results, including: Obtaining the screenshot result from the camera management component through a target application; generating a search term for the target application based on at least part of the screenshot results; The target application is called to perform an image search for the search item.

4. The method according to claim 3, wherein: Generating a search term for the target application according to at least part of the screenshot results includes: Displaying the screenshot result in a preview page of the camera management component; Acquire a screenshot operation for the screenshot result to obtain a content screenshot including at least part of the content; In response to a confirmation operation on the content screenshot, a search item for the target application is generated.

5. The method according to claim 4, wherein: Bringing up a small window or jumping to a second browsing page to display the search results of at least part of the screenshot result includes: In response to a confirmation operation on the content screenshot, switching from the preview page to a search and browse page of the target application; In the search browsing page, the search results of the content screenshot are displayed.

6. The method according to claim 4, wherein: Bringing up a small window or jumping to a second browsing page to display the search results of at least part of the screenshot result includes: In response to a confirmation operation on the content screenshot, switching from the preview page to the first browsing page; The search result of the content screenshot is displayed as a floating small window of the mobile operating system on the upper layer of the first browsing page.

7. The method according to claim 3, wherein: Generating a search term for the target application according to at least part of the screenshot results includes: Performing content element identification on the screenshot result; At least a portion of the screenshot result including the content element is determined as a search term for the target application.

8. The method according to claim 3, wherein: Bringing up a small window or jumping to a second browsing page to display the search results of at least part of the screenshot result includes: Switching from the first browsing page to the search browsing page of the target application; In the search browsing page, search results of at least a portion of the screenshot results are displayed.

9. The method according to claim 3, wherein: Bringing up a small window or jumping to a second browsing page to display the search results of at least part of the screenshot result includes: The search result of at least part of the screenshot result is displayed as a floating small window of the mobile operating system on the upper layer of the first browsing page.

10. The method according to claim 1, wherein: In response to the trigger operation, calling the target application to take a screenshot of the first browsing page includes: Execute a calling instruction in a predefined combination instruction, wherein the calling instruction is specifically configured to: respond to a triggering operation of a floating control of the mobile operating system, call a target application, wherein when the target application is called, takes a screenshot of a current display page of the electronic device through a camera component of the mobile operating system, and stores the screenshot result in a camera management component.

11. The method according to claim 10, wherein: Perform an image search on at least some of the screenshot results, including: In response to the calling instruction, a search instruction in the predefined combination instruction is triggered, and the search instruction is specifically configured to use a target application of the mobile operating system to obtain the screenshot result from the camera management component, and use the screenshot result as a search item for the target application.

12. An operating system configuration method, applied to an electronic device installed with a mobile operating system, comprising: Obtaining a predefined combination instruction, the predefined combination instruction comprising a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction is configured to respond to a triggering operation of a floating control of the mobile operating system, call a target application to perform a screenshot of a current display page of the electronic device, and the search instruction is configured to perform an image search on at least part of the screenshot results, and call up a small window or jump to a second browsing page to display search results of at least part of the screenshot results; The combined instruction is configured into the mobile operating system.

13. The method according to claim 12, wherein: The calling instruction is specifically configured to respond to the triggering operation of the floating control of the mobile operating system, call the camera component of the mobile operating system through the target application to take a screenshot of the current display page of the electronic device, and store the screenshot result in the camera management component.

14. The method according to claim 13, wherein: The search instruction is specifically configured to use a target application of the mobile operating system to obtain the screenshot result from the camera management component, and use the screenshot result as a search item of the target application.

15. An electronic device, equipped with a mobile operating system, comprising: An acquisition module, in a first browsing page, acquires a triggering operation for a floating control of the mobile operating system, the floating control being suspended on an upper layer of the first browsing page; A calling module, in response to the triggering operation, calls a target application to take a screenshot of the first browsing page, the target application being a current desktop application or a second desktop application; The search module performs an image search on at least part of the screenshot results, and opens a small window or jumps to a second browsing page to display search results of at least part of the screenshot results.

16. An electronic device, equipped with a mobile operating system, comprising: an acquisition module, acquiring a predefined combination instruction, the predefined combination instruction comprising a call instruction and a search instruction, wherein the call instruction is used to trigger the search instruction, the call instruction is configured to respond to a trigger operation of a floating control of the mobile operating system, call a target application to perform a screenshot of a current display page of the electronic device, the target application is a current desktop application or a second desktop application, wherein the search instruction is configured to perform an image search on at least part of the screenshot results, and bring up a small window or jump to a second browsing page to display search results of at least part of the screenshot results; A configuration module configures the combined instruction into the mobile operating system.

17. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the method according to any one of claims 1-14.

18. A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 14.

19. A computer program product, comprising computer instructions, which implement the method according to any one of claims 1 to 14 when executed by a processor.

Images