Information recommendation method and electronic equipment

Abstract

The invention provides an information recommendation method and electronic equipment. The method comprises the following steps: a display interface of the electronic equipment contains a replicable text; in response to copying operation executed by the user on the display interface, the electronic equipment recommends an application program for the user; and when the user selects a certain application program, the electronic equipment displays an interface of the application program, and the text copied by the user is displayed at a corresponding information editing position in the interface. Therefore, according to the method, service recommendation based on the text copied by the user is realized, and the use experience of the user is improved.

Description

[0001] This application is a divisional application. The name of the original application is information recommendation method and electronic device. The application number of the original application is 202111123937.4. The original application date is September 24, 2021. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal devices, and in particular to an information recommendation method and electronic device. Background Art

[0003] In daily life, users can find information of interest everywhere. For example, the information of interest can exist in electronic images, paper documents, etc. However, users cannot use this information of interest conveniently. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides an information recommendation method and electronic device. In this method, the electronic device can identify any form of user information of interest and recommend services matching the information of interest to the user, thereby simplifying user operations, allowing the user to conveniently use the information of interest, and improving the user experience.

[0005] In the first aspect, the embodiment of the present application provides an information recommendation method. The method includes: the electronic device displays a first interface in response to a received first operation; wherein the target image and the first icon are displayed in the first interface, and the first icon is used to indicate that the key information of a preset type is recognized in the target image; the first operation includes: a screenshot operation, and an operation of viewing an image in a gallery; the electronic device displays a second interface in response to a second operation on the first icon; wherein the target image and the annotation of the key information are displayed in the second interface; the electronic device displays a third interface in response to a third operation on one of the annotations; wherein the identification of one or more applications is displayed in the third interface, and the application is recommended according to the information type of the key information corresponding to one of the annotations; the electronic device displays a fourth interface in response to a fourth operation on the identification of one of the applications; wherein the display interface of one of the applications is displayed in the fourth interface, and the content of the display interface is related to the key information corresponding to one of the annotations. In this way, after the electronic device identifies the user's information of interest in the image, it annotates the user's information of interest in the image, and recommends the application to the user based on the annotation selected by the user. When the user selects an application, the electronic device displays the interface of the application, and the content of the interface is related to the user's information of interest. This enables service recommendations based on information of interest to users in images, improving the user experience.

[0006] Exemplarily, the second operation may be a click operation.

[0007] Exemplarily, the third operation may be a click operation.

[0008] Exemplarily, the first icon may be referred to as an entity recognition result viewing icon. When the user clicks the first icon, the electronic device displays the recognition result of the icon, that is, displays the annotation of the key information of the preset type in the icon.

[0009] Exemplarily, the key information of the preset type may be address information, code information, ID number information, mobile phone number, English text, express order number, website, email address, network disk download link, password (such as Taobao password, Douyin password) information, etc. For example, the address information may be a detailed street address, such as No. XX, XX Street, XX District, XX City, or a scenic spot name (such as the name of a famous cultural or natural attraction), or a famous building name, or an address with corresponding information of interest on a map, and so on.

[0010] According to the first aspect, the electronic device displays a first interface in response to a received first operation, including: the electronic device identifies a target image corresponding to the first operation in response to the received first operation; when the identification is completed and there is key information of a preset type in the target image, the first interface is displayed.

[0011] Exemplarily, when the first operation is a screenshot operation, the target image corresponding to the first operation is the screenshot image; when the first operation is an operation of viewing an image in a gallery, the target image corresponding to the first operation is the image viewed in the gallery.

[0012] According to the first aspect, or any implementation of the first aspect above, the electronic device recognizes the target image corresponding to the first operation in response to the received first operation, including: the electronic device recognizes the target image in response to the received first operation when the target image meets the real-time recognition condition. In this way, before recognizing the target image, the electronic device determines whether the real-time recognition condition is met, and only when the target image meets the real-time recognition condition does it perform the recognition operation on the target image, thereby avoiding unnecessary image recognition operations and reducing the power consumption of the electronic device.

[0013] According to the first aspect, or any implementation of the first aspect above, when the first operation is a screenshot operation, the target image satisfies the real-time recognition condition, including: when the application displayed on the interface meets the first recognition condition when the screenshot operation is performed, and the user's screenshot sharing habit meets the second recognition condition, the target image meets the real-time recognition condition; wherein the first recognition condition is used to indicate the existence of an application requiring recognition, and the second recognition condition is used to indicate the user's operating habits that meet the real-time recognition. In the application scenario of screenshots, the electronic device determines whether the target image meets the real-time recognition condition, taking into account two factors, one is the application corresponding to the screenshot operation, and the other is the user's operating habits. Only when both of these aspects meet the recognition conditions, it is determined that the target image meets the real-time recognition condition, thereby ensuring the accuracy of the result of the electronic device determining whether the target image meets the real-time recognition condition.

[0014] Exemplarily, the first recognition condition may be that the application belongs to a preset application set. That is, each application in the preset application set is an application with entity recognition requirements. For example, the preset application set may include applications that generally involve entity recognition, such as WPS applications, PPT applications, etc., and the preset application set may not include music applications, taxi applications, and other applications that do not involve entity recognition.

[0015] Exemplarily, the second recognition condition may be a preset set of user operation habits. That is, each user operation habit in the preset set of user operation habits is a user operation habit that meets the requirements for real-time entity recognition. Exemplarily, the preset set of user operation habits may include a user screenshot sharing habit of opening the screenshot before sharing. The preset set of user operation habits does not include direct sharing without opening the screenshot. If the user screenshot sharing habit is direct sharing without opening the screenshot (such as swiping up to share directly after taking the screenshot), the user screenshot sharing habit does not meet the second recognition condition.

[0016] According to the first aspect, or any implementation of the first aspect above, when the first operation is an operation of viewing an image in a gallery, the target image satisfies the real-time recognition condition, including: when the target image is a camera-shot image, when the shooting attribute of the target image satisfies the third recognition condition, the target image satisfies the real-time recognition condition; when the target image is a screenshot image, when the application displayed on the interface when the screenshot operation is performed satisfies the first recognition condition, the target image satisfies the real-time recognition condition; wherein the first recognition condition is used to indicate that there is an application with recognition requirements, and the third recognition condition is used to indicate that there is a camera shooting mode with image recognition requirements. In the application scenario of viewing images in a gallery, the electronic device determines whether the target image meets the real-time recognition condition, taking into account the source of the image, whether it is from a screenshot or from shooting. For images from different sources, different recognition conditions are used to determine whether the target image meets the real-time recognition condition, thereby ensuring the accuracy of the result of the electronic device determining whether the target image meets the real-time recognition condition.

[0017] Exemplarily, the first recognition condition may be that the application belongs to a preset application set. That is, each application in the preset application set is an application with entity recognition requirements. For example, the preset application set may include applications that generally involve entity recognition, such as WPS applications, PPT applications, etc., and the preset application set may not include music applications, taxi applications, and other applications that do not involve entity recognition.

[0018] Exemplarily, the third recognition condition may be that the camera shooting classification result belongs to a preset classification set (or the camera shooting mode belongs to a preset mode set). Among them, each camera shooting classification in the preset classification set is a camera shooting classification for which entity recognition needs exist. For example, the preset classification set may include classifications that generally involve entity recognition, such as documents, etc. The preset classification set does not include classifications that do not involve entity recognition, such as landscapes and portraits. Both the camera shooting classification result and the camera shooting mode may include classification tags (or mode tags) for further classification or identification. The same camera shooting mode may include multiple mode tags, some of which indicate that there is a need for entity recognition, and some of which indicate that there is no need for entity recognition.

[0019] According to the first aspect, or any implementation of the first aspect above, the electronic device displays a first camera shooting interface before displaying the first interface in response to the received first operation; in response to the received shooting operation, the target image obtained by shooting is stored in a gallery, and the target image is identified. When the electronic device completes the identification and there is a preset type of key information in the target image, the electronic device displays the first interface in response to the received first operation, including: the electronic device displays the first interface in response to the operation of viewing the target image in the gallery. In this way, in the application scenario of shooting, the electronic device will also identify the information of interest to the user for the image obtained by shooting. Therefore, after the user takes a photo of the information of interest, the electronic device can identify the information of interest to the user and recommend services matching the information of interest to the user, thereby simplifying the user operation, allowing the user to conveniently use the information of interest, and improving the user experience.

[0020] According to the first aspect, or any implementation of the first aspect above, the electronic device recognizes the target image, including: the electronic device recognizes the target image when the shooting attribute of the target image meets the third recognition condition; wherein the third recognition condition is used to indicate a camera shooting mode that requires image recognition. In this way, before the electronic device recognizes the image acquired through the user's camera operation, it first determines whether the image meets the recognition condition. Only when the image meets the corresponding recognition condition will the electronic device recognize the image, thereby avoiding unnecessary image recognition operations and reducing the power consumption of the electronic device.

[0021] According to the first aspect, or any implementation of the first aspect above, after the electronic device recognizes the target image, it also includes: when the recognition is completed and there is a preset type of key information in the target image, the electronic device displays a second camera shooting interface; wherein the second camera shooting interface also displays a second icon, and the second icon is used to indicate that the target image has been recognized and the preset type of key information is recognized in the target image. In this way, in the application scenario of shooting, the image obtained by the user's previous photo operation is recognized, and there is key information in the image, the electronic device will display an icon in the camera shooting interface to prompt the user that the image it has taken has been recognized and there is key information in the image, which can be viewed by the user. At this time, the user can view the recognition result of the image. The display of the second icon can avoid the user's sense of loss when viewing the image recognition result too early when the image is not recognized, thereby improving the user experience.

[0022] Exemplarily, the second icon may be referred to as a picture entity recognition completion mark, which is used to prompt the user that the image captured at the last moment has been recognized and that key information of a preset type has been recognized in the image.

[0023] According to the first aspect, or any implementation of the first aspect above, the electronic device responds to the received charging operation, and if there are unrecognized images in the gallery, the unrecognized images are recognized in sequence; the electronic device responds to the received charging stop operation, and if there are unrecognized images in the gallery, the operation of recognizing the unrecognized images is stopped. In this way, in the application scenario of electronic device charging, the electronic device recognizes the unrecognized images in the gallery in sequence, avoiding the power consumption generated by the recognition of the image to affect the normal use of the electronic device by the user. In particular, in the scenario of electronic device system upgrade, there will be a large number of unrecognized images in the gallery. Compared with recognizing these images when the user uses the electronic device normally, recognizing these images when the electronic device is charging can avoid affecting the normal use of the electronic device by the user.

[0024] According to the first aspect, or any implementation of the first aspect above, when an application logo is displayed in the third interface, the application logo is displayed in the form of a floating ball; when multiple application logos are displayed in the third interface, the multiple application logos are displayed in the form of a list; the list also displays the content of key information corresponding to one of the annotations. In this way, when the electronic device recommends applications to users, it displays the application logos in different forms depending on the number of applications. At the same time, when the electronic device displays the application logos in the form of a list, it also displays the key information content, which can facilitate users to confirm whether the information on which the recommended application is based is the information of interest to the user, thereby ensuring the accuracy of the recommendation service.

[0025] According to the first aspect, or any implementation of the first aspect above, the method can be applied to a folding screen mobile phone, the folding screen is in an unfolded state, including a first display area and a second display area. The electronic device displays a first interface in response to the received first operation, including: the electronic device displays the first interface in the first display area in response to the received first operation; the electronic device displays a second interface in response to the second operation on the first icon, including: the electronic device displays the second interface in the first display area in response to the second operation on the first icon; the electronic device displays a third interface in response to a third operation on one of the annotations, including: the electronic device displays the third interface in the first display area in response to the third operation on one of the annotations; the electronic device displays a fourth interface in response to a fourth operation on the identification of one of the applications, including: the electronic device displays the fourth interface in the second display area in response to the fourth operation on the identification of one of the applications; or, the electronic device displays the fourth interface in a half-screen card window on the first display area in response to the fourth operation on the identification of one of the applications. In this way, in the scenario where the electronic device is a folding screen mobile phone, the electronic device can display the display interface of the recommended service in different forms, thereby improving the user experience.

[0026] According to the first aspect, or any implementation of the first aspect above, the electronic device displays a first interface in response to a received first operation, including: the electronic device displays the first interface in a first floating window in response to the received first operation; the electronic device displays a second interface in response to a second operation on a first icon, including: the electronic device displays the second interface in the first floating window in response to the second operation on the first icon; the electronic device displays a third interface in response to a third operation on one of the annotations, including: the electronic device displays the third interface in the first floating window in response to the third operation on one of the annotations; the electronic device displays a fourth interface in response to a fourth operation on the identification of one of the applications, including: the electronic device displays the fourth interface in the second floating window in response to the fourth operation on the identification of one of the applications. In this way, the electronic device can display the display interface of the recommended service in different forms, thereby improving the user experience.

[0027] Illustratively, the second floating window may partially overlap with the first floating window, or may not overlap at all.

[0028] For example, when the electronic device is a foldable screen mobile phone, the foldable screen is in an unfolded state, including a first display area and a second display area. When the first floating window is displayed on the first display area, the second floating window can be displayed on the first display area or on the second display area.

[0029] According to the first aspect, or any implementation of the first aspect above, the method can be applied to a folding screen mobile phone, the folding screen is in an unfolded state, including a first display area and a second display area. The display interface of the first application is displayed in the first display area, and the display interface of the second application is displayed in the second display area. The electronic device displays the first interface in response to the received first operation, including: the electronic device displays the first interface in the first display area in response to the received first operation on the first application; the electronic device displays the second interface in response to the second operation on the first icon, including: the electronic device displays the second interface in the first display area in response to the second operation on the first icon. The method also includes: the electronic device displays a third floating window on the first display area in response to a long press operation and a drag operation on one of the annotations, and the third floating window moves to the second display area; wherein the drag operation is directed from the first display area to the second display area, and the key information content corresponding to one of the annotations is displayed in the third floating window; in response to the long press operation and the drag operation stopping, the key information content is displayed at the corresponding information editing in the display interface of the second application. In this way, in the application scenario where the electronic device is a foldable screen mobile phone, the user can drag the information of interest identified by the electronic device in one display area to the application displayed in another display area for use, which simplifies user operations and improves user experience.

[0030] According to the first aspect, or any implementation of the first aspect above, when the information type of the key information corresponding to one of the annotations is a code type, the content of the display interface is related to the key information corresponding to one of the annotations, including: displaying a link interface corresponding to the key information in the display interface; when the information type of the key information corresponding to one of the annotations is a character type, the content of the display interface is related to the key information corresponding to one of the annotations, including: displaying the content of the key information corresponding to one of the annotations in the corresponding information editing location in the display interface. In this way, the service interface displayed by the electronic device after recommending a service to the user adopts different display methods depending on the information type of the information of interest. It can be directly displaying and using the information of interest in the recommended service, or it can be directly jumping to the link interface matching the information of interest, realizing a closed loop from the identification of the information of interest to the service recommendation, and improving the user experience.

[0031] According to the first aspect, or any implementation of the first aspect above, the application is recommended based on the information type of the key information corresponding to one of the annotations, and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type of the key information corresponding to one of the annotations, and user habits; or, the application is recommended based on the information type of the key information corresponding to one of the annotations, user operations, and user portraits. In this way, when the electronic device recommends services to the user, it is not only based on the information type of the identified key information, but also based on the default recommendation rules determined by big data, or, also based on user habits, or also based on perceived user operations and user portraits, thereby improving the accuracy of the application recommended by the electronic device.

[0032] In the second aspect, the embodiment of the present application provides an information recommendation method. The method is applied to a folding screen mobile phone, and the folding screen is in an unfolded state, including a first display area and a second display area. The method includes: the folding screen mobile phone displays a first interface in the first display area; wherein the first interface includes a copyable text; the folding screen mobile phone displays a second interface in the first display area in response to a copy operation received on the first interface; wherein one or more application logos are displayed in the second interface, and the application is recommended according to the information type corresponding to the copied text; the folding screen mobile phone displays a third interface in a half-screen card window in the first display area or displays a third interface in the second display area in response to a click operation on the logo of one of the applications; wherein the display interface of one of the applications is displayed in the third interface, and the copied text is displayed in the corresponding information editing place in the display interface. In this way, the folding screen mobile phone recommends applications to users based on the text copied by the user. When the user selects an application, the folding screen mobile phone displays the interface of the application, and the copied text is displayed in the corresponding information editing place of the interface. In this way, the service recommendation based on the information of interest to the user in the copied text is realized, and the user experience is improved. Moreover, the folding screen mobile phone can display the display interface of the recommended service in different forms, thereby improving the user experience.

[0033] According to the second aspect, when an application logo is displayed in the second interface, the application logo is displayed in the form of a floating ball; when multiple application logos are displayed in the second interface, the multiple application logos are displayed in the form of a list; the list also displays the content of key information corresponding to one of the annotations. In this way, when a folding screen mobile phone recommends applications to users, the application logos are displayed in different forms depending on the number of applications. At the same time, when the folding screen mobile phone displays the application logos in the form of a list, it also displays the key information content, which can facilitate users to confirm whether the information based on the recommended application is the information of interest to the user, thereby ensuring the accuracy of the recommendation service.

[0034] According to the second aspect, or any implementation of the second aspect above, the foldable screen mobile phone displays the second interface in the first display area in response to the copy operation received on the first interface, including: the foldable screen mobile phone recognizes the copied text in response to the copy operation received on the first interface; when the copied text belongs to a preset type of key information, the foldable screen mobile phone recommends one or more pending applications according to the information type to which the copied text belongs; the foldable screen mobile phone sends the copied text to the SDK (Software Development Kit) of each pending application respectively, and receives confirmation information fed back by the SDK of the pending application; wherein the confirmation information is used to indicate whether the recommendation is correct; the foldable screen mobile phone selects the application to be displayed from one or more pending applications according to the confirmation information fed back by the SDK of each pending application, and displays the second interface in the first display area. In this way, the application recommended by the foldable screen mobile phone is confirmed twice by the SDK of the corresponding application, which further improves the accuracy of the recommended application.

[0035] According to the second aspect, or any implementation of the second aspect above, the application is recommended based on the information type corresponding to the copied text and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type corresponding to the copied text and user habits; or, the application is recommended based on the information type corresponding to the copied text, user operations, and user portraits. In this way, when the folding screen mobile phone recommends services to users, it is not only based on the information type of the copied text, but also based on the default recommendation rules determined by big data, or, based on user habits, or based on perceived user operations and user portraits, thereby improving the accuracy of the applications recommended by the electronic device.

[0036] In the third aspect, the embodiment of the present application provides an information recommendation method. The method includes: a folding screen mobile phone displays a first interface in a first floating window; wherein the first interface includes a copyable text; the folding screen mobile phone displays a second interface in the first floating window in response to a copy operation received on the first interface; wherein one or more application logos are displayed in the second interface, and the application is recommended according to the information type corresponding to the copied text; the folding screen mobile phone displays a third interface in the second floating window in response to a click operation on the logo of one of the applications; wherein the display interface of one of the applications is displayed in the third interface, and the copied text is displayed in the corresponding information editing place in the display interface. In this way, the folding screen mobile phone recommends applications to users based on the text copied by the user. When the user selects an application, the folding screen mobile phone displays the interface of the application, and the copied text is displayed in the information editing place corresponding to the interface. In this way, service recommendations based on the information of interest to the user in the copied text are realized, and the user experience is improved. Moreover, the folding screen mobile phone can display the display interface of the recommended service in the form of a floating window, thereby improving the user experience.

[0037] Illustratively, the second floating window may partially overlap with the first floating window, or may not overlap at all.

[0038] Exemplarily, the method is applied to a foldable screen mobile phone, the foldable screen is in an unfolded state, and includes a first display area and a second display area. When the first floating window is displayed on the first display area, the second floating window can be displayed on the first display area or on the second display area.

[0039] According to the third aspect, when an application logo is displayed in the second interface, the application logo is displayed in the form of a floating ball; when multiple application logos are displayed in the second interface, the multiple application logos are displayed in the form of a list; the list also displays the content of key information corresponding to one of the annotations. In this way, when a folding screen mobile phone recommends applications to users, the application logos are displayed in different forms depending on the number of applications. At the same time, when the folding screen mobile phone displays the application logos in the form of a list, it also displays the key information content, which can facilitate users to confirm whether the information based on the recommended application is the information of interest to the user, thereby ensuring the accuracy of the recommendation service.

[0040] According to the third aspect, or any implementation of the third aspect above, the method is applied to a folding screen mobile phone, the folding screen is in an unfolded state, and includes a first display area and a second display area. The folding screen mobile phone responds to the copy operation received on the first interface and displays the second interface in the first floating window, including: the folding screen mobile phone responds to the copy operation received on the first interface, identifies the copied text; when the copied text belongs to the key information of a preset type, the folding screen mobile phone recommends one or more pending applications according to the information type to which the copied text belongs; the folding screen mobile phone sends the copied text to the SDK (Software Development Kit) of each pending application, and receives the confirmation information fed back by the SDK of the pending application; wherein the confirmation information is used to indicate whether the recommendation is correct; the folding screen mobile phone selects the application to be displayed from one or more pending applications according to the confirmation information fed back by the SDK of each pending application, and displays the second interface in the first floating window. In this way, the application recommended by the folding screen mobile phone is confirmed by the SDK of the corresponding application twice, which further improves the accuracy of the recommended application.

[0041] According to the third aspect, or any implementation of the third aspect above, the application is recommended based on the information type corresponding to the copied text and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type corresponding to the copied text and user habits; or, the application is recommended based on the information type corresponding to the copied text, user operations, and user portraits. In this way, when the folding screen mobile phone recommends services to users, it is not only based on the information type of the copied text, but also based on the default recommendation rules determined by big data, or, also based on user habits, or also based on perceived user operations and user portraits, thereby improving the accuracy of the applications recommended by the electronic device.

[0042] In a fourth aspect, an embodiment of the present application provides an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps:

[0043] The electronic device displays a first interface in response to a received first operation; wherein a target image and a first icon are displayed in the first interface, and the first icon is used to indicate that key information of a preset type is recognized in the target image; the first operation includes: a screenshot operation, an operation of viewing an image in a gallery; the electronic device displays a second interface in response to a second operation on the first icon; wherein the target image and annotations of the key information are displayed in the second interface; the electronic device displays a third interface in response to a third operation on one of the annotations; wherein the identification of one or more applications is displayed in the third interface, and the application is recommended based on the information type of the key information corresponding to one of the annotations; the electronic device displays a fourth interface in response to a fourth operation on the identification of one of the applications; wherein a display interface of one of the applications is displayed in the fourth interface, and the content of the display interface is related to the key information corresponding to one of the annotations.

[0044] According to the fourth aspect, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device identifies a target image corresponding to the first operation in response to receiving the first operation; when the electronic device completes the identification and there is key information of a preset type in the target image, the electronic device displays a first interface.

[0045] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device responds to the received first operation and identifies the target image when the target image meets the real-time recognition condition.

[0046] According to the fourth aspect, or any implementation method of the fourth aspect above, when the first operation is a screenshot operation, the application displayed on the interface when the screenshot operation is performed meets the first recognition condition, and the user's screenshot sharing habit meets the second recognition condition, the target image meets the real-time recognition condition; wherein the first recognition condition is used to indicate the existence of an application that requires recognition, and the second recognition condition is used to indicate the user's operation habit that meets the requirements for real-time recognition.

[0047] According to the fourth aspect, or any implementation method of the fourth aspect above, when the first operation is an operation of viewing an image in a gallery, when the target image is an image taken by a camera, when the shooting attribute of the target image satisfies the third recognition condition, the target image satisfies the real-time recognition condition; when the first operation is an operation of viewing an image in a gallery, when the target image is a screenshot image, when the application displayed on the interface when the screenshot operation is performed satisfies the first recognition condition, the target image satisfies the real-time recognition condition; wherein the first recognition condition is used to indicate an application that has recognition requirements, and the third recognition condition is used to indicate a camera shooting mode that has image recognition requirements.

[0048] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device also performs the following steps: the electronic device displays a first camera shooting interface; the electronic device responds to the received photo operation, stores the target image obtained by the photo in a gallery, and identifies the target image; the electronic device responds to the operation of viewing the target image in the gallery, and displays the first interface.

[0049] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device identifies the target image when the shooting attributes of the target image meet the third recognition condition; wherein the third recognition condition is used to indicate a camera shooting mode that requires image recognition.

[0050] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device also performs the following steps: the electronic device displays a second camera shooting interface when the recognition is completed and there is key information of a preset type in the target image; wherein a second icon is also displayed in the second camera shooting interface, and the second icon is used to indicate that the recognition of the target image has been completed and key information of a preset type is recognized in the target image.

[0051] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device further performs the following steps: the electronic device responds to a received charging operation, and if there are unrecognized images in the gallery, the unrecognized images are identified in sequence; the electronic device responds to a received charging stop operation, and if there are unrecognized images in the gallery, the operation of identifying the unrecognized images is stopped.

[0052] According to the fourth aspect, or any implementation method of the fourth aspect above, when the logo of an application is displayed in the third interface, the logo of the application is displayed in the form of a floating ball; when the logos of multiple applications are displayed in the third interface, the logos of multiple applications are displayed in the form of a list; the list also displays the content of the key information corresponding to one of the annotations.

[0053] According to the fourth aspect, or any implementation manner of the fourth aspect above, the electronic device is a foldable screen mobile phone, and the foldable screen is in an unfolded state, including a first display area and a second display area; when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device displays a first interface in the first display area in response to a first operation received; the electronic device displays a second interface in the first display area in response to a second operation on a first icon; the electronic device displays a third interface in the first display area in response to a third operation on one of the annotations; the electronic device displays a fourth interface in the second display area in response to a fourth operation on identification of one of the applications; or, the electronic device displays the fourth interface in a half-screen card window on the first display area in response to a fourth operation on identification of one of the applications.

[0054] According to the fourth aspect, or any implementation of the fourth aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device displays a first interface in a first floating window in response to a first operation received; the electronic device displays a second interface in the first floating window in response to a second operation on the first icon; the electronic device displays a third interface in the first floating window in response to a third operation on one of the annotations; the electronic device displays a fourth interface in the second floating window in response to a fourth operation on the identification of one of the applications.

[0055] According to the fourth aspect, or any implementation of the fourth aspect above, the electronic device is a folding screen mobile phone, the folding screen is in an unfolded state, including a first display area and a second display area; the display interface of the first application is displayed in the first display area, and the display interface of the second application is displayed in the second display area. When the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device displays the first interface in the first display area in response to the first operation received on the first application; the electronic device displays the second interface in the first display area in response to the second operation on the first icon. When the computer program is executed by one or more processors, the electronic device also performs the following steps: the electronic device displays a third floating window on the first display area in response to a long press operation and a drag operation on one of the annotations, and the third floating window moves to the second display area; wherein the drag operation is directed from the first display area to the second display area, and the key information content corresponding to one of the annotations is displayed in the third floating window; the electronic device displays the key information content at the corresponding information editing position in the display interface of the second application in response to the long press operation and the drag operation stopping.

[0056] According to the fourth aspect, or any implementation method of the fourth aspect above, when the information type of the key information corresponding to one of the annotations is a code type, a link interface corresponding to the key information is displayed in the display interface; when the information type of the key information corresponding to one of the annotations is a character type, the content of the key information corresponding to one of the annotations is displayed in the corresponding information editing position in the display interface.

[0057] According to the fourth aspect, or any implementation of the fourth aspect above, the application is recommended based on the information type of the key information corresponding to one of the annotations, and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type of the key information corresponding to one of the annotations, and user habits; or, the application is recommended based on the information type of the key information corresponding to one of the annotations, user operations and user portraits.

[0058] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0059] In a fifth aspect, an embodiment of the present application provides a folding screen mobile phone. The folding screen of the folding screen mobile phone is in an unfolded state, including a first display area and a second display area; the folding screen mobile phone includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the folding screen mobile phone performs the following steps: the folding screen mobile phone displays a first interface in the first display area; the first interface includes copyable text; the folding screen mobile phone responds to the copy operation received on the first interface, and displays the second interface in the first display area; wherein, the logo of one or more applications is displayed in the second interface, and the application is recommended according to the information type corresponding to the copied text; the folding screen mobile phone responds to the click operation of the logo of one of the applications, and displays the third interface in the half-screen card window of the first display area, or displays the third interface in the second display area; wherein, the display interface of one of the applications is displayed in the third interface, and the copied text is displayed in the corresponding information editing position in the display interface.

[0060] According to the fifth aspect, when the logo of an application is displayed in the second interface, the logo of the application is displayed in the form of a floating ball; when the logos of multiple applications are displayed in the second interface, the logos of multiple applications are displayed in the form of a list; the list also displays the content of the key information corresponding to one of the annotations.

[0061] According to the fifth aspect, or any implementation method of the fifth aspect above, when the computer program is executed by one or more processors, the foldable screen mobile phone performs the following steps: the foldable screen mobile phone responds to the copy operation received on the first interface, and identifies the copied text; when the copied text belongs to a preset type of key information, the foldable screen mobile phone recommends one or more pending applications according to the information type to which the copied text belongs; the foldable screen mobile phone sends the copied text to the software tool development kit SDK of each pending application respectively, and receives confirmation information fed back by the SDK of the pending application; wherein the confirmation information is used to indicate whether the recommendation is correct; the foldable screen mobile phone filters out the applications to be displayed from one or more pending applications according to the confirmation information fed back by the SDK of each pending application, and displays the second interface in the first display area.

[0062] According to the fifth aspect, or any implementation of the fifth aspect above, the application is recommended based on the information type corresponding to the copied text and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type corresponding to the copied text and user habits; or, the application is recommended based on the information type corresponding to the copied text, user operations and user portraits.

[0063] The fifth aspect and any implementation of the fifth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

[0064] In a sixth aspect, an embodiment of the present application provides a folding screen mobile phone. The folding screen mobile phone includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the folding screen mobile phone performs the following steps: the folding screen mobile phone displays a first interface in a first floating window; wherein the first interface includes copyable text; the folding screen mobile phone responds to a copy operation received on the first interface, and displays a second interface in the first floating window; wherein, in the second interface, one or more application logos are displayed, and the application is recommended according to the information type corresponding to the copied text; the folding screen mobile phone responds to a click operation on the logo of one of the applications, and displays a third interface in the second floating window; wherein, in the third interface, the display interface of one of the applications is displayed, and the copied text is displayed in the corresponding information editing position in the display interface.

[0065] According to the sixth aspect, when an application logo is displayed in the second interface, the application logo is displayed in the form of a floating ball; when multiple application logos are displayed in the second interface, the multiple application logos are displayed in the form of a list; the list also displays the content of the key information corresponding to one of the annotations.

[0066] According to the sixth aspect, or any implementation of the sixth aspect above, the folding screen of the folding screen mobile phone is in an unfolded state, including a first display area and a second display area. When the computer program is executed by one or more processors, the folding screen mobile phone performs the following steps: the folding screen mobile phone responds to the copy operation received on the first interface and identifies the copied text; when the copied text belongs to a preset type of key information, the folding screen mobile phone recommends one or more pending applications according to the type of information to which the copied text belongs; the folding screen mobile phone sends the copied text to the SDK of each pending application respectively, and receives confirmation information fed back by the SDK of the pending application; wherein the confirmation information is used to indicate whether the recommendation is correct; the folding screen mobile phone selects the application to be displayed from one or more pending applications according to the confirmation information fed back by the SDK of each pending application, and displays the second interface in the first floating window.

[0067] According to the sixth aspect, or any implementation of the sixth aspect above, the application is recommended based on the information type corresponding to the copied text and the default recommendation rule corresponding to the information type; or, the application is recommended based on the information type corresponding to the copied text and user habits; or, the application is recommended based on the information type corresponding to the copied text, user operations and user portraits.

[0068] The sixth aspect and any implementation of the sixth aspect correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the third aspect and any implementation of the third aspect, which will not be repeated here.

[0069] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is run on an electronic device, the electronic device executes the information recommendation method of the first aspect and any one of the first aspects. Alternatively, the computer-readable storage medium includes a computer program, and when the computer program is run on a folding screen mobile phone, the folding screen mobile phone executes the information recommendation method of the second aspect and any one of the second aspects, or the folding screen mobile phone executes the information recommendation method of the third aspect and any one of the third aspects.

[0070] The seventh aspect and any implementation of the seventh aspect correspond to the first aspect and any implementation of the first aspect, or correspond to the second aspect and any implementation of the second aspect, or correspond to the third aspect and any implementation of the third aspect. Accordingly, the technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, or refer to the technical effects corresponding to the third aspect and any implementation of the third aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a schematic diagram of an application scenario shown as an example;

[0072] Figure 2 is a schematic diagram of the structure of an electronic device shown as an example;

[0073] Figure 3 A schematic diagram of the software structure of an electronic device is shown as an example;

[0074] Figure 4 A schematic diagram of module interaction provided in an embodiment of the present application;

[0075] Figure 5 This is one of the schematic diagrams of application scenarios shown as an example;

[0076] Figure 6 This is one of the schematic diagrams of application scenarios shown as an example;

[0077] Figure 7 This is one of the schematic diagrams of an exemplary application scenario;

[0078] Figure 8 A schematic diagram of module interaction provided in an embodiment of the present application;

[0079] Fig. 9 This is one of the schematic diagrams of an exemplary application scenario;

[0080] Fig.10 A schematic diagram of module interaction provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of a judgment process provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of module interaction provided in an embodiment of the present application;

[0083] Fig.13 This is one of the schematic diagrams of an exemplary application scenario;

[0084] Fig.14 A schematic diagram of module interaction provided in an embodiment of the present application;

[0085] Fig.15 This is one of the schematic diagrams of an exemplary application scenario;

[0086] Fig.16 This is one of the schematic diagrams of an exemplary application scenario;

[0087] Fig.17 A schematic diagram of a window display is shown as an example;

[0088] Fig.18 This is one of the schematic diagrams of an exemplary application scenario;

[0089] Fig.19 This is one of the schematic diagrams of an exemplary application scenario;

[0090] Fig. 20 This is one of the schematic diagrams of an exemplary application scenario;

[0091] Figure 21a to Figure 21c This is one of the schematic diagrams of exemplary application scenarios. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0093] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0094] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0095] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0096] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0097] like Figure 1 FIG. 1 is a schematic diagram of an exemplary application scenario. For example, when a user opens a mobile phone gallery to view pictures, refer to Figure 1 (1) The image 101 may contain information of interest to the user, such as text 1011 and QR code 1012 in the image 101. For example, in the scenario where the user takes a screenshot of the mobile phone display interface, refer to Figure 1 (2) The screenshot preview image 102 may contain information of interest to the user, such as text 1021 in the screenshot preview image 102. Taking the screenshot preview image 102 as a screenshot of the waybill details as an example, the text 1021 of interest to the user may be the waybill number information, address information, etc. Figure 1 (3) The entity 1031 collected in the mobile phone shooting collection interface 103 may contain information of interest to the user, which may be text information or code information. Taking the entity 1031 as an ID card as an example, the information of interest to the user may be the ID card number or the address information. For example, in the scenario where the user selects a copyable text from the browsing interface, refer to Figure 1 (4) The text selected by the user in the browsing interface 104 may be information of interest to the user, and these texts can usually be directly copied. Figure 1 The application scenario diagrams are illustrated using various display interfaces of tablet phones as examples. These application scenario diagrams are also applicable to foldable screen phones, tablets, etc., and this application does not limit this.

[0098] like Figure 2 FIG. 1 is a schematic diagram of the structure of the electronic device 100. Optionally, the electronic device 100 may be a terminal, which may also be referred to as a terminal device. The terminal may be a device with a camera such as a cellular phone (including a flat-type cellular phone and a folding-screen cellular phone) or a tablet computer (pad), which is not limited in this application. It should be noted that the schematic diagram of the structure of the electronic device 100 may be applicable to Figure 1The candy-bar phone in the candy-bar phone can also be applied to foldable screen phones and tablets. It should be understood that Figure 2 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0099] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0100] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0101] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0102] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0103] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0104] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0105] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0106] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0107] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0108] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0109] The wireless communication module 160 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0110] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0111] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0112] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0113] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0114] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0115] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0116] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0117] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0118] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0119] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121, so that the electronic device 100 implements the information recommendation method in the present application. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0120] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0121] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0122] The pressure sensor is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be set on the display screen 194. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor. The electronic device 100 can also calculate the touch position based on the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0123] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0124] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects.

[0125] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0126] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0127] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0128] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into three layers, from top to bottom, namely, the application layer, the application framework layer, and the kernel layer.

[0129] The application layer can include a series of application packages.

[0130] like Figure 3 As shown, the application package may include applications such as camera, gallery, map, browser, translation, shopping, short message, memo, entity recognition application, computing engine application, etc.

[0131] Among them, the entity recognition application is used to realize the recognition of text information, code information and other information, and to display the entity recognition results and the recommendation results of related applications corresponding to the entity recognition results; the computing engine application is used to identify entities of interest to users based on the information recognition results, and recommend related applications based on the entity recognition results.

[0132] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0133] like Figure 3As shown, the application framework layer may include a window manager, a perception service, an application execution management service, and may also include a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0134] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, etc.

[0135] The perception service is used to perceive the application life cycle and monitor user operations, such as copying text to the clipboard, viewing pictures, taking screenshots, etc.

[0136] The application operation management service is used to implement the operation management of each application at the application layer.

[0137] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0138] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0139] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, or notifications that appear in the system top status bar in the form of icons or scroll bar text, or notifications that appear on the screen in the form of dialog windows.

[0140] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, sensor driver, charging driver, etc.

[0141] Understandably, Figure 3 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0142] It is understandable that in order to implement the information recommendation method in this application, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0143] An embodiment of the present application provides an information recommendation method. Specifically, the electronic device in the embodiment of the present application can perform entity recognition when the user triggers entity recognition, and recommend associated applications that match the entity selected by the user, so that the user can directly open the associated applications of interest. Among them, entity refers to something that objectively exists and can be distinguished from each other. In the embodiment of the present application, an entity can be understood as key information, or information of interest to the user. For example, an address entity can be understood as information indicating an address. For another example, an ID card number entity can be understood as information indicating an ID card number. The following uses the term "entity" to explain the embodiment of the present application. It can be understood that replacing the term "entity" with "key information" or "information of interest" can also serve as an explanation of the embodiment of the present application.

[0144] It should be noted that the embodiments of the present application are only described by taking entities of interest to some users as examples, such as address entities, code entities, ID card number entities, mobile phone number entities, English text entities, etc. Exemplarily, the address involved in the address entity can be a detailed street address, such as No. XX, XX Street, XX District, XX City, or it can be a scenic spot name (such as the name of a famous cultural attraction or natural attraction), or it can be a famous building name, or it can be an address with corresponding information of interest on a map, etc. In other embodiments, the specific implementation methods of the present application can also be applied to entities of interest to other users, such as express number entities, website entities, email address entities, network disk download link entities (wherein users can obtain information of interest stored in the network disk through network disk download links), password entities (such as Taobao passwords, Douyin passwords, etc.), etc., and the present application will not repeat the description. In addition, it should be pointed out that in the embodiments of the present application, "image" and "picture" can be understood as the same concept, and "application" and "application program" can be understood as the same concept.

[0145] One possible application scenario is: when a user views or takes a screenshot of an image, there is an entity that the user is interested in, and in this case, entity recognition needs to be performed on the image. Another possible application scenario is: when a user copies text, there is an entity that the user is interested in, and in this case, entity recognition needs to be performed on the text.

[0146] Scene 1

[0147] In this scenario, taking the possible existence of entities of interest to the user in a picture of a bar phone as an example, the specific implementation method of the embodiment of the present application is described in detail.

[0148] like Figure 4 The following is a schematic diagram of the interaction process of each module. Figure 4 The process of the information recommendation method provided in the embodiment of the present application specifically includes:

[0149] S401, the perception service receives an entity recognition trigger operation.

[0150] The entity recognition trigger operation refers to an operation that can trigger the mobile phone to execute the information recommendation method. Specifically, it can be a user operation that triggers the perception service to send an entity recognition instruction, so that the entity recognition application and the computing engine application can jointly complete the entity recognition and determine the associated application to be recommended. In this embodiment, the entity recognition trigger operation can optionally be a user operation on a picture, such as a picture viewing operation, a screenshot operation, etc. Among them, the picture viewing operation can be an operation of viewing a picture in the gallery, or an operation of viewing a captured image in the camera shooting interface.

[0151] S402, the perception service sends an entity recognition instruction to the entity recognition application.

[0152] In this embodiment, the entity recognition instruction can be used to instruct the entity recognition application to identify the image, specifically, it can be to perform text recognition and code (such as QR code) recognition on the image, so that the entity recognition application can be combined with the computing engine application to jointly complete the entity recognition and determine the associated application to be recommended. Among them, the picture that the entity recognition instruction indicates to be identified refers to the picture corresponding to the entity recognition trigger operation, which can be a picture opened by the user in the gallery, a captured image that can be viewed in the camera shooting interface, or a screenshot image obtained by the user's screenshot operation. Optionally, the entity recognition instruction includes but is not limited to the picture information to be identified.

[0153] S403, the entity recognition application performs text recognition and code recognition on the image, and sends the text recognition and code recognition results to the computing engine application.

[0154] The entity recognition application receives the entity recognition instruction, determines the image to be recognized according to the entity recognition instruction, and performs text recognition and code recognition on the image.

[0155] Optionally, when the entity recognition application performs text recognition and code recognition on an image, it can call an OCR (Optical Character Recognition) application to complete the recognition. Optionally, the OCR application called by the entity recognition application can be installed on the mobile phone. In this way, the entity recognition application does not need to call the OCR application from the cloud to complete the recognition, which improves data security and can eliminate users' concerns about data security.

[0156] After the entity recognition application completes text recognition and code recognition on the image, it sends the text recognition results and code recognition results to the computing engine application. The text recognition results may include but are not limited to whether there is text and the recognized text information; the code recognition results may include but are not limited to whether there is a QR code and the recognized code information.

[0157] S404, the computing engine application performs entity recognition based on the text recognition result and the code recognition result, and sends the entity recognition result to the entity recognition application.

[0158] After receiving the text recognition result and the code recognition result sent by the entity recognition application, the computing engine application performs entity recognition based on the text recognition result and the code recognition result to determine one or more entities that the user may be interested in.

[0159] Optionally, the computing engine application obtains a plurality of entity types that the preset user may be interested in, and identifies one or more entities included in the image based on the plurality of entity types. Among them, the preset entity types that the user may be interested in include but are not limited to: telephone entity, address entity, express number entity, ID number entity, QR code entity, website entity, email entity, password entity, language text entity, etc. Exemplarily, the address entity may include but is not limited to information such as country, province, city, district, street (road), number, etc. Exemplarily, the language text entity refers to the text corresponding to a language, such as English text entity, Korean text entity, Chinese text entity, etc.

[0160] Exemplarily, the computing engine application can call an NLU (Natural Language Understanding) application to implement entity recognition. Optionally, the NLU application called by the computing engine application can be installed on the mobile phone. In this way, the computing engine application does not need to call the NLU application from the cloud to complete entity recognition, which improves data security and can eliminate users' concerns about data security.

[0161] After completing the entity recognition result, the computing engine application sends the entity recognition result to the entity recognition application. The entity recognition result includes but is not limited to: entity content, and the entity type corresponding to the entity content. It should be noted that the entity recognition result includes all entity content recognized in the image, and the entity type corresponding to each entity content.

[0162] S405, the entity recognition application adds an entity recognition result viewing icon in the first layer mask and displays the first layer mask.

[0163] The entity recognition result viewing icon is used to indicate that a preset type of entity (or key information) is recognized in the image.

[0164] After the entity recognition application receives the entity recognition result sent by the computing engine application, it determines that the image has completed entity recognition and recognizes the preset type of entity, then adds a first layer mask to the current display interface of the mobile phone, and adds an entity recognition result viewing icon in the first layer mask (which can be at any position). Among them, the entity recognition result viewing icon is clickable by the user so that the user can view the entity recognition result of the image. It should be noted that the first layer mask only includes the entity recognition result viewing icon. Adding the entity recognition result viewing icon through the layer mask will not cause damage to the image.

[0165] Optionally, the entity recognition application adds an entity recognition result viewing icon at any corner of the first layer mask, such as the lower right corner or the lower left corner. When the entity recognition application displays the first layer mask, the user can view the entity recognition result viewing icon on the current display interface of the mobile phone, for example, the entity recognition result viewing icon can be viewed on the currently viewed image, or the entity recognition result viewing icon can be viewed on the screenshot preview display interface.

[0166] S406, the perception service receives the operation of clicking the entity recognition result viewing icon.

[0167] The user can trigger the display of the entity recognition result of the picture by clicking (such as single-clicking or double-clicking) the entity recognition result viewing icon. The user can also trigger the display of the entity recognition result of the picture by other triggering operations on the entity recognition result viewing icon, such as long pressing, which is not limited in this embodiment.

[0168] S407, the perception service sends entity labeling instructions to the entity recognition application.

[0169] In this embodiment, the entity labeling instruction can be used to instruct the entity recognition application to perform entity labeling on the image, and the entity labeling result can show the entity recognition status of the image to the user.

[0170] S408, the entity recognition application marks the entity recognition result in the second layer mask and displays the second layer mask.

[0171] After receiving the entity annotation instruction, the entity recognition application obtains the entity recognition result of the image, adds a second layer mask to the current display interface of the mobile phone, and annotates the entity recognition result of the image in the second layer mask. The annotation is clickable by the user so that the user can select an entity that he is actually interested in. Optionally, the entity recognition result obtained by the entity recognition application includes, in addition to all entity contents recognized in the image and the entity type corresponding to each entity content, the location information of each entity content in the image or screenshot preview image, such as coordinate information.

[0172] The entity recognition application is applied in the second layer mask to mark the corresponding entity content according to the location information of the entity content in the picture or screenshot preview image. Optionally, the entity recognition application is applied in the second layer mask to mark the corresponding entity content according to the location information of the entity content in the picture or screenshot preview image, and the marking method matching the entity type. Exemplarily, the marking method can be divided into two types, one for text entities, such as telephone number entities, address entities, etc., and the other for code entities, such as QR code entities. For example, for text entities, the entity recognition application can use the underline marking method, that is, underline the text entity content; for code entities, the entity recognition application can use the dot marking method, that is, add a dot mark on the code entity content. It should be noted that the second layer mask only includes the entity annotation of the picture. Similarly, adding entity annotations through the layer mask will not cause damage to the picture.

[0173] After the entity recognition application completes the entity recognition result annotation in the second layer mask, the second layer mask is displayed, and the display of the first layer mask is canceled. At this time, the user sees one or more entity annotations of the image on the current display interface.

[0174] S409: The perception service receives the operation of clicking the entity annotation.

[0175] The user can trigger the display of recommended services matching the entity annotation (or the entity content corresponding to the entity annotation) by clicking (such as single-clicking or double-clicking) any entity annotation. The user can also trigger the display of recommended services matching the entity annotation by other triggering operations on an entity annotation, such as long pressing, which is not limited in this embodiment.

[0176] S410, the perception service sends a recommended application display instruction to the computing engine application.

[0177] In this embodiment, the recommended application display instruction can be used to instruct the computing engine application to recommend related applications, and to display related applications in combination with the entity recognition application. The displayed related applications can be clicked by the user to enable the functions of the related applications. Optionally, the recommended application display instruction includes but is not limited to: the entity type and entity content corresponding to the entity annotation clicked by the user.

[0178] S411, the computing engine application recommends associated applications that match the clicked entity annotation, and sends the associated application information to the entity recognition application.

[0179] The computing engine application receives the recommended application display instruction, parses the recommended application display instruction, determines the entity type to be recommended by the associated application, and recommends the associated application (also called associated service) that matches the entity type, that is, recommends the associated application that matches the clicked entity label. Taking the entity type as an address entity as an example, the associated services corresponding to the address entity recommended by the computing engine application include but are not limited to: display on the map, get route, copy, add to address book, add to memo, share, etc. Optionally, when the associated services recommended by the computing engine application include multiple types, the computing engine application can set priorities for these multiple associated services respectively.

[0180] Optionally, when recommending associated services corresponding to entity types, the computing engine application may recommend associated services corresponding to entity types according to the default recommendation rules corresponding to the entity types, and set the priorities of various associated services. Among them, the default recommendation rules may be determined in combination with the results of big data analysis. For example, the computing engine application obtains various user intentions after users obtain information of a certain entity type on a daily basis, as well as the priorities of various user intentions. These user intentions may be determined by the cloud in combination with big data statistics. Exemplarily, taking the telephone number entity type as an example, the user intentions of users after obtaining telephone numbers on a daily basis include but are not limited to: making calls, sending text messages, adding to the address book, copying, sharing, etc., then the associated services corresponding to the telephone number entity type may be recommended based on these user intentions. Assuming that among these user intentions, the frequency of making calls is the highest, the priority of the associated service such as making calls can be set to the highest priority.

[0181] For example, when the computing engine application recommends an associated service corresponding to an entity type, if it is determined that multiple applications corresponding to the same associated service are installed in the mobile phone, the multiple applications may be recommended at the same time. For example, when the computing engine application recommends a map service, if it is determined that two applications, A map and B map, are installed in the mobile phone, the two map applications may be recommended at the same time.

[0182] Optionally, when recommending associated services corresponding to entity types, the computing engine application may also recommend associated services corresponding to entity types in combination with user usage habits, and set priorities for various associated services. That is to say, the computing engine application learns the user's habits of operating the phone, and recommends associated services corresponding to the entity type based on the learning results. Exemplarily, taking the address entity type as an example, suppose that after obtaining the address, the user has always queried the address or shared the address, and has never added the address to the address book contact. The computing engine application learns the user's operating habits and can use querying the address and sharing the address as recommended associated services corresponding to the address entity, and set the priorities of the two associated services, querying the address and sharing the address, to the highest priority.

[0183] Optionally, when recommending associated services corresponding to entity types, the computing engine application can also combine the user operations and user portraits obtained by the perception service to recommend associated services corresponding to entity types, and set the priority of various associated services. For example, there is a meeting arrangement in the user's schedule on a certain day, and the schedule information includes the meeting time and meeting details, but does not include the meeting address. The perception service perceives that the screenshot of the meeting details saved in the schedule was viewed before the meeting started (such as one hour ago), and the entity label corresponding to the meeting address was clicked in the entity recognition result of the screenshot of the meeting details. Assume that the user portrait obtained by the perception service is: the user has never driven a car when traveling, but traveled by taxi or public transportation. The perception service sends the obtained user operations and user portraits to the computing engine application, and the computing engine application can recommend services in combination with the user operations and user portraits. At this time, the computing engine application can determine with a high probability that the user wants to take a taxi to the meeting address, and then recommend the associated service corresponding to the meeting address as a taxi service, and recommend the meeting address as the destination address of the taxi service.

[0184] After the computing engine application completes the recommendation of the associated application matching the clicked entity annotation, the computing engine application sends the corresponding associated application information to the entity recognition application. Optionally, the associated application information includes but is not limited to: the name of the associated application and the recommendation priority of the associated application.

[0185] S412: The entity recognition application displays a recommended application list card or a recommended application icon.

[0186] The entity recognition application receives the associated application information sent by the computing engine application and can display the associated application information. Exemplarily, the entity recognition application displays the identifier of the associated application, such as an icon and a brief description of the function.

[0187] Optionally, when the number of recommended applications sent by the computing engine application is one, the entity recognition application can directly obtain the icon of the recommended application and display the icon for the user to click to open the application; when the number of recommended applications sent by the computing engine application is multiple, the entity recognition application can obtain the icons of the multiple recommended applications and display them in the form of list cards for the user to select one recommendation from them and click to open.

[0188] For example, when the entity recognition application displays the icon of the recommended application, it can also display the name of the recommended application or a brief description of the application function (such as "open in map", "add to memo", etc.)

[0189] Exemplarily, when the entity recognition application displays icons of multiple recommended applications in the form of a list card, it can also display entity content that matches the entity annotation clicked by the user in one of the rows (for example, the first row) to facilitate the user to confirm the accuracy of the entity recognition result.

[0190] S413: In response to the click operation on the recommended application, the application operation management service sends an application start instruction to the recommended application.

[0191] When the only recommended application displayed by the entity recognition application meets the user's intention, the user can click the icon of the recommended application to open the recommended application; when the recommended application list card displayed by the entity recognition application includes a recommended application that meets the user's intention, the user can click the icon of the recommended application to open the recommended application.

[0192] The perception service receives a click operation of the user on the recommended application and sends an application start instruction for the recommended application to the application operation management service. After receiving the application start instruction, the application operation management service sends an application start instruction to the corresponding recommended application.

[0193] S414: Recommend the application to be enabled, and send an indication that the application has been enabled to the application operation management service.

[0194] After receiving the application start instruction, the recommended application executes the application start operation, and after the start is completed, sends an indication message that the application has been started to the application operation management service.

[0195] S415, the application operation management service sends an indication message to the entity recognition application that the recommended application has been opened.

[0196] The application operation management service receives the indication information that the recommended application has been started, and sends the indication information to the entity recognition application.

[0197] S416, the entity recognition application sends the entity content that matches the clicked entity annotation to the recommendation application.

[0198] After receiving the indication that the recommended application has been opened, the entity recognition application confirms that the recommended application has been opened, and then sends the entity content matching the clicked entity annotation, such as address information, phone number information, etc., to the recommended application.

[0199] S417, recommending applications to implement corresponding application functions according to entity content.

[0200] The recommended application receives the entity content that matches the clicked entity annotation, adds the entity content to the matching information edit, and then can implement the corresponding function of the recommended application based on the entity content. Among them, the recommended application implements the corresponding function of the recommended application based on the entity content, which can be to directly implement the corresponding function or to implement the corresponding function under the relevant operation of the user. Taking the entity content as a phone number and the recommended application as an example, the phone receives the phone number and adds it to the dial number edit, and then the phone dialing function can be directly implemented based on the phone number. Taking the entity content as address information and the recommended application as a taxi application as an example, the taxi application receives the address information and adds it to the destination edit. At this time, the departure place edit of the taxi application can default to the current location, and the user clicks to confirm to implement the taxi function of the taxi application.

[0201] In the embodiment of the present application, when triggered by a user's action, the mobile phone performs entity recognition on the picture corresponding to the action, and displays an entity recognition result viewing icon when entity recognition is completed, so that the user can click to view the entity recognition result of the picture. If the user clicks on the annotation information of an entity in the picture, the mobile phone can recommend related services that meet the user's intention for the user to choose, thereby improving the user's experience.

[0202] The following is combined with Figure 5 The application scenario shown, that is, the application scenario in which the user takes a screenshot on the mobile phone, explains the information recommendation method provided in the embodiment of the present application. Figure 5 (1) The user double-clicks the phone screen to take a screenshot of the waybill details interface. The phone's perception service receives the user's screenshot operation. In response to the user's operation, the phone's perception service sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation on the screenshot image. Before the entity recognition is completed, the screenshot preview interface can refer to Figure 5 (2) When entity recognition is completed, the entity recognition application displays the entity recognition result viewing icon. At this time, the preview interface can refer to Figure 5 (3) Figure 5 (3), the entity recognition result viewing icon 501 is exemplarily displayed in the lower right corner of the mobile phone display interface. Figure 5As shown in (3), the user clicks the entity recognition result viewing icon 501. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, the perception service of the mobile phone sends an entity labeling instruction to the entity recognition application, so that the entity recognition application completes the entity labeling operation. Figure 5 (4) The entity recognition application performs entity annotation operations based on the entity recognition results. Since the entities involved in the entity recognition results are all text entities, annotations (such as underscores) 502 can be used to annotate the entities involved in the entity recognition results. Figure 5 In (4), the annotated entities include the express order number entity, the telephone number entity and the address entity. Assume that the entity that the user is interested in is the address entity annotated with the annotation 5021, and the user can click on the annotation, that is, click on the annotation 5021. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, sends a recommended application display instruction to the computing engine application, so that the computing engine application combines with the entity recognition application to complete the operation of recommending related services. Figure 5 (5) When there are multiple recommended applications, the entity recognition application can display a recommended application list card 503 for the user. In the recommended application list card 503, each row displays an icon of a recommended application and a brief description of the application function. These recommended applications can be arranged in order of priority from high to low. Exemplarily, in each row of the recommended application list card, the icon of the recommended application is displayed on the left, and the brief description of the application function is displayed on the right. Optionally, Figure 5 As shown in (5), in the recommended application list card 503, the first line 5031 can also display entity content that matches the entity annotation clicked by the user, such as recognized address information, so that the user can verify whether it is the information of interest. Exemplarily, in the first line of the recommended application list card, an icon is displayed on the left side of the entity content to indicate that this behavior is recognized entity content. When the recommended application list card 503 includes recommended applications that meet the user's intentions, such as "Get Directions", the user can click on the icon 5032 of the recommended application to perform a route acquisition operation for the address in the first line 5031 of the recommended application list card 503. Figure 5 In (6), the map application is already open, and the address information filled in the destination address edit 504 is the address in the first row 5031 of the recommended application list card 503, that is, the entity content matching the annotation 5021 clicked by the user. Figure 5 (6) in the route.

[0203] The following is combined with Figure 6 The application scenario shown, that is, the application scenario in which a user views pictures in a gallery, explains the information recommendation method provided in the embodiment of the present application. Figure 6(1) The user clicks on the picture 601 in the mobile phone gallery to view the picture. The mobile phone's perception service receives the user's picture viewing operation. In response to the user's operation behavior, the mobile phone's perception service sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation of the user viewing the picture. When the entity recognition is completed, the entity recognition application displays the entity recognition result viewing icon. At this time, the picture display interface can refer to Figure 6 (2) Figure 6 In (2), the entity recognition result viewing icon 602 is exemplarily displayed in the lower right corner of the mobile phone display interface. Figure 6 As shown in (2), the user clicks the entity recognition result viewing icon 602. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, the perception service of the mobile phone sends an entity labeling instruction to the entity recognition application, so that the entity recognition application completes the entity labeling operation. Figure 6 (3) The entity recognition application performs entity annotation operations based on the entity recognition results. Since the entity recognition results involve entity code-type entities, annotations (such as dots) 603 can be used to annotate the entities (i.e., QR codes) involved in the entity recognition results. Assume that the entity that the user is interested in is an entity annotated with annotation 603. The user can click on the annotation. The perception service of the mobile phone receives the user's click operation and, in response to the user's operation, sends a recommended application display instruction to the computing engine application, so that the computing engine application can complete the operation of recommending related services in conjunction with the entity recognition application. Refer to Figure 6 (4) When there are multiple recommended applications, the entity recognition application can display a recommended application list card 604 for the user. In the recommended application list card 604, each row displays an icon of a recommended application and a brief description of the application function. These recommended applications can be arranged in descending order of priority. Figure 6 As shown in (4), if the QR code corresponds to a certain APP (Application), the entity recognition result of the QR code, such as link information, can be displayed in the first line 6041 of the recommended application list card 604. Exemplarily, in the first line of the recommended application list card, an icon is displayed on the left side of the entity recognition result to indicate that this behavior is an entity recognition result. When the recommended application list card 604 includes recommended applications that meet the user's intention, such as "Open in APP", the user can click on the APP icon 6042 in the corresponding row of the recommended application list card 604 to open the recognized QR code in the APP. The recognition result of the QR code can be exemplarily referred to. Figure 6 (5).

[0204] The following is combined with Figure 7The application scenario shown, that is, the application scenario in which the user takes a photo on the mobile phone, explains the information recommendation method provided in the embodiment of the present application. Figure 7 (1) The user clicks the camera icon 701 on the mobile camera interface to complete the camera operation. On the mobile camera interface, if the user wants to view the taken pictures, he can click Figure 7 (2) The captured picture viewing icon 702 in the camera interface shown. The perception service of the mobile phone receives the user's captured picture viewing operation. In response to the user's operation behavior, the perception service of the mobile phone sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation of the captured picture. When the entity recognition is completed, the entity recognition application displays the entity recognition result viewing icon. At this time, the captured picture viewing interface can refer to Figure 7 (3) Figure 7 (3), the entity recognition result viewing icon 703 is exemplarily displayed in the lower right corner of the mobile phone display interface. Figure 7 (3) The user clicks the entity recognition result viewing icon 703. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, the perception service of the mobile phone sends an entity labeling instruction to the entity recognition application, so that the entity recognition application completes the entity labeling operation. Figure 7 (4) The entity recognition application performs entity annotation operations based on the entity recognition results. Since the entities involved in the entity recognition results are all text entities, annotations (such as underscores) 704 can be used to annotate the entities involved in the entity recognition results. Figure 7 In (4), the annotated entities include the ID card number entity and the address entity. Assume that the entity that the user is interested in is the ID card number entity annotated with the annotation 7041, and the user can click on the annotation, that is, click on the annotation 7041. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, sends a recommended application display instruction to the computing engine application, so that the computing engine application combines with the entity recognition application to complete the operation of recommending related services. Figure 7 (5) When there are multiple recommended applications, the entity recognition application can display a recommended application list card 705 for the user. In the recommended application list card 705, each row displays an icon of a recommended application and a brief description of the application function. These recommended applications can be arranged in order of priority from high to low. Figure 7As shown in (5), in the recommended application list card 705, the first line 7051 can also display entity content that matches the entity annotation clicked by the user, such as an identified ID number, so that the user can verify whether it is the information of interest. Exemplarily, in the first line of the recommended application list card, an icon is displayed on the left side of the entity recognition result to indicate that this behavior is an entity recognition result. When the recommended application list card 705 includes a recommended application that meets the user's intention, such as "Add to Memo", the user can click on the icon 7052 of the recommended application to add the ID number in the first line 7051 of the recommended application list card 705 to the memo, refer to Figure 7 (6) Figure 7 In (6), the memo is opened, and the ID number filled in the memo editing interface is the address in the first row 7051 of the recommended application list card 705, that is, the entity content matching the mark 7041 clicked by the user.

[0205] In each of the above application scenarios, optionally, the user clicks on the application function description corresponding to the icon of the recommended application in the recommended application list card, which can achieve the same effect as clicking the icon of the recommended application.

[0206] Next, combine Figure 7 The application scenario shown, that is, the application scenario in which the user takes a photo on the mobile phone, explains the information recommendation method provided in the embodiment of the present application. Figure 7 (1) The user clicks the camera icon 701 on the camera interface of the mobile phone to complete the camera operation. The perception service of the mobile phone receives the user's camera operation. In response to the user's operation, the perception service of the mobile phone sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation of the taken picture. On the camera interface of the mobile phone, if the user wants to view the taken pictures, he can click Figure 7 (2) The picture viewing icon 702 in the camera interface shown in FIG. 1 is displayed. At this time, if the entity recognition is completed and the preset entity type exists in the picture, the entity recognition application displays the entity recognition result viewing icon. At this time, the viewing interface of the taken picture can refer to Figure 7 (3) Figure 7 (3), the entity recognition result viewing icon 703 is exemplarily displayed in the lower right corner of the mobile phone display interface. Figure 7 (3) The user clicks the entity recognition result viewing icon 703. The perception service of the mobile phone receives the user's click operation, and in response to the user's operation behavior, the perception service of the mobile phone sends an entity labeling instruction to the entity recognition application, so that the entity recognition application completes the entity labeling operation. Figure 7 (4)~ Figure 7 The explanation of (6) can be found above and will not be repeated here.

[0207] In the application scenario where the photo operation triggers entity recognition, in order to give the user a better user experience, the embodiment of the present application can be implemented by marking the completion of the picture entity recognition to prompt the user that the entity recognition of the picture just taken has been completed. This embodiment can avoid the phenomenon that the user clicks to view the taken photo too early and cannot view the picture entity recognition result, thereby improving the user experience.

[0208] like Figure 8 The following is a schematic diagram of the interaction process of each module. Figure 8 The process of the information recommendation method provided in the embodiment of the present application specifically includes:

[0209] S801, the perception service receives a photo taking operation.

[0210] S802, the perception service sends an entity recognition instruction to the entity recognition application.

[0211] Optionally, when the perception service determines that the image meets the entity recognition condition, it sends an entity recognition instruction to the entity recognition application. At this time, the entity recognition condition can be optionally the third recognition condition described below, which will not be described in detail here.

[0212] S803, the entity recognition application performs text recognition and code recognition on the image, and sends the text recognition and code recognition results to the computing engine application.

[0213] S804, the computing engine application performs entity recognition based on the text recognition result and the code recognition result, and sends the entity recognition result to the entity recognition application.

[0214] S805, the entity recognition application adds an entity recognition result viewing icon in the first layer mask and displays the first layer mask.

[0215] S806, the entity recognition application adds a picture entity recognition completion mark in the photo taking interface.

[0216] The image entity recognition completion flag is used to indicate that the image taken by the user just now has been recognized and a preset type of entity (or key information) has been recognized in the image.

[0217] When entity recognition is completed for the picture just taken, the entity recognition application can add a picture entity recognition completion mark in the photo taking interface (can be at any position) to prompt the user that entity recognition is completed for the picture just taken. Fig. 9 , at the taken picture viewing icon 702 in the camera interface, the entity recognition application can add a picture entity recognition completion mark 707. At this time, the user clicks the taken picture viewing icon 702, and the mobile phone interface jumps to the following Figure 7 (3) As shown. The user clicks Figure 7 For the process after the entity recognition result viewing icon 703 in (3), please refer to the above related explanations and will not be repeated here.

[0218] In the process of the information recommendation method provided in the embodiment of the present application, the process section after S806 can refer to the following. Figure 4 As for S406 to S417 shown, the relevant explanations of this embodiment can be found in the aforementioned embodiment and will not be repeated here.

[0219] In this scenario one, considering the power consumption of the mobile phone when performing entity recognition on the image, the specific implementation of the embodiment of the present application needs to analyze the triggering timing of the image entity recognition, and try to avoid performing image entity recognition when it is not necessary, so as to reduce the power consumption of the mobile phone.

[0220] like Fig.10 The following is a schematic diagram of the interaction process of each module. Fig.10 The process of the information recommendation method provided in the embodiment of the present application specifically includes:

[0221] S1001, the perception service receives an entity recognition trigger operation.

[0222] S1002: When the perception service determines that the image meets the entity recognition conditions, it sends an entity recognition instruction to the entity recognition application.

[0223] In this embodiment, considering the power consumption of the mobile phone, when the perception service receives an entity recognition trigger operation, it no longer directly sends an entity recognition instruction to the entity recognition application, but instead determines whether the image corresponding to the entity recognition trigger operation meets the entity recognition conditions. If so, the entity recognition instruction is sent to the entity recognition application; otherwise, the entity recognition instruction is not sent to the entity recognition application.

[0224] Among them, the types of entity recognition trigger operations of users are different, and the corresponding entity recognition conditions may be different. Optionally, the types of entity recognition trigger operations can be divided into two categories, one is a screenshot operation, and the other is a picture viewing operation. Exemplarily, the entity recognition conditions related to the screenshot operation may include, but are not limited to: a first recognition condition related to the application information corresponding to the screenshot operation, and a second recognition condition related to the user's screenshot sharing habits. Exemplarily, the entity recognition conditions related to the picture viewing operation may include, but are not limited to: a third recognition condition related to the camera shooting classification result (or camera shooting mode), and a first recognition condition related to the application information corresponding to the screenshot operation.

[0225] Exemplarily, the first identification condition may be that the application belongs to a preset application set. The first identification condition is used to indicate an application that has entity recognition requirements. That is, each application in the preset application set is an application that has entity recognition requirements. For example, the preset application set may include applications that generally involve entity recognition, such as WPS applications, PPT applications, etc., and the preset application set may not include music applications, taxi applications, and other applications that do not involve entity recognition.

[0226] Exemplarily, the second identification condition may be a preset set of user operation habits. The second identification condition is used to indicate user operation habits that are consistent with real-time entity recognition. That is, each user operation habit within the preset set of user operation habits is a user operation habit that is consistent with real-time entity recognition. Exemplarily, the preset set of user operation habits may include a user screenshot sharing habit of opening the screenshot before sharing. The preset set of user operation habits does not include direct sharing without opening the screenshot. If the user's screenshot sharing habit is direct sharing without opening the screenshot (such as swiping up directly to share after taking the screenshot), the user's screenshot sharing habit does not meet the second identification condition.

[0227] Exemplarily, the third identification condition may be that the camera shooting classification result belongs to a preset classification set (or the camera shooting mode belongs to a preset mode set). The third identification condition is used to indicate a camera shooting classification or camera shooting mode for which entity recognition needs exist. Among them, each camera shooting classification in the preset classification set is a camera shooting classification for which entity recognition needs exist. For example, the preset classification set may include classifications that generally involve entity recognition, such as documents, etc. The preset classification set does not include classifications that do not involve entity recognition, such as landscapes and portraits. Whether it is a camera shooting classification result or a camera shooting mode, it can include a classification label (or mode label) for further classification or identification. The same camera shooting mode may include multiple mode labels, some of which indicate that there is a need for entity recognition, and some of which indicate that there is no need for entity recognition.

[0228] like Fig.11 The following is a schematic diagram of the judgment process of the perception service. Fig.11 , the perception service determines whether an image meets the entity recognition conditions, including:

[0229] S1101, the perception service receives an entity recognition trigger operation and determines the operation type. If the operation type is a screenshot operation, the perception service executes S1102; if the operation type is a picture viewing operation, the perception service executes S1105.

[0230] S1102, the perception service obtains the application information corresponding to the screenshot operation.

[0231] When the user performs a screenshot operation, the application displayed on the display interface of the mobile phone is the application corresponding to the screenshot operation. Optionally, the application information includes but is not limited to the application name.

[0232] S1103, the perception service determines whether the application information meets the entity identification conditions, if so, execute S1104, if not, execute S1108.

[0233] In this step, the entity recognition condition may refer to the first recognition condition. The perception service determines whether the application information meets the first recognition condition, and if so, determines other entity recognition conditions, otherwise, determines that the image does not meet the entity recognition condition.

[0234] S1104, the perception service determines whether the user's screenshot sharing habits meet the screenshot operation trigger recognition conditions, if so, execute S1109, if not, execute S1101.

[0235] In this step, the screenshot operation triggering identification condition may refer to the above-mentioned second identification condition. The perception service obtains the user's screenshot sharing habit, and determines whether the user's screenshot sharing habit meets the above-mentioned second identification condition. If so, the perception service determines that the image meets the entity recognition condition, and then sends an entity recognition instruction to the entity recognition application. If not, the perception service determines that the image does not meet the entity recognition condition. At this point, the perception service needs to perceive whether the user's next operation is to view the screenshot, and re-execute the judgment process of whether the image meets the entity recognition condition.

[0236] S1105, the perception service determines the source of the image. If the image source is taken by a camera, the perception service executes S1106; if the image source is an application screenshot, the perception service executes S1102.

[0237] When a user performs an image viewing operation, the perception service needs to determine the source of the image. Optionally, the perception service can determine the source of the image from the attribute information of the image.

[0238] S1106, the perception service obtains the camera shooting classification result.

[0239] When the perception service determines that the image is taken by a camera, the perception service needs to further obtain the camera classification result of the image. Optionally, the perception service can determine the camera classification result of the image in the attribute information of the image.

[0240] S1107, the perception service determines whether the camera shooting classification result meets the entity recognition conditions. If so, execute S1109, otherwise execute S1108.

[0241] In this step, the entity recognition condition may refer to the third recognition condition. The perception service obtains the camera shooting classification result of the image, and determines whether the camera shooting classification result meets the third recognition condition. If so, the perception service determines that the image meets the entity recognition condition, and then sends an entity recognition instruction to the entity recognition application. If not, the perception service determines that the image does not meet the entity recognition condition.

[0242] Similarly, the perception service obtains the camera shooting mode of the image, and determines whether the icon meets the entity recognition condition by determining whether the camera shooting mode meets the entity recognition condition. Accordingly, the entity recognition condition may refer to the third recognition condition mentioned above. The perception service obtains the shooting mode of the image, and determines whether the shooting mode meets the corresponding third recognition condition. If so, the perception service determines that the image meets the entity recognition condition, and then can send an entity recognition instruction to the entity recognition application. If not, the perception service determines that the image does not meet the entity recognition condition.

[0243] It should be noted that the attribute information of the pictures taken by the camera and stored in the gallery may include multiple identifiers, such as the camera shooting classification result (or camera shooting mode), and one or more classification tags (or mode tags), etc. Furthermore, when the perception service determines whether the picture meets the entity recognition condition, it can determine whether the various identifiers included in its attribute information meet the entity recognition condition.

[0244] S1108, the perception service determines that the image does not meet the entity recognition conditions.

[0245] S1109, the perception service determines that the image meets the entity recognition conditions.

[0246] S1003, the entity recognition application performs text recognition and code recognition on the image, and sends the text recognition and code recognition results to the computing engine application.

[0247] S1004, the computing engine application performs entity recognition based on the text recognition results and the code recognition results, and sends the entity recognition results to the entity recognition application.

[0248] S1005, the entity recognition application adds an entity recognition result viewing icon in the first layer mask and displays the first layer mask.

[0249] In the process of the information recommendation method provided in the embodiment of the present application, the process section after S1005 can refer to the following. Figure 4 As for S406 to S417 shown, the relevant explanations of this embodiment can be found in the aforementioned embodiment and will not be repeated here.

[0250] In the application scenario of viewing pictures in the gallery, especially when there are a large number of pictures in the gallery that have not been recognized as entities. For example, after the mobile phone upgrades the operating system, since the old system does not have the function of picture entity recognition, and the new system has the function of picture entity recognition, it will cause a large number of pictures in the mobile phone gallery that have not been recognized as entities after the operating system is upgraded. Considering the power consumption problem of the mobile phone for entity recognition of pictures, the specific implementation of the embodiment of the present application performs picture entity recognition operations in batches when the mobile phone is charging, so as to reduce the power consumption of the mobile phone when the user is using it.

[0251] like Fig.12 The following is a schematic diagram of the interaction process of each module. Fig.12 The process of image entity recognition provided by the embodiment of the present application specifically includes:

[0252] S1201, the perception service receives a charging operation.

[0253] The charging operation may be an operation of connecting the mobile phone to the mains via a charger for charging, or an operation of connecting the mobile phone to an electronic device (such as a power bank or other terminal) via a data cable for charging.

[0254] S1202, the perception service sends an entity recognition instruction to the entity recognition application.

[0255] In response to the user's charging operation, the perception service sends an entity recognition instruction to the entity recognition application. In this embodiment, the entity recognition instruction can be used to instruct the entity recognition application to perform batch image entity recognition.

[0256] S1203, the entity recognition application performs image text recognition and code recognition on the entity unrecognized images in the gallery in turn, and sends the text recognition and code recognition results to the computing engine application.

[0257] S1204, the computing engine application performs entity recognition based on the text recognition results and the code recognition results, and sends the entity recognition results to the entity recognition application.

[0258] S1205, the entity recognition application adds an entity recognition result viewing icon in the first layer mask and displays the first layer mask.

[0259] S1206, the perception service receives the stop charging operation.

[0260] S1207, the perception service sends an entity recognition stop instruction to the entity recognition application.

[0261] In response to the user's stop charging operation, the perception service sends an entity recognition stop instruction to the entity recognition application. In this embodiment, the entity recognition stop instruction can be used to instruct the entity recognition application to stop performing image entity recognition.

[0262] In this embodiment, the entity unrecognized picture refers to a picture that has not been subjected to entity recognition, and does not include pictures that cannot be subjected to entity recognition or pictures for which the entity recognition result is empty.

[0263] Optionally, after determining the source of the image, the perception service can determine the images that cannot be entity recognized based on the first recognition condition related to the application information corresponding to the screenshot operation and the third recognition condition related to the camera shooting classification result, and mark these images. Optionally, the entity recognition application can also mark the images with empty entity recognition results based on whether the entity recognition results sent by the computing engine application are empty.

[0264] The entity recognition application combines the identifiers of the pictures in the gallery to obtain one entity unrecognized picture in turn, and combines with the computing engine application to complete entity recognition of the entity unrecognized picture until entity recognition of all entity unrecognized pictures in the gallery is completed, or until the mobile phone stops charging.

[0265] Reference Fig.13 Mobile phone charging application scenarios, Fig.13 (1) An example of an unrecognized entity picture in the mobile phone gallery is shown (there is a recognizable entity in the picture, but there is no entity recognition result viewing icon on its display interface). Fig.13 (2) The charging operation shown in the figure, the display interface changes of the entity unrecognized image can be referred to Fig.13 (3). At this time, if Fig.13 As shown in (3), an entity recognition result viewing icon 1301 is displayed on the image display interface, which means that the entity recognition of the image has been completed and the user can view the entity recognition result.

[0266] In the application scenario of charging a mobile phone, if the user operates the mobile phone and initiates an entity recognition trigger operation, the process of the information recommendation method can continue to refer to the above embodiment and will not be repeated here. It can be carried out in parallel with the process of batch entity recognition for a large number of entity unrecognized pictures. In this way, this embodiment combines the real-time method and the non-real-time method of using picture entity recognition to achieve effective control of mobile phone power consumption.

[0267] Scene 2

[0268] In this scenario, considering that there may be entities of interest to the user in the copyable text on the straight-screen mobile phone interface, the specific implementation method of the embodiment of the present application is described in detail using this as an example.

[0269] like Fig.14 The following is a schematic diagram of the interaction process of each module. Fig.14The process of the information recommendation method provided in the embodiment of the present application specifically includes:

[0270] S1401, the perception service receives the operation of copying text to the clipboard.

[0271] S1402, the perception service sends an entity recognition instruction to the entity recognition application.

[0272] In this embodiment, the entity recognition instruction may be used to instruct the entity recognition application to perform entity recognition on the text copied by the user. Optionally, the text copied by the user may include one or more characters.

[0273] S1403, the entity recognition application sends the copied text to the computing engine application.

[0274] In this embodiment, the entity recognition application does not need to repeatedly perform text recognition on the text copied by the user, and can directly obtain the text copied by the user to the clipboard and send it to the computing engine application for entity recognition.

[0275] S1404: The computing engine application performs entity recognition based on the received text and determines a recommended application corresponding to the entity recognition result.

[0276] Optionally, the computing engine application performs entity recognition based on the received text to determine whether the received text includes only one entity, such as only an address entity, or only a phone number entity, etc. If so, it recommends associated applications based on the entity type, and sends the entity recognition result and the associated application that matches the entity recognition result to the entity recognition application. If the computing engine application recognizes that the received text includes multiple entities, such as both an address entity and a phone number entity, the computing engine application may not perform the recommended operation of the associated application, and at the same time, it may also send the entity recognition result indicating that "a single entity cannot be recognized" to the entity recognition application.

[0277] Exemplarily, if the computing engine application recognizes that the received text only includes characters of one language, such as English characters, the text may be recognized as an entity to be translated, and a translation application may be recommended.

[0278] In this step, the computing engine application preliminarily recommends related applications corresponding to the entity recognition results.

[0279] S1405: The computing engine application sends instruction information to the recommended application SDK.

[0280] The indication information is used to instruct the recommended application SDK to determine whether the copied text corresponds to the current application. Optionally, the indication information includes but is not limited to the copied text.

[0281] When there are multiple associated applications recommended by the computing engine application, the computing engine application sends instruction information to the SDK of each recommended application respectively, so that each recommended application SDK respectively confirms the recommended application.

[0282] S1406: When the recommended application SDK determines that the copied text corresponds to the current application, the current application is determined to be a recommended application.

[0283] Exemplarily, the recommended application SDK performs semantic analysis on the received copied text to determine whether it is text corresponding to the current application. If so, the recommended application SDK confirms that the current application is a recommended application.

[0284] In this step, the recommended application SDK confirms the associated applications initially recommended by the computing engine application, further ensuring the accuracy of the recommended applications.

[0285] S1407, recommending that the application SDK sends instruction information to the computing engine application.

[0286] The indication information is used to indicate to the computing engine application whether the application is confirmed as a recommended application. Exemplarily, the indication information includes but is not limited to a confirmation mark and a denial mark.

[0287] Each recommended application SDK that receives the indication information sent by the computing engine application will feed back the indication information to the computing engine application to indicate to the computing engine application whether the application is a recommended application.

[0288] S1408: The computing engine application sends the entity recognition result and the recommended application information to the entity recognition application according to the instruction information sent by the recommended application SDK.

[0289] If the indication information sent by the recommended application SDK indicates that it is a recommended application, the computing engine application will finally confirm the recommended application as a recommended application; if the indication information sent by the recommended application SDK indicates that it is not a recommended application, the computing engine application will no longer recommend the application.

[0290] The computing engine application generates final recommended application information according to the instruction information sent by each recommended application SDK, and sends it to the entity recognition application.

[0291] In this way, the accuracy of the recommended applications is greatly guaranteed through the preliminary recommendation of the computing engine application and the secondary confirmation of each recommended application SDK.

[0292] S1409, the entity recognition application displays a recommended application list card or a recommended application icon.

[0293] Optionally, when the number of recommended applications sent by the computing engine application is one, the entity recognition application can directly obtain the icon of the recommended application and display the icon for the user to click to open the application; when the number of recommended applications sent by the computing engine application is multiple, the entity recognition application can obtain the icons of the multiple recommended applications and display them in the form of list cards for the user to select one recommendation from them and click to open.

[0294] In this scenario, since entity recognition is triggered by the user copying text to the clipboard, the computing engine application does not need to repeatedly recommend the copy service. Accordingly, the entity recognition application will not include the copy service in the recommended application list card, thus avoiding the problem of repeated service recommendations.

[0295] S1410: In response to a click operation on a recommended application, the application operation management service sends an application start instruction to the recommended application.

[0296] S1411, recommending that the application be enabled, and sending an indication that the application has been enabled to the application operation management service.

[0297] S1412, the application operation management service sends an indication message to the entity recognition application that the recommended application has been turned on.

[0298] S1413, the entity recognition application sends the recognized entity content to the recommendation application.

[0299] S1414, recommending applications to implement corresponding application functions according to entity content.

[0300] For matters not fully explained in this embodiment, please refer to the aforementioned embodiments and will not be repeated here.

[0301] The following is combined with Fig.15 and Fig.16 The application scenario shown, that is, the application scenario in which the user copies text to the clipboard, explains the information recommendation method provided in the embodiment of the present application.

[0302] Reference Fig.15 (1) After the user selects some characters in the mobile phone browsing interface, an icon 1501 including a "copy" option and a "search" option appears in the mobile phone browsing interface. The user clicks the "copy" option in icon 1501, and the mobile phone's perception service receives the user's operation of copying text to the clipboard. In response to the user's operation, the mobile phone's perception service sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation of the user's copied text, and recommends related service operations. In this application scenario, the computing engine application recognizes that the user's copied text belongs to the entity to be translated, so it recommends the translation application. Fig.15 As shown in (2), the entity recognition application is a translation application icon 1502, which allows the user to click to start the translation application. Fig.15 In (2), the translation application icon 1502 is illustratively displayed in the form of a floating ball. If the translation application meets the user's intention, the user can click the translation application icon 1502 to translate the text copied by the user. Fig.15 In (3), the translation application is opened in window 1503, and the text copied by the user is displayed in the original text editing area, and the translation function from the original text to the translated text is implemented.

[0303] Reference Fig.16 (1) After the user selects some characters in the mobile phone browsing interface, an icon 1601 including a "copy" option and a "search" option appears in the mobile phone browsing interface. The user clicks the "copy" option in icon 1601, and the perception service of the mobile phone receives the user's operation of copying text to the clipboard. In response to the user's operation, the perception service of the mobile phone sends an entity recognition instruction to the entity recognition application, so that the entity recognition application combines with the computing engine application to complete the entity recognition operation of the user's copied text, and recommends related service operations. In this application scenario, the computing engine application recognizes that the user's copied text belongs to a phone number entity, so it recommends multiple recommended applications that match the phone number entity. Fig.16 As shown in (2), the entity recognition application can display a recommended application list card 1602 for the user. In the recommended application list card 1602, each row displays an icon of a recommended application and a brief description of the application function. These recommended applications can be arranged in order of priority from high to low. Fig.16 As shown in (2), in the recommended application list card 1602, the first row 16021 can also display entity content that matches the entity annotation clicked by the user, such as a recognized phone number, so that the user can verify whether it is the information of interest. Exemplarily, in the first row of the recommended application list card, an icon is displayed on the left side of the entity recognition result to indicate that this behavior is an entity recognition result. It should be noted that although the copy service is also a recommended service that matches the phone number entity, in this application scenario, the copy service does not need to be recommended repeatedly, so the recommended application list card 1602 does not include the copy service. When the recommended application list card 1602 includes a recommended application that meets the user's intention, such as "Send Message", the user can click on the icon 16022 of the recommended application to send a message to the copied phone number, refer to Fig.16 (3) Fig.16 In (3), the message application is already open, and the phone number filled in the recipient phone number edit space 16031 in the message application display interface 1603 is the phone number displayed in the first row 16021 of the recommended application list card 1602, that is, the text copied by the user.

[0304] In the embodiment of the present application, when triggered by the action of the user copying text to the clipboard, the mobile phone performs entity recognition on the copied text and recommends related services that meet the user's intention based on the entity recognition results for the user to choose, thereby improving the user's experience.

[0305] On the basis of the above-mentioned embodiment, a folding screen mobile phone can also be used to implement the process of the information recommendation method, which will not be described in detail here. In the application scenario of a folding screen mobile phone, since the interface display form of the folding screen mobile phone is diverse, the display interface of different recommended services can be displayed in different display areas.

[0306] like Fig.17 The following is a diagram showing the recommended application display area. Fig.17 (1) When the foldable screen mobile phone is in the folded state (or straight state), the recommended application can be directly opened in the display area 1701. Please refer to the example in the above embodiment, which will not be repeated here. Fig.17 (2) When the foldable screen mobile phone is in the unfolded state, the recommended application can be directly opened in the display area 1702, and Fig.17 The display situation of (1) is similar, except for the size of the display area, which will not be described here.

[0307] When the foldable screen mobile phone is in the unfolded state, the left and right display areas of the foldable screen mobile phone can display different applications respectively, and the display interface of the recommended application can be displayed in the left display area or the right display area. Fig.17 (3), illustratively, if the original application (the application for which the user performs the entity recognition triggering operation) is opened in the display area 1703, the recommended application can be opened in the display area 1704, so that the user can view the display interfaces of the two applications at the same time. Fig.18 In the application scenario shown, when the foldable screen mobile phone is in the unfolded state, the left and right display areas of the foldable screen display different applications respectively. Fig.18 As shown in (1), the interface of the browsing application is displayed in display area 1801, and the interface of the chat application is displayed in display area 1802. The user takes a screenshot in the browsing application, triggering entity recognition of the screenshot image and service recommendation based on the entity selected by the user. The user selects the map service "Open in Map" to open the map application. Fig.18 As shown in (2), the screenshot interface of the browsing application continues to be displayed in the display area 1801, while the map application, as a recommended application, can be displayed in the display area 1802.

[0308] When the foldable phone is in the unfolded state, the left and right display areas of the foldable phone can display different applications respectively, and the recommended application can also be opened in a half-screen card window. Optionally, when the recommended application is not an independent APP, it can be opened in a half-screen card window. Fig.17 (4) For example, the interface of the original application 1 (the application for which the user performs the entity recognition trigger operation) is displayed in the display area 1705, and the interface of the recommended application can be displayed and opened in the window (half-screen card window) 1706 on the display area 1705. Optionally, if the original application 2 is the application for which the user performs the entity recognition trigger operation, the recommended application can also be opened in the window (half-screen card window) 1706, which is not limited to this. Fig.19 In the application scenario shown, when the foldable screen mobile phone is in the unfolded state, the left and right display areas of the foldable screen display different applications respectively. Fig.19 As shown in (1), the interface of the browsing application is displayed in display area 1901, and the interface of the memo application is displayed in area 1902. The user copies text to the clipboard in the browsing application, triggering entity recognition of the copied text and service recommendation. The user selects the recommended translation application and starts the translation application. Fig.19 As shown in (2), the interface of the browsing application continues to be displayed in display area 1901, the interface of the memo application continues to be displayed in display area 1902, and the translation application, as a recommended application, can be displayed in window 1903.

[0309] When the foldable screen phone is in the unfolded state, the left and right display areas of the foldable screen display different applications respectively, and the recommended applications can also be opened in the floating window. Fig.17 (5), illustratively, if the original application 3 (the application in which the user performs the entity recognition trigger operation) is opened in the floating window 1707, the recommended application can be opened in the newly created floating window 1708. illustratively, the floating window 1707 and the floating window 1708 can be respectively suspended in the left and right display areas of the folding screen mobile phone, such as Fig.17 (5) As shown. For example, floating window 1708 can also be suspended in the left display area (or right display area) of the folding screen mobile phone at the same time as floating window 1707, and floating window 1708 partially covers floating window 1707. Fig. 20 In the application scenario shown, when the folding screen mobile phone is in the unfolded state, different applications are displayed in the left and right display areas and floating windows of the folding screen. Fig. 20As shown in (1), the interface of the browsing application is displayed in display area 2001, the interface of the memo application is displayed in area 2002, and the interface of the gallery application is displayed in floating window 2003. Among them, floating window 2003 floats on display area 2001. When a user views a picture in the gallery, entity recognition of the picture is triggered, and service recommendations are made. The user selects the recommended one and opens it in a certain APP. Fig. 20 As shown in (2), the interface of the browsing application continues to be displayed in the display area 2001, the interface of the memo application continues to be displayed in the display area 2002, the interface of the gallery application continues to be displayed in the floating window 2003, and a certain APP is displayed as a recommended application in the newly created floating window 2004. Among them, the floating window 2004 is suspended on the window 2002.

[0310] Exemplarily, the floating window 2004 may also be suspended on the display area 2001 together with the floating window 2003. At this time, the floating window 2004 may also be suspended on the floating window 2003, covering part of the floating window 2003. The size of the floating window may be adjusted, and the embodiment of the present application does not limit the size of the floating window.

[0311] In the above scenario one, in an application scenario where a foldable screen mobile phone is used and the left and right display areas of the foldable screen display different applications respectively, the user initiates an entity recognition trigger operation in an application displayed in one of the display areas, triggering the entity recognition application and the computing engine service to complete the image entity recognition. The user clicks on the entity recognition result viewing icon, and in response to the user's operation behavior, the entity recognition application displays the entity recognition application. At this time, the user can long press and drag the entity annotation, and the entity content corresponding to the entity annotation will follow the movement in the interface. Optionally, the user can drag the entity content directly to another display area display application.

[0312] like Figure 21a to Figure 21c The figure shows an application scenario diagram, which describes in detail the process of a user long pressing and dragging an entity mark to move the entity content. Fig.21a As shown in FIG21 , the left and right display areas of the folding screen display the gallery interface and the memo interface respectively. When a user views an image in the gallery, entity recognition of the image is triggered, and the entity annotation result shown in FIG21 is displayed. Fig.21a and Figure 21b When the user long presses the entity annotation 2101 and drags it, the entity content 2102 corresponding to the entity annotation 2101 moves with the movement of the user's finger. If the user long presses the entity annotation 2101 and drags it to an editable position in the memo interface, the entity content 2102 corresponding to the entity annotation 2101 will be directly displayed at the editable position.

[0313] In this way, in the application scenario where different applications are displayed in the left and right display areas of a foldable screen mobile phone, in one of the display areas, the user long presses and drags the entity annotation, and the entity content corresponding to the entity annotation will move with it in the interface. The user can directly drag the entity content to the application displayed in the other display area. The user operation is simple and convenient, which improves the user experience.

[0314] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the information recommendation method in the above-mentioned embodiment.

[0315] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the information recommendation method in the above-mentioned embodiment.

[0316] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor may execute the computer execution instructions stored in the memory so that the chip executes the information recommendation method in the above-mentioned method embodiments.

[0317] Among them, the electronic device (such as a foldable screen mobile phone), computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0318] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0319] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0320] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An information recommendation method, characterized in that: Used in electronic equipment, including: Displaying a first interface; the first interface includes copyable text; In response to the copy operation received on the first interface, displaying a second interface; wherein icons of one or more applications are displayed in the second interface, and the applications are recommended based on the copied text; In response to a click operation on the icon of one of the applications, a third interface is displayed; wherein the display interface of one of the applications is displayed in the third interface, and the copied text is displayed in the corresponding information editing location in the display interface.

2. The method according to claim 1, characterized in that The electronic device is a folding screen mobile phone, and the folding screen is in an unfolded state and includes a first display area and a second display area; The displaying the first interface comprises: displaying the first interface in the first display area; The displaying the second interface comprises: displaying the second interface in the first display area; Displaying the third interface includes: displaying the third interface in a half-screen card window in the first display area, or displaying the third interface in a floating window in the first display area, or displaying the third interface in the second display area.

3. The method according to claim 1, characterized in that When an application icon is displayed in the second interface, the application icon is displayed in the form of a floating ball; When icons of multiple application programs are displayed in the second interface, the icons of the multiple application programs are displayed in the form of a list.

4. The method according to claim 1, characterized in that: The step of displaying a second interface in response to a copy operation received on the first interface includes: In response to a copy operation received on the first interface, recognizing the copied text; When the copied text belongs to a preset type of key information, recommend one or more pending applications according to the information type to which the copied text belongs; Sending the copied text to the software development kits SDK of each of the pending applications respectively, and receiving confirmation information fed back by the SDK of the pending application; wherein the confirmation information is used to indicate whether the recommendation is correct; According to the confirmation information fed back by the SDK of each of the pending applications, the application program to be displayed is screened out from the one or more pending applications, and the second interface is displayed.

5. The method according to claim 1, characterized in that The copied text is in English characters; Displaying icons of one or more application programs in the second interface includes: displaying an icon of a translation application in the second interface; In response to a click operation on an icon of one of the application programs, a third interface is displayed, including: In response to a click operation on the icon of the translation application, a third interface is displayed; wherein the third interface includes a program interface of the translation application, and the program interface includes the English characters and translations corresponding to the English characters.

6. The method according to claim 1, characterized in that The copied text is a telephone number; Displaying one or more application icons in the second interface includes: displaying an icon of a short message application in the second interface; In response to a click operation on an icon of one of the application programs, a third interface is displayed, including: In response to a click operation on the icon of the short message application, a third interface is displayed; wherein the third interface includes a program interface of the short message application, and the recipient's phone number editing area in the program interface is filled with the phone number.

7. The method according to claim 1, characterized in that The copied text is address information; Displaying one or more application icons in the second interface includes: displaying an icon of a map application in the second interface; In response to a click operation on an icon of one of the application programs, a third interface is displayed, including: In response to a click operation on the icon of the map application, a third interface is displayed; wherein the third interface includes a program interface of the map application, and the destination address edit in the program interface is filled with the address information.

8. The method according to claim 1, characterized in that The application is recommended based on the type of information corresponding to the copied text; The application is recommended based on the information type corresponding to the copied text and a default recommendation rule corresponding to the information type; Alternatively, the application is recommended based on the type of information corresponding to the copied text and user habits; Alternatively, the application is recommended based on the information type corresponding to the copied text, the user operation, and the user portrait.

9. The method according to claim 1, characterized in that: Also, includes: In response to the received first operation, a fourth interface is displayed; wherein the fourth interface displays a target image and a first icon, the first icon being used to indicate that key information of a preset type is recognized in the target image; the first operation includes: a screenshot operation, and an operation of viewing an image in a gallery; In response to a second operation on the first icon, a fifth interface is displayed; wherein the target image and a mark of the key information are displayed in the fifth interface; In response to a third operation on one of the annotations, a sixth interface is displayed; wherein icons of one or more applications are displayed in the sixth interface, and the applications are recommended according to the information type of the key information corresponding to the one of the annotations; In response to a fourth operation on the icon of one of the applications, a seventh interface is displayed; wherein a display interface of one of the applications is displayed in the seventh interface, and content of the display interface is related to key information corresponding to one of the annotations.

10. The method according to claim 9, characterized in that The displaying of the fourth interface in response to the received first operation includes: In response to the received first operation, identifying a target image corresponding to the first operation; When the recognition is completed and there is key information of a preset type in the target image, the fourth interface is displayed.

11. The method according to claim 10, characterized in that The step of identifying, in response to the received first operation, a target image corresponding to the first operation includes: In response to the received first operation, when the target image meets the real-time recognition condition, the target image is recognized.

12. The method according to claim 11, characterized in that When the first operation is a screenshot operation, the target image satisfies the real-time recognition condition, including: When the application displayed on the interface during the screenshot operation satisfies the first recognition condition, and the screenshot sharing habit of the user satisfies the second recognition condition, the target image satisfies the real-time recognition condition; The first identification condition is used to describe an application program that has an identification requirement, and the second identification condition is used to describe a user operation habit that complies with real-time identification.

13. The method according to claim 11, characterized in that When the first operation is an operation of viewing an image in a gallery, the target image satisfies a real-time recognition condition, including: In the case where the target image is an image taken by a camera, when the shooting attribute of the target image meets the third recognition condition, the target image meets the real-time recognition condition; In the case where the target image is a screenshot image, when the application displayed on the interface during the screenshot operation satisfies the first recognition condition, the target image satisfies the real-time recognition condition; The first recognition condition is used to describe an application program that has a recognition requirement, and the third recognition condition is used to describe a camera shooting mode that has an image recognition requirement.

14. The method according to claim 9, characterized in that Before displaying the fourth interface in response to the received first operation, the method further includes: Display the first camera shooting interface; In response to the received photographing operation, storing the target image obtained by photographing in a gallery, and identifying the target image; When the recognition is completed and there is key information of a preset type in the target image, the fourth interface is displayed in response to the received first operation, including: In response to an operation of viewing the target image in a gallery, the fourth interface is displayed.

15. The method according to claim 14, characterized in that The identifying the target image includes: When the shooting attribute of the target image satisfies a third recognition condition, the target image is recognized; wherein the third recognition condition is used to describe a camera shooting mode that requires image recognition.

16. The method according to claim 14, characterized in that After identifying the target image, the method further includes: When the recognition is completed and there is key information of a preset type in the target image, displaying a second camera shooting interface; Among them, a second icon is also displayed in the second camera shooting interface, and the second icon is used to indicate that the target image has been recognized and key information of a preset type has been recognized in the target image.

17. The method according to claim 9, characterized in that Also includes: In response to the received charging operation, if there are unrecognized images in the gallery, the unrecognized images are recognized in sequence; In response to the received charging stop operation, if there are unrecognized images in the gallery, the operation of recognizing the unrecognized images is stopped.

18. The method according to claim 9, characterized in that When an application icon is displayed in the sixth interface, the application icon is displayed in the form of a floating ball; When icons of multiple applications are displayed in the sixth interface, the icons of the multiple applications are displayed in the form of a list; the list also displays the content of the key information corresponding to one of the annotations.

19. The method according to claim 9, characterized in that The electronic device is a folding screen mobile phone, and the folding screen is in an unfolded state and includes a third display area and a fourth display area; The displaying of the fourth interface in response to the received first operation includes: displaying the fourth interface in the third display area in response to the received first operation; The displaying of the fifth interface in response to the second operation on the first icon comprises: displaying the fifth interface in the third display area in response to the second operation on the first icon; The displaying of the sixth interface in response to the third operation on one of the annotations includes: displaying the sixth interface in the third display area in response to the third operation on one of the annotations; The displaying of the seventh interface in response to the fourth operation on the icon of one of the applications comprises: displaying the seventh interface in the fourth display area in response to the fourth operation on the icon of one of the applications; or In response to a fourth operation on the icon of one of the applications, a seventh interface is displayed in a half-screen card window on the third display area.

20. The method according to claim 9, characterized in that The displaying of the fourth interface in response to the received first operation includes: displaying the fourth interface in the first floating window in response to the received first operation; The displaying the fifth interface in response to the second operation on the first icon includes: displaying the fifth interface in the first floating window in response to the second operation on the first icon; The displaying the sixth interface in response to the third operation on one of the annotations includes: displaying the sixth interface in the first floating window in response to the third operation on one of the annotations; The displaying of the seventh interface in response to the fourth operation on the icon of one of the applications includes: displaying the seventh interface in the second floating window in response to the fourth operation on the icon of one of the applications.

21. The method according to claim 9, characterized in that The electronic device is a foldable screen mobile phone, the foldable screen is in an unfolded state, and includes a third display area and a fourth display area, the third display area displays a display interface of a first application, and the fourth display area displays a display interface of a second application; The displaying of the fourth interface in response to the received first operation includes: In response to receiving a first operation on the first application, displaying a fourth interface in the third display area; The displaying a fifth interface in response to the second operation on the first icon includes: In response to a second operation on the first icon, displaying a fifth interface in the third display area; The method further comprises: In response to a long press operation and a drag operation on one of the annotations, a third floating window is displayed on the third display area, and the third floating window is moved to the fourth display area; wherein the drag operation is directed from the third display area to the fourth display area, and the third floating window displays key information content corresponding to the one of the annotations; In response to the long press operation and the drag operation stopping, the key information content is displayed in the corresponding information editing location in the display interface of the second application.

22. The method according to claim 9, characterized in that When the information type of the key information corresponding to one of the annotations is a code type, the content of the display interface is related to the key information corresponding to one of the annotations, including: The display interface displays a link interface corresponding to the key information; When the information type of the key information corresponding to one of the annotations is character type, the content of the display interface is related to the key information corresponding to one of the annotations, including: In the corresponding information editing area in the display interface, the content of the key information corresponding to one of the annotations is displayed.

23. The method according to claim 9, characterized in that The application is recommended based on the information type of the key information corresponding to one of the annotations and a default recommendation rule corresponding to the information type; Alternatively, the application is recommended based on the information type of the key information corresponding to one of the annotations and user habits; Alternatively, the application is recommended based on the information type of the key information corresponding to one of the annotations, the user operation, and the user portrait.

24. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the information recommendation method as described in any one of claims 1-23.

25. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program runs on an electronic device, the electronic device executes the information recommendation method as described in any one of claims 1 to 23.

Images