Application Recommendation Method, Device, Equipment and Computer Readable Storage Medium

Abstract

An embodiment of the present application discloses an application recommendation method, apparatus, device, and computer-readable storage medium. The method includes: obtaining touch operation information acting on a user interface; obtaining screen element information corresponding to the user interface based on the touch operation information; both the screen element information and the touch operation information are interaction data related to the user's current operation and can meet the user's immediate service requirements. Based on the touch operation information and the screen element information, obtain the association degrees between the screen element information and multiple applications, where the multiple applications are applications of the same type; recommend and display the applications whose association degrees meet preset conditions. By combining the dual-modal information of the screen element information and the touch operation information, the embodiment of the present application considers the interaction data of the user interface from two dimensions, and then combines with the association degrees with multiple applications, improving the accuracy of the application recommendation results.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an application recommendation method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the continuous development of intelligent interaction technology, the interaction mode centered on intelligent devices is gradually developing towards a service interaction mode centered on people. By providing intelligent recommendation services to users on intelligent devices, the convenience of user operations is improved.

[0003] In the prior art, by collecting the historical behavior data of users on intelligent devices and combining it with the usage data of popular services, application recommendation results are generated.

[0004] However, in the prior art, relying on the historical behavior data of users and the usage data of popular services, when a user performs a current operation and interacts with the screen, the user can only passively receive application recommendations. The application recommendation results often have no association with the current operation, cannot meet the immediate service needs of users, and reduce the accuracy of application recommendation results. Summary of the Invention

[0005] Embodiments of this application are expected to provide an application recommendation method, apparatus, device, and computer-readable storage medium. By combining the dual-modal information of screen element information and touch operation information, the interaction data of the user interface is considered from two dimensions, and then combined with the association degrees with multiple applications, the accuracy of application recommendation results is improved.

[0006] In a first aspect, an embodiment of this application provides an application recommendation method, the method includes: obtaining touch operation information acting on a user interface; based on the touch operation information, obtaining screen element information corresponding to the user interface; based on the touch operation information and the screen element information, obtaining the association degrees between the screen element information and multiple applications, where the multiple applications are applications of the same type; recommending and displaying applications whose association degrees meet a preset condition.

[0007] In a second aspect, an embodiment of this application provides an application recommendation method, the method includes: obtaining touch operation information for a user interface, and obtaining screen element information corresponding to the user interface; performing joint learning based on the screen element information and the touch operation information, and performing association processing with preset sample fusion information and preset sample sequence behavior information of each application to obtain the association degrees between the screen element information and multiple applications; recommending and displaying applications whose association degrees meet a preset condition.

[0008] In a third aspect, an embodiment of the present application provides an application recommendation device, the device includes: a first acquisition unit, configured to acquire touch operation information acting on a user interface; and based on the touch operation information, acquire screen element information corresponding to the user interface; a first association unit, configured to acquire the association degree between the screen element information and multiple applications based on the touch operation information and the screen element information, the multiple applications being applications of the same type; a first display unit, configured to recommend and display the applications whose association degree meets a preset condition.

[0009] In a fourth aspect, an embodiment of the present application provides an application recommendation device, the device includes: a second acquisition unit, configured to acquire touch operation information for a user interface, and acquire screen element information corresponding to the user interface; a second association unit, configured to perform joint learning based on the screen element information and the touch operation information, and perform association processing with preset sample fusion information and preset sample sequence behavior information of each application, to acquire the association degree between the screen element information and multiple applications; a second display unit, configured to recommend and display the applications whose association degree meets a preset condition.

[0010] In a fifth aspect, an embodiment of the present application provides an application recommendation device, the device includes a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the application recommendation method described in the first aspect or the second aspect above.

[0011] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which an executable instruction is stored, and when being executed by a processor, it implements the application recommendation method described in the first aspect or the second aspect above.

[0012] An embodiment of the present application provides an application recommendation method, device, device and computer-readable storage medium. According to the solution provided by the embodiment of the present application, touch operation information acting on a user interface is acquired; based on the touch operation information, screen element information corresponding to the user interface is acquired; both the screen element information and the touch operation information are interaction data related to the user's current operation, and can meet the user's immediate service requirements. Based on the touch operation information and the screen element information, the association degree between the screen element information and multiple applications is acquired, the multiple applications being applications of the same type; the applications whose association degree meets a preset condition are recommended and displayed. By combining the dual-modal information of the screen element information and the touch operation information, the embodiment of the present application considers the interaction data of the user interface from two dimensions, and then combines with the association degree with multiple applications, improving the accuracy of the application recommendation result. Description of the Drawings

[0013] Figure 1An exemplary schematic diagram of an application recommendation result provided by an embodiment of the present application;

[0014] Figure 2 An optional step flowchart of an application recommendation method provided by an embodiment of the present application;

[0015] Figure 3 An optional step flowchart of another application recommendation method provided by an embodiment of the present application;

[0016] Figure 4 An optional step flowchart of yet another application recommendation method provided by an embodiment of the present application;

[0017] Figure 5 An optional flowchart for calculating the hidden vector of a screen element provided by an embodiment of the present application;

[0018] Figure 6 An optional flowchart for calculating the hidden vector of a touch action provided by an embodiment of the present application;

[0019] Figure 7 An optional step flowchart of yet another application recommendation method provided by an embodiment of the present application;

[0020] Figure 8 An optional flowchart for generating a recommendation result provided by an embodiment of the present application;

[0021] Figure 9 An exemplary representation schematic diagram of an application recommendation result provided by an embodiment of the present application;

[0022] Figure 10 Another exemplary representation schematic diagram of an application recommendation result provided by an embodiment of the present application;

[0023] Figure 11 An optional step flowchart of yet another application recommendation method provided by an embodiment of the present application;

[0024] Figure 12 A structural schematic diagram of an application recommendation device provided by an embodiment of the present application;

[0025] Figure 13 A structural schematic diagram of another application recommendation device provided by an embodiment of the present application;

[0026] Figure 14 A schematic diagram of the composition structure of an application recommendation device provided by an embodiment of the present application. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that some of the embodiments described herein are only used to explain the technical solutions of the present application and do not limit the technical scope of the present application.

[0028] To better understand the application recommendation method provided in the embodiments of the present application, before introducing the technical solutions of the embodiments of the present application, the related technologies will be described first.

[0029] In the related art, an intelligent application recommendation device generates application recommendation results by collecting historical behavior data of a user on an intelligent device and combining popular usage service data, such as Figure 1 shown Figure 1 is an exemplary schematic diagram of an application recommendation result provided for the embodiments of the present application. Figure 1 Taking the terminal as a mobile phone as an example, the application recommendation scenario of the mobile phone screen is shown. Figure 1 The content shown in the "Application Suggestions" and "Popular Applications" columns in Figure 1 belongs to the application recommendation results obtained by the intelligent application recommendation device of the terminal through calculation and analysis. Among them, "Application Suggestions" is an application recommendation generated based on the collected historical behavior data of the user on the intelligent device, and "Popular Applications" is an application recommendation generated based on the popular services connected to the intelligent device, that is, an application recommendation generated based on popular usage service data. Different recommended contents are represented by different letters in

[0030] The intelligent application recommendation method has the following characteristics: In the use of recommendation data, historical browsing, click and other interaction data of users, as well as Internet popular data, are usually used as the reference basis for application recommendation to users. In the content of the recommendation service, it mainly provides users with application recommendations, advertisement recommendations or information-based messages, etc. In the nature of the recommendation service, it is mainly to increase the click-through rate and activity rate of application software. Therefore, it focuses on guiding users to use the application software that the intelligent application recommendation device wants users to use.

[0031] Based on the above characteristics in the related art, intelligent application recommendation has at least one or more of the following disadvantages in terms of user experience and product performance: (1) Since the related art relies on the user's historical behavior data and popular usage service data to recommend the services that the user is about to use, it is difficult to cover the user's immediate service needs. (2) Since only the user's historical behavior data is used in the related art and the user's active intention operations are not considered, that is, there is no interaction between the intelligent application recommendation device and the user's active behavior, and the user can only passively receive application recommendations. (3) Since only the user's historical behavior data is used in the related art and the information modality is single, the accuracy of the application recommendation results in the related art is reduced.

[0032] An embodiment of the present application provides an application recommendation method. As Figure 2 shown, Figure 2 is a flowchart of the steps of an application recommendation method provided by an embodiment of the present application. The application recommendation method includes the following steps:

[0033] S201. Obtain touch operation information acting on the user interface.

[0034] In an embodiment of the present application, when the user interacts with the application recommendation device, the user operates on the information of interest on the user interface, and touch operation information is generated during the operation. The touch operation information is information acting on the user interface, and the application recommendation device can obtain the touch operation information.

[0035] In an embodiment of the present application, the user interface is the page when the user interacts with the application recommendation device. The user interface can be understood as the current page, the current user interface. The touch operation information represents the interaction data between the user and the user interface. The touch operation information may include: operation content and touch operation. The touch operation includes, but is not limited to, click, double-click, long-press, two-finger pinch, two-finger spread, slide, and drag; the operation content includes, but is not limited to, copy, forward, share, drag, reminder, save, favorite, and stay, etc. Among them, stay means that the user operates on the control on the user interface, and the operation content is all content other than other user operations. Taking the optional controls in the sub-menu popped up by the operating system including copy, forward, share, drag, reminder, save, and favorite as an example, the user does not select the optional controls in the sub-menu, but selects other controls, or the user selects the control outside the sub-menu.

[0036] In some embodiments, the touch operation information includes: touch operation and operation content; the above S201 can be implemented through the following three examples. The first example is to obtain the touch operation acting on the user interface; display the operation content corresponding to the touch operation.

[0037] In an embodiment of the present application, when obtaining the touch operation information, it is triggered by the user's touch operation. First, obtain the touch operation acting on the user interface, and display the operation content corresponding to the touch operation in the user interface. Exemplarily, taking the touch operation as a touch gesture as an example, when the user needs to perform a copy operation on a certain text information, when it is detected that the user's touch gesture is "long-press", generate the operation content corresponding to "long-press", and the operation content is copy, share, forward, save, favorite, etc., and the user selects the operation content "copy".

[0038] In the embodiments of the present application, a touch operation acting on a user interface is obtained, operation content corresponding to the touch operation is displayed, and if the user selects the operation content, the operation content is obtained, so as to obtain touch operation information, which reduces resource consumption compared with the solution of directly obtaining touch operation information.

[0039] In some embodiments, the touch operation information includes: a touch operation and operation content; the above S201 is implemented through the following second example. Second example, obtain a touch operation acting on the user interface; based on the touch operation, obtain screen element information corresponding to the user interface; based on the touch operation and the screen element information, display corresponding operation content.

[0040] In the embodiments of the present application, when obtaining touch operation information, it is triggered by the user's touch operation. The touch operation reflects that the user needs to operate on some content in the user interface so as to select operation content next, and the screen element information reflects the operation content that can be supported. Therefore, the displayed operation content is related to the touch operation and the screen element information. Based on the touch operation and the screen element information, corresponding operation content is displayed, which improves the accuracy of the operation content. Exemplarily, taking the touch operation as a touch gesture as an example, when the user needs to perform a copy operation on a certain text information, when the user's touch gesture "long press" is detected, the screen element information corresponding to the user interface is obtained, and the operation content is generated based on the position where the touch gesture is located and the screen element information. The operation content is copy, share, forward, save, favorite, etc., and the user selects the operation content "copy".

[0041] In the embodiments of the present application, first obtain a touch operation acting on the user interface, based on the touch operation, obtain screen element information corresponding to the user interface; based on the touch operation and the screen element information, display corresponding operation content, and if the user selects the operation content, obtain the operation content, so as to obtain touch operation information, which reduces resource consumption compared with the solution of directly obtaining touch operation information.

[0042] In some embodiments, the touch operation information includes: a touch operation and operation content; the above S201 is implemented through the following third example. Third example, under the action of a selection operation on the user interface, display the operation content corresponding to the selected content; within a preset time range, obtain a touch operation on the operation content; the touch operation includes a selection operation.

[0043] In the embodiments of the present application, during the process of a user interacting with an application recommendation device, the operating system continuously generates data, including touch operation information and screen element information. The embodiments of the present application do not continuously obtain touch operation information and screen element information, nor do they continuously recommend applications to the user, and the operating system does not continuously obtain touch operations and screen element information. It can be understood that the user can continuously perform touch operations on the application recommendation device, but the application recommendation device does not respond to all touch operations. Under the effect of a selection operation on the user interface, the operation content corresponding to the selected content is displayed. For example, when the selected operation content is at least one of copy, forward, share, drag, reminder, save, and favorite, and it is detected that the user has a strong intention such as wanting to super share, forward, or open, the touch operation for the operation content will be obtained within a certain time range before and after the selection operation. This also illustrates that the application recommendation method in the embodiments of the present application is triggered by the operation content.

[0044] In the embodiments of the present application, under the effect of a selection operation on the user interface, the operation content corresponding to the selected content is displayed; thereby, within a preset time range, the touch operation for the operation content is obtained, which reduces resource consumption compared with the method of continuously obtaining touch operations.

[0045] S202. Based on the touch operation information, obtain the screen element information corresponding to the user interface.

[0046] In the embodiments of the present application, as the user has real-time interaction with the user interface, the screen element information corresponding to the user interface changes. The operating system continuously generates data, including touch operation information and screen element information, and the application recommendation device does not continuously recommend applications to the user. Based on the touch operation information, the screen element information corresponding to the user interface is obtained, and this screen element information is the screen element corresponding to the user's touch operation information, which improves the accuracy of the screen element information.

[0047] In the embodiments of the present application, the screen element information represents the information of all elements displayed in the user interface, including but not limited to control information, text information, and image information.

[0048] In some embodiments, when obtaining the screen element information corresponding to the user interface in S202 above, it can be implemented in the following manner. Identify the control information of the user interface or the area to be recognized to obtain structural knowledge; identify the text content of the user interface or the area to be recognized to obtain text knowledge; identify the image content of the user interface or the area to be recognized to obtain visual knowledge; perform service parsing on the user interface to determine the current application service set; at least one of the text knowledge, visual knowledge, and the current application service set, and the structural knowledge are used as the screen element information.

[0049] In the embodiments of the present application, the relationships between structural knowledge representation entities and entities can also be understood as the relationships between controls and controls. Textual knowledge represents text content such as movies, texts, or links, and visual knowledge represents image content such as covers or posters.

[0050] Exemplarily, determining the current application service set by performing service parsing on the current user interface can be understood as selecting the application services related to the current user interface as the current application service set, reducing the number of application services that need to be coded, and thus reducing resource consumption.

[0051] It should be noted that the area to be recognized can be a partial area selected by those skilled in the art from all areas corresponding to the user interface according to actual needs, or the area to be recognized can be determined according to the user operation area, so as to improve the accuracy of the area to be recognized. The embodiments of the present application do not limit this.

[0052] In the embodiments of the present application, for any application recommendation process, when obtaining the screen element information corresponding to the user interface, the above-mentioned control information recognition is required. However, since there is no image and / or text in some user interfaces, therefore, when performing parsing, it is not necessarily required to perform text content recognition and image content recognition, that is, the user interface does not necessarily include textual knowledge, visual knowledge, and the current application service set. Therefore, including at least one of textual knowledge, visual knowledge, and the current application service set, as well as structural knowledge as the screen element information improves the richness and accuracy of the screen element information.

[0053] In some embodiments, when performing control information recognition on the user interface or the area to be recognized to obtain structural knowledge, the following steps may be included: obtaining the structural information of the control tree corresponding to the user interface or the area to be recognized through the accessibility service interface; performing control information recognition on the structural information of the control tree to obtain structural knowledge.

[0054] In the embodiments of the present application, the Accessibility Service is a set of system-level application programming interfaces (APIs) that can simulate operations. After the user agrees to grant the application the permission to access the Accessibility Service, the application can simulate operations and sequentially control the user's application recommendation device. The Accessibility Service can be used in applications such as red packet grabbing, automatic reply, and one-key permission acquisition to achieve one-key operations. Exemplarily, the method for identifying control information can adopt the Accessibility Service interface provided by the operating system to obtain the structural information of the entire control tree corresponding to the user interface or the area to be recognized. When the user interface or the area to be recognized is refreshed, the updated structural information of the entire control tree of the interface can be obtained by calling the Accessibility Service interface. Then, based on the structural information of the entire control tree, the control information is identified from the structural information of the control tree through topology technology to obtain structural knowledge.

[0055] In the embodiments of the present application, the structural information of the control tree corresponding to the user interface or the area to be recognized is obtained through the Accessibility Service interface; the control information is identified from the structural information of the control tree to obtain structural knowledge, improving the accuracy of the structural knowledge.

[0056] S203. Based on the touch operation information and the screen element information, obtain the association degree between the screen element information and multiple applications, where the multiple applications are of the same type.

[0057] In the embodiments of the present application, the application recommendation device is to meet the user's immediate service requirements. Therefore, it is necessary to obtain the touch operation information of the user interface and the screen element information corresponding to the user interface. The screen element information and the touch action information reflect strong intentions related to the user's immediate needs and can be understood as user-related immediate information. After obtaining the immediate information, it is necessary to calculate the association degree between the screen element information and multiple applications.

[0058] In the embodiments of the present application, the multiple applications can be of the same type. Applications of the same type can be understood as those whose corresponding function manifestation forms are of the same type. For example, multiple different playback software for playing images or videos for users are of the same type of applications; multiple different shopping software for shopping are of the same type of applications; multiple different social software for socializing are of the same type of applications.

[0059] Exemplarily, multiple applications can be determined based on the touch operation information and the screen element information. The multiple applications are of the same type, and this type of applications can all support the operation content in the touch operation information and the screen element information in the current user interface. Then, collaborative joint learning and similarity processing are performed between the screen element information and the multiple applications to obtain multiple association degrees.

[0060] In the embodiments of the present application, collaborative joint learning is a process of fusing the implicit information of screen elements and the implicit information of touch actions, and similarity processing is a process of combining the fused information with collaborative filtering. Taking product recommendation as an example, the collaborative filtering algorithm discovers the user's preference bias based on the mining of the user's historical behavior data, and predicts the products that the user may like for recommendation. It can be understood as functions such as "Guess You Like" and "People Who Bought This Item Also Like". It can be achieved in the following ways: recommending to you based on people who have the same preferences as you, recommending similar items based on the items you like, and making comprehensive recommendations based on the above conditions.

[0061] In some embodiments, the application includes: application type, and / or, application service.

[0062] In the embodiments of the present application, there are many forms of expression of the application. The application can be an application type, an application service, or a combination of an application type and an application service. The application type can be understood as the type of application software, such as Figure 1 the application suggestions listed in [application suggestions] and various different application services of popular applications. The application service can be understood as displaying text or images in the user interface in the application software. For example, opening image, video and other files in the playback software, opening the shopping link in the shopping software, opening a normal link in the browser, making a call to other users in the dialing software, and opening a document in the office software. Different users will produce different selection results when facing each piece of interaction data. For example, selecting different types of application software; selecting to open an image or text in the application software; or, first selecting different types of application software and then opening an image or text. Correspondingly, the application recommendation result can also be a combination of an application type and an application service. The application recommendation in the embodiments of the present application can also be understood as service recommendation, application service recommendation, application software recommendation, etc., and the embodiments of the present application do not limit this.

[0063] In the embodiments of the present application, the application or the application recommendation result includes an application type, and / or, an application service, which improves the richness of the application recommendation result.

[0064] In the embodiments of the present application, the touch operation information and the screen element information are related to the current user interface and can meet the immediate needs of the user. Based on the touch operation information and the screen element information, multiple applications of the same type are determined. Compared with directly obtaining the association degree between the screen element information and all applications, obtaining the association degree between the screen element information and multiple applications of the same type reduces resource consumption and improves the accuracy of the association degree.

[0065] S204. Recommend and display applications whose relevance meets the preset conditions.

[0066] In the embodiments of the present application, among the multiple relevance degrees obtained in S203, the applications corresponding to the relevance degrees that meet the preset conditions are used as the application recommendation results, and these applications are recommended and displayed. Exemplarily, first display to the user a flag indicating whether to open the application. This flag can be shown in the form of a window, text, picture, folder, etc. Taking the window as an example, if the user clicks on the window, then display the applications whose relevance meets the preset conditions. It is also possible to directly display to the user the applications whose relevance meets the preset conditions, and the user can directly select the applications.

[0067] In the embodiments of the present application, the preset conditions can be appropriately set by those skilled in the art according to actual needs. For example, if the preset condition is that the relevance degree is greater than a preset threshold, then recommend one or more applications to the user; if the preset condition is the maximum relevance degree, then recommend one application to the user. The embodiments of the present application do not limit this.

[0068] In some embodiments, the above S204 can be implemented in the following manner. On the user interface, recommend and display the application with the highest similarity to each application; or, on the user interface, recommend and display a preset number of applications with the highest similarity to each application.

[0069] In the embodiments of the present application, recommend and display the application with the highest similarity to the user interface. Exemplarily, directly enter this application, and it is possible to jump to this application without the user's further operation, improving the application recommendation efficiency. Or, recommend this application to the user on the user interface, and the user only needs to select whether to open or enter this application, without the user having to select among multiple applications, improving the application recommendation efficiency.

[0070] In the embodiments of the present application, recommend and display a preset number of applications with the highest similarity to each application, recommend a preset number of applications to the user, and the user can select the applications that are strongly relevant to their own needs, improving the diversity of application recommendations. It should be noted that the preset number can be appropriately set by those skilled in the art according to the actual situation, such as 2, 3, 4, etc. The embodiments of the present application do not limit this.

[0071] According to the solution provided by the embodiments of the present application, obtain touch operation information acting on the user interface; based on the touch operation information, obtain screen element information corresponding to the user interface; both the screen element information and the touch operation information are interaction data related to the user's current operation and can meet the user's immediate service needs. Based on the touch operation information and the screen element information, obtain the association degree between the screen element information and multiple applications, where the multiple applications are of the same type; recommend and display the applications whose association degree meets the preset conditions. By combining the dual-modal information of the screen element information and the touch operation information, the embodiments of the present application consider the interaction data of the user interface from two dimensions, and then combine the association degree with multiple applications, improving the accuracy of the application recommendation results.

[0072] In some embodiments, the touch operation information includes the operation content; obtain the screen element information corresponding to the user interface; the above S202 can be implemented through the following two examples. The first example, if the touch operation information indicates that its operation content meets the preset trigger recommendation condition, obtain the screen element information of the entire interface corresponding to the user interface; the preset trigger recommendation condition indicates the expected operation intention.

[0073] In the embodiments of the present application, during the process of the user interacting with the application recommendation device in real time, the application recommendation device does not always recommend applications to the user. If the touch operation information indicates that its operation content does not reach the expected operation intention, indicating that the user has no strong intention, at this time, no application recommendation will be made to the user, and it is not necessary to obtain the screen element information of the entire interface corresponding to the user interface. If the touch operation information indicates that its operation content reaches the expected operation intention, indicating that the user has a strong intention such as wanting to super share, forward, or open, then obtain the screen element information of the entire interface corresponding to the user interface.

[0074] It should be noted that the entire interface corresponding to the user interface can be understood as the entire area of the user interface, and the preset trigger recommendation condition can be appropriately set by those skilled in the art according to actual needs, as long as it can distinguish whether the user has a strong intention.

[0075] In some embodiments, the preset trigger recommendation condition indicates the expected operation intention. For example, the preset trigger condition can be a preset operation content, and the preset operation content can be copy, forward, share, drag, reminder, save, or favorite, that is, the preset trigger recommendation condition can be at least one of the following: copy, forward, share, drag, reminder, save, and favorite, but the embodiments of the present application do not limit this. When the operation content of the touch operation information or its corresponding function meets the preset trigger recommendation condition, obtain the screen element information of the entire interface corresponding to the user interface.

[0076] For example, exemplarily, the preset trigger condition is "copy" in the operation content. The user's touch operation on a certain link is "long press", and then the operating system pops up a sub-menu. The user's operation content is to select "copy" in the sub-menu, indicating that the user has a strong intention to represent super sharing, forwarding, or opening, etc., which shows that the user's operation content meets the preset trigger recommendation condition.

[0077] In the embodiments of the present application, by setting the preset trigger condition, compared with the method of directly obtaining the screen element information of the entire interface corresponding to the user interface, the resource consumption is reduced.

[0078] In some embodiments, the above S202 is implemented through the following second example. Second example, at the touch position where the touch operation information acts, determine the current area to be recognized; recognize the current area to be recognized to determine the screen element information.

[0079] In the embodiments of the present application, the screen element information is information about each different type of element displayed in the user interface, and the touch position where the touch operation information acts is a part of the user interface, that is, the user's touch operation is not directed at the entire area of the user interface. Therefore, according to the touch position where the touch operation information acts, determine the current area to be recognized. The area of the current area to be recognized is smaller than the entire area of the user interface, and then recognize the current area to be recognized to determine the screen element information in the area to be recognized.

[0080] In the embodiments of the present application, determine the current area to be recognized according to the touch position where the touch operation information acts, and then recognize the current area to be recognized to determine the screen element information, without obtaining the screen element information of the entire area of the user interface, reducing the amount of data processing and improving the application recommendation efficiency.

[0081] In some embodiments, in the above second example, after determining the current area to be recognized at the touch position where the touch operation information acts, the application recommendation method further includes the following steps: if no screen element information is recognized in the current area to be recognized, determine the next area to be recognized, and recognize in the next area to be recognized until screen element information exists or the entire interface is recognized; the next area to be recognized is larger than the current area to be recognized.

[0082] In the embodiment of the present application, with the touch position where the touch operation information acts as the center, the area within the preset range is determined as the current area to be recognized. If no screen element information is recognized in the current area to be recognized, it indicates that the area of the area to be recognized needs to be further enlarged. For example, the preset range is enlarged, or, based on the current area to be recognized, it is extended in a certain direction or in all directions to determine the next area to be recognized, and the next area to be recognized is larger than the current area to be recognized. Then, recognition is performed in the next area to be recognized until screen element information exists or the entire interface is recognized.

[0083] In the embodiment of the present application, according to the result of whether screen element information is recognized, it is judged whether the range of the area to be recognized needs to be enlarged until screen element information is obtained. By gradually determining the area to be recognized, the accuracy of the area to be recognized is improved. After screen element information is obtained, the area to be recognized is no longer enlarged. Compared with the method of directly recognizing all areas of the user interface to obtain screen element information, resource consumption is reduced.

[0084] In some embodiments, the touch operation information includes: touch operation and operation content; the above S203 can be implemented in the following manner. In response to the selection of the operation content, the association degree between the screen element information corresponding to the user interface and multiple applications is obtained.

[0085] In the embodiment of the present application, after the application recommendation device displays the corresponding operation content to the user, the user selects the operation content to obtain the operation content. In response to the selection of the operation content, the association degree between the screen element information corresponding to the user interface and multiple applications is obtained. It can be understood that the application recommendation method is triggered by the operation content. After the user selects the operation content, according to the operation content, touch operation, and screen element information, multiple applications are determined. The multiple applications are applications that can support the touch operation information and screen element information in the current user interface, so as to obtain the association degree between the screen element information and multiple applications. Compared with directly obtaining the association degree between the screen element information and all applications, resource consumption is reduced and the accuracy of the association degree is improved.

[0086] In some embodiments, the above S203 can also be implemented in the following manner. Based on the screen element information and touch operation information, the association is made with the preset sample fusion information and the sample sequence behavior information of each preset application on each application, so as to obtain the association degree between the screen element information and each application of multiple applications.

[0087] In the embodiments of the present application, the preset sample fusion information and the sample sequence behavior information of each preset application can be understood as the preset training data. The preset training data is the data generated by the user's interaction with the mobile phone before application recommendation, and can be used as a reference for this application recommendation. The preset training data includes multiple training samples. After performing knowledge base embedding and collaborative joint learning on the preset training data, the preset sample fusion information is obtained. The preset training data also includes the sample service information of multiple training samples. There will be the same application among the multiple sample service information. The preference probability of each application is predicted based on the multiple sample service information, so as to obtain the sample sequence behavior information of each preset application. A training sample includes a sample fusion information and the sample service information corresponding to the sample fusion information. The sample service information represents the application selected by the user for each subsequent interaction data (the interaction data corresponds to the sample fusion information) in the training sample. The sample service information can be understood as the application true value.

[0088] In the embodiments of the present application, based on multiple sample fusion information and the sample service information corresponding to each sample fusion information, the sample sequence behavior information of each application is obtained. The sample sequence behavior information of each application represents the preference probability of the user for each application for different sample fusion information. Exemplarily, the sample sequence behavior information of the j-th application for the i-th sample fusion information can also be understood as the preference probability of the i-th sample fusion information for the j-th application. The preference probability is determined by the number of sample service information corresponding to the sample fusion information in the training data, and can also be understood as the preference probability of the user for each application under the same sample fusion information. The higher the preference probability, the greater the value of the sample sequence behavior information. For example, assuming that there are only two applications, that is, j = 1 or 2, and the first application is selected 3 times and the second application is selected 1 time for the i-th interaction data, then the sample sequence behavior information of the i-th sample fusion information for the 1st application is 3 / (1 + 3) = 0.75. Correspondingly, the sample sequence behavior information of the i-th sample fusion information for the 2nd application is 0.25. Performing the above calculations on all sample fusion information in the fusion information set can obtain the sample sequence behavior information of each application.

[0089] In the embodiments of the present application, knowledge base embedding and collaborative joint learning are performed based on the screen element information and the touch operation information to obtain the fusion data to be analyzed, and then collaborative joint learning and correlation processing are performed with the preset sample fusion information and the sample sequence behavior information of each preset application on each application, so as to obtain the association degree between the screen element information and each application of multiple applications, improving the accuracy of the association degree.

[0090] In some embodiments, before the above S203, the application recommendation method further includes the following steps: determining, according to preset training data, sample screen element hidden information, sample touch action hidden information, and sample service information corresponding to each training sample; performing collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain preset sample fusion information for each training sample; and determining preset sample sequence behavior information for each application according to the sample service information of each training sample.

[0091] In an embodiment of the present application, the preset training data includes a plurality of training samples, and one training sample includes sample screen element information, sample touch action information, and sample service information. By encoding the sample screen element information and the sample touch action information respectively, sample screen element hidden information and sample touch action hidden information are obtained.

[0092] In an embodiment of the present application, for the sample screen element hidden information and the sample touch action hidden information of each training sample, they are respectively parsed and encoded through a knowledge base embedding method to obtain the sample screen element hidden information and the sample touch action hidden information, and then through collaborative joint learning, the sample screen element hidden information and the sample touch action hidden information are fused to obtain sample fusion information.

[0093] In an embodiment of the present application, according to the sample fusion information of a plurality of training samples and the selection results of applications in the plurality of sample service information, the number of times (or probabilities) of selection of each application is obtained, so as to determine the sample sequence behavior information of each application.

[0094] In an embodiment of the present application, according to the preset training data, the sample fusion information of each training sample and the sample sequence behavior information of each application are determined, so as to perform collaborative joint learning and similarity processing with the screen element information and the touch operation information corresponding to the user interface, thereby obtaining the association degree of each application.

[0095] In some embodiments, based on the screen element information and the touch operation information, associating with the preset sample fusion information and the preset sample sequence behavior information of each application on each application, so as to obtain the association degree of the screen element information with each application of the multiple applications, may include S301 - S303. As Figure 3 shown, Figure 3 is an optional step flowchart of another application recommendation method provided by an embodiment of the present application.

[0096] S301. Encoding the screen element information to obtain screen element hidden information.

[0097] In the embodiments of the present application, an auto encoder can be used to encode screen element information to obtain screen element hidden information. An auto encoder is an artificial neural network that can learn an efficient representation of input data through unsupervised learning. This efficient representation of the input data is called codings, and its dimension is generally much smaller than that of the input data, enabling the auto encoder to be used for dimensionality reduction. The auto encoder in the embodiments of the present application can be an encoder of any form, and there is no limitation on the structure of the adopted auto encoder, as long as it can encode screen element information and touch operation information respectively, including but not limited to vanilla auto encoder, multi-layer auto encoder, convolutional auto encoder, and regular auto encoder. Among them, the regular auto encoder includes a sparse auto encoder and a denoising auto encoder, and the denoising auto encoder can be a stacked denoising auto-encoders (SDAE), and the convolutional auto encoder can be a stacked convolutional auto-encoders (SCAE).

[0098] It should be noted that an auto encoder generally includes two parts: the encoder (also called the recognition network) converts the input into an internal representation, and the decoder (also called the generation network) converts the internal representation into an output. The output is an attempt to reconstruct the input, and the loss function is the reconstruction loss. During the training process, the encoding part of the auto encoder (i.e., the encoder) is used to encode the information of the original image to obtain a high-dimensional vector, and then the decoding part of the auto encoder (i.e., the decoder) is used to decode the high-dimensional vector to obtain the original image. During the application process, the encoding part of the trained auto encoder (i.e., the encoder) is used to encode the image to be encoded to obtain a high-dimensional vector, and the high-dimensional vector can include all the information of the image to be encoded.

[0099] S302. Encode the touch operation information to obtain touch action hidden information.

[0100] In the embodiments of the present application, a common encoding method can be used to encode the touch operation information to obtain touch action hidden information. There is no limitation on the encoding method in the embodiments of the present application, as long as the encoding method can distinguish which touch operations the user has performed.

[0101] Exemplarily, the 0-1 encoding method is used for encoding, where 1 represents that the user is currently performing this touch operation, and 0 represents that the user is not currently performing this touch operation. In the embodiments of the present application, other forms of encoding methods can also be used to distinguish between performing and not performing this touch operation, and there is no limitation in this regard in the embodiments of the present application.

[0102] S303. Based on the hidden information of screen elements, the hidden information of touch actions, the sample fusion information, and the sample sequence behavior information of each application, perform the association of each application to obtain the association degree of each application.

[0103] In the embodiments of the present application, since the screen element information is data related to the user interface and the touch operation information is interaction data related to the user's current operation, therefore, by using different coding forms to code the dual-modal information (the hidden information of screen elements and the hidden information of touch actions), the obtained hidden information of screen elements and the hidden information of touch actions are both interaction data related to the user's current operation, which can meet the user's immediate service needs. Based on the hidden information of screen elements, the hidden information of touch actions, the sample fusion information, and the sample sequence behavior information of each application, perform the association of each application to obtain the association degree of each application, improving the accuracy of each association degree.

[0104] In some embodiments, the screen element information includes: the structural knowledge corresponding to the user interface; the above S301 can be implemented in the following manner. Code the structural knowledge to obtain the hidden information of screen elements.

[0105] Since there are no images and texts in some user interfaces, but there must be structural knowledge in the user interface, therefore, in the embodiments of the present application, code the structural knowledge to obtain the structural information, and use the structural information as the hidden information of screen elements. Exemplarily, taking the case where the coded structural information is in the form of a structural vector as an example, for the structural knowledge, use the Translating Relationship (TransR) method to transform the structural knowledge into a structural vector, and the TransR model can represent the characteristics of the structure type.

[0106] In the embodiments of the present application, coding the structural knowledge to obtain the hidden information of screen elements improves the accuracy of the hidden information of screen elements.

[0107] In some embodiments, the screen element information includes: the structural knowledge corresponding to the user interface, the text knowledge, the visual knowledge, and the set of current application services related to the user interface; the set of current application services is determined by performing service parsing on the user interface. The above Figure 3 S301 may include S3011 - S3013, as Figure 4 shown. Figure 4 This is an optional step flowchart of another application recommendation method provided by the embodiments of the present application.

[0108] S3011. Code the structural knowledge, the text knowledge, and the visual knowledge respectively to obtain the structural information, the text information, and the visual information.

[0109] In the embodiments of the present application, visual knowledge, text-based knowledge, and structural knowledge need to be converted into encoded information through visual embedding, text embedding, and structural embedding respectively. The data embedding method is used to determine the hidden information of the screen elements. Exemplarily, the data embedding method can be the Collaborative Knowledge base Embedding (CKE) algorithm. Among them, the CKE algorithm includes knowledge base embedding and collaborative joint learning. Among them, knowledge base embedding can also be understood as knowledge graph embedding. The knowledge base embedding method is used to determine the hidden information of the screen elements. After obtaining the hidden information of the touch action subsequently, collaborative joint learning can be used to fuse the hidden information of the screen elements and the hidden information of the touch action to obtain the fusion information to be analyzed. The encoding processes of visual information, text information, and structural information are described below respectively. Among them, encoding the structural knowledge to obtain structural information is as specifically described above and will not be elaborated here.

[0110] Exemplarily, taking the case where the encoded visual information is in the form of a visual vector as an example, for visual knowledge, the embedding method that can be used is the Stacked Convolutional Autoencoder (SCAE). During the training process of the SCAE, through the stacked convolutional neural network, the information of the original image is mapped (encoded) into a high-dimensional vector, and then the original image is reconstructed (decoded) using the high-dimensional vector. Therefore, the high-dimensional vector contains all the information of the original image. During the application process, the encoding part of the trained SCAE is used to encode the visual knowledge to obtain a high-dimensional vector. This high-dimensional vector is concatenated with the encoded vector of the image source analysis result to obtain a visual vector.

[0111] Exemplarily, taking the case where the encoded text information is in the form of a text vector as an example, for text-based knowledge, the embedding method that can be used is the Stacked Denoising Autoencoder (SDAE). The idea of the SDAE is similar to that of the SCAE and will not be elaborated here. During the application process, the encoding part of the trained SDAE is used to encode the text-based knowledge to obtain a high-dimensional vector, and this high-dimensional vector is concatenated with the encoded vector of the additional information of the text to obtain a text vector.

[0112] It should be noted that autoencoders with any structure can be used to encode structural knowledge and text-based knowledge. The above examples are only for illustrative purposes with SDAE and SCAE as examples, and do not represent that the embodiments of the present application are limited thereto.

[0113] In some embodiments, the structural knowledge in S3011 above includes at least one of control type, control function, and control structure distribution; the text-based knowledge includes at least one of character recognition information and special symbol recognition information, and the original text information; the visual knowledge includes at least one of image content recognition information and image source information, and the original image information.

[0114] In the embodiments of the present application, control information recognition obtains the types of controls, control functions, control structure distributions, etc. on the user interface. Exemplarily, the types of controls include buttons, texts, labels, and image points, etc. Taking video selection as an example, the control structure distribution includes the distribution of selection buttons.

[0115] In the embodiments of the present application, text content recognition obtains character recognition information, link recognition information, special symbol recognition information, etc. Moreover, for text containing additional information, for example, a sharing link from a certain application software, its additional information will also be encoded into the recognition result. For example, the additional information corresponding to a normal website and a shopping link is different. Therefore, compared with the structural knowledge, the text-based knowledge also includes the original text information, which can be understood as the information of the text itself, and can not only reflect characters and special symbols, but also reflect additional information such as links.

[0116] In the embodiments of the present application, image content recognition is the analysis and description of the image content (for example, the description of the content of the image itself), and the image content label (for example, identifying which specific tourist attraction the scenic photo in the image is, whether the image is an emoji, etc.). Image content recognition can also include image source analysis, and the image source analysis will analyze the source of the image according to the style data of the image (for example, analyzing whether the image comes from the dialing interface, analyzing whether the image is a screenshot from the interface of a certain application software). Therefore, compared with the structural knowledge, the visual knowledge also includes the original image information, which can be understood as the information of the image itself, and can not only reflect the image content and image label, but also reflect the original information such as the image source.

[0117] It should be noted that the structural knowledge only includes the recognized structural data. That is to say, the structural knowledge does not include the substantial content related to user interaction in the user interface, while the text-based knowledge includes both the recognized character recognition information and special symbol recognition information, and the original text information, and the visual knowledge includes both the recognized image content recognition information and image source information, and the original image information.

[0118] In the embodiments of the present application, the structural knowledge includes at least one of control types, control functions, and control structure distributions; the text-based knowledge includes at least one of character recognition information and special symbol recognition information, as well as the original text information; the visual knowledge includes at least one of image content recognition information and image source information, as well as the original image information, improving the richness of the structural knowledge, text-based knowledge, and visual knowledge.

[0119] In some embodiments, Figure 4 S3011 may include the following steps: performing vector quantization encoding on the structural knowledge to obtain structural information; performing vector quantization encoding on at least one of the character recognition information and special symbol recognition information to obtain text content understanding encoding information; and performing stacked auto-encoding on the original text information to obtain high-dimensional text information; splicing the high-dimensional text information and the text content understanding encoding information to obtain text information; performing information encoding on at least one of the image content recognition information and image source information to obtain image content understanding encoding information; and performing stacked auto-encoding on the original image information to obtain high-dimensional image information; splicing the high-dimensional image information and the image content understanding encoding information to obtain visual information.

[0120] In the embodiments of the present application, since the structural knowledge includes at least one of control types, control functions, and control structure distributions and does not have any substantial content related to user interaction, when encoding the structural knowledge, performing vector quantization encoding on the structural knowledge can obtain the structural information, improving the encoding efficiency of the structural information.

[0121] In the embodiments of the present application, since the text-based knowledge includes at least one of the character recognition information and special symbol recognition information, as well as the original text information, when encoding the text-based knowledge, it is necessary to perform vector quantization encoding on at least one of the character recognition information and special symbol recognition information to obtain text content understanding encoding information; and perform stacked auto-encoding on the original text information to obtain high-dimensional text information, which can reflect the original text information. Splicing the high-dimensional text information and the text content understanding encoding information to obtain text information improves the accuracy of the text information.

[0122] In the embodiments of the present application, since the visual knowledge includes at least one of the image content recognition information and image source information, as well as the original image information, when encoding the visual knowledge, it is necessary to perform information encoding on at least one of the image content recognition information and image source information to obtain image content understanding encoding information; and perform stacked auto-encoding on the original image information to obtain high-dimensional image information, which can reflect the original image information. Splicing the high-dimensional image information and the image content understanding encoding information to obtain visual information improves the accuracy of the visual information.

[0123] S3012. Encode the current application service set to obtain service offset information.

[0124] In the embodiment of the present application, since the data to be encoded for the current application service is fixed, the update frequency is low, and the variability is usually small. Therefore, the current application service set is directly and simply encoded to obtain service offset information.

[0125] Exemplarily, since there are multiple application services included in the application service set, not all application services are related to the current user interface. For example, the current user interface is an A chat interface. In the A chat interface, for a shopping link sent by the other party, if the user's touch operation is "long press" and the operation content is "copy", the application service set related to the current A chat interface may include another B application software (for sharing the link in another social software), C application software (for opening the shopping link to understand the detailed information of the items in the link), and D application software (for querying in the website). In the embodiment of the present application, by performing service parsing on the current user interface to determine the current application service set, it can be understood as selecting the application services related to the current user interface as the current application service set, reducing the number of application services to be encoded, thereby reducing resource consumption. Since the data to be encoded for the current application service is fixed and the update frequency is low, for example, a certain browser supports opening and parsing a certain type of link. Therefore, the current application service set is directly and simply encoded to obtain service offset information. Different numbers are used to distinguish the application services. Exemplarily, taking the service offset information as a vector expression form as an example, the application service A is encoded to obtain the service offset vector 0001, and the application service B is encoded to obtain the service offset vector 0010. The service offset information and the screen element encoding information are spliced to obtain the screen element hidden information.

[0126] S3013. Splice at least one of the text information, visual information, and service offset information, and the structure information to obtain the screen element hidden information.

[0127] In the embodiments of the present application, since there is no image and text in some user interfaces, that is, the screen element information does not include text information and visual information; and there is no set of application services related to the user interface in some user interfaces, that is, the current application service set is empty. It can be understood that the user interface includes structural information; or, the user interface includes structural information, and at least one of text information, visual information, and service bias information. Therefore, when obtaining the hidden information of the screen element, exemplarily, it can be the splicing of at least one of text information, visual information, and service bias information, and the structural information to obtain the hidden information of the screen element. It can also be the splicing of structural information, text information, visual information, and service bias information to obtain the hidden information of the screen element; the embodiments of the present application do not limit this. Among them, when splicing structural information, text information, visual information, and service bias information to obtain the hidden information of the screen element, if there is no image, text, and the corresponding current application service set in the user interface, the text-based knowledge, visual-based knowledge, and the current application service set are still encoded respectively, and the encoding result is information representing no substantial content, which can be understood that the text information, visual information, and service bias information all represent emptiness. Since the sample fusion information in the preset training data is obtained by splicing the encoded structural information, text information, visual information, and service bias information respectively, therefore, the hidden information of the screen element obtained by processing the data to be analyzed through the same processing method as the training data can ensure that the dimension is consistent with the sample fusion information, and improve the accuracy of the calculation result when calculating the similarity in the subsequent process.

[0128] In the embodiments of the present application, the structural knowledge, text-based knowledge, visual-based knowledge, and the current application service set are encoded respectively, and the encoded information is spliced to obtain the hidden information of the screen element, which improves the accuracy of the hidden information of the screen element.

[0129] Next, through the screen element hidden vector acquisition module, the exemplary application of the embodiments of the present application in an actual application scenario will be described. It is described by taking the expression forms of the hidden information of the screen element, the hidden information of the touch action, the service bias information, the structural information, the visual information, and the text information as vectors, and the screen element information is the mobile phone screen element information as an example.

[0130] In the embodiments of the present application, the screen element hidden vector acquisition module is used to obtain the vector expression form of the mobile phone screen element of the user interface, and the vector expression form here can be regarded as a vector embedding expression based on the knowledge graph. As Figure 5 shown, Figure 5 is an optional flowchart for calculating the hidden vector of the screen element provided by the embodiments of the present application. Figure 5 It is described by taking the application recommendation device as a mobile phone as an example. Figure 5The data set for calculating the hidden vector of screen elements includes the screen element information corresponding to the user interface of the current mobile phone screen and the set of application services that the user interface of the current mobile phone can provide.

[0131] In the embodiments of the present application, for the mobile phone screen element information, it is parsed through the following three dimensions: control information recognition, text content recognition, and image content recognition. Figure 5 Among them, the tree - structured knowledge obtained by control information recognition is represented by structural knowledge, the information recognized by text content is represented by text - type knowledge, and the information recognized by image content is represented by visual - type knowledge.

[0132] It should be noted that the above - mentioned parsing and recognition of mobile phone screen element information are triggered by the user operation content in the user touch action. When the user touch action meets the preset trigger condition, the above - mentioned parsing and recognition steps of mobile phone screen element information are triggered.

[0133] In the embodiments of the present application, both control information, image content, and text content are recognized and encoded. If there is no image and / or text in the user interface, the encoding vector corresponding to the image content and / or text content is a vector representing no content, which can be understood as an empty encoding vector. For example, the encoding vector is 0000. Before understanding the text content and image content, it can be first determined whether there is an image and / or text in the user interface. If there is an image and / or text, then the mobile phone screen element information is subjected to text content understanding and / or image content understanding.

[0134] In the embodiments of the present application, Figure 5 Among them, the visual - type knowledge, text - type knowledge, and structural knowledge need to be converted into encoding vectors through visual embedding, text embedding, and structural embedding respectively. The encoding vectors include visual vectors, text vectors, and structural vectors. And simply encode the current application service set to obtain a service bias vector. For example, different numbers can be used to distinguish application services. The service bias vector, visual vector, text vector, and structural vector are concatenated to obtain the hidden vector of the screen element.

[0135] In some embodiments, the touch operation information includes: operation content and touch operation; when encoding the touch operation information to obtain the hidden information of the touch action, it can be achieved through the following method. The touch operation and the operation content are encoded respectively to obtain touch operation information and operation content information; the touch operation information and the operation content information are concatenated to obtain the hidden information of the touch action.

[0136] In the embodiments of the present application, ordinary encoding methods can be used to encode the operation content and touch operations, as long as it is possible to distinguish which operations the user has performed and which gestures.

[0137] Exemplarily, touch operations include, but are not limited to, clicking, double-clicking, long-pressing, two-finger pinching, two-finger zooming, swiping, and dragging; using a 0-1 encoding method for encoding, 1 represents that the user has performed this gesture currently, and 0 represents that the user has not performed this gesture currently. In the embodiments of the present application, other forms of encoding methods can also be used to distinguish whether this gesture has been made. The embodiments of the present application do not limit this.

[0138] Exemplarily, operation content includes, but is not limited to, copying, forwarding, sharing, dragging, reminding, saving, favoriting, and staying, etc. Using a 0-1 encoding method for encoding, 1 represents that the user has selected this operation currently, and 0 represents that the user has not selected this operation currently. In the embodiments of the present application, other forms of encoding methods can also be used to distinguish whether this operation has been selected. The embodiments of the present application do not limit this.

[0139] In the embodiments of the present application, after encoding the operation content and touch operations to obtain operation content information and touch operation information, a splicing method is adopted to splice the operation content information and touch operation information to obtain touch action hidden information, which improves the accuracy of the touch action hidden information.

[0140] Next, through the touch action hidden vector acquisition module, an exemplary application of the embodiments of the present application in an actual application scenario will be described. Taking the expression form of the touch action hidden information as a vector and the touch operation information as the user's touch action as an example for description. In the embodiments of the present application, the touch action hidden vector acquisition module is used to obtain the vector expression form of the user's touch action. The user's touch action includes the user's operation content and the user's touch operation, as Figure 6 shown, Figure 6 is an optional flowchart for calculating the touch action hidden vector provided by the embodiments of the present application. Figure 6 In it, the touch gesture represents a touch operation. The operation gestures supported by the user's touch gesture include, but are not limited to, clicking, double-clicking, long-pressing, two-finger pinching, two-finger zooming, swiping, and dragging; the user operations supported by the user's operation content include, but are not limited to, copying, forwarding, sharing, dragging, reminding, saving, favoriting, and staying, etc. After encoding the user's operation content and the user's touch gesture to obtain the user's operation content vector and the user's touch gesture vector, a vector splicing method is adopted to splice the user's operation content vector and the user's touch gesture vector to obtain the touch action hidden vector.

[0141] In some embodiments, S303 in the above Figure 3 may include S3031 - S3033. AsFigure 7 As shown Figure 7 FIG. is an alternative step flowchart of another application recommendation method provided by an embodiment of the present application.

[0142] S3031. Coordinately jointly learn the hidden information of screen elements and the hidden information of touch actions to obtain the fusion information to be analyzed.

[0143] In the embodiment of the present application, the coordinated joint learning is a process of fusing the hidden information of screen elements and the hidden information of touch actions to obtain the fusion information to be analyzed. The main function of the coordinated joint learning is feature mapping and feature dimensionality reduction. By using the coordinated joint learning method, the hidden information of touch actions and the hidden information of screen elements are used as inputs, and the fusion information representing the user's intention, that is, the fusion information to be analyzed, is output.

[0144] Exemplarily, the process of coordinated joint learning can be understood as a process of realizing hidden information fusion by using a coordinated joint model. The hidden information of screen elements and the hidden information of touch actions are input into the trained coordinated joint model, and the fusion information is output. The output fusion information is used as the fusion information to be analyzed. The coordinated joint model in the embodiment of the present application can be any form of network, and there is no limitation on the structure of the coordinated joint model used, as long as it can fuse the hidden information of screen elements and the hidden information of touch actions to obtain the correct representation of the fused information. Including but not limited to a mapping structure of a fully convolutional network (FCN).

[0145] S3032. Obtain the historical sequence behavior information of each application determined based on historical behavior data.

[0146] In the embodiment of the present application, the historical behavior data is the historical behavior data of the user, which can reflect the preference probability of the user for each application. For each application, the historical sequence behavior information of each application is obtained by combining the historical behavior data.

[0147] Exemplarily, taking m applications as an example, for the kth application, the historical sequence behavior information is generated according to the user's historical behavior data, denoted as w k , and thus m historical sequence behavior information can be generated. The dimension of the historical sequence behavior information w k is 1×m.

[0148] In some embodiments Figure 7 S3032 in can be implemented in the following manner. Based on the historical behavior data, determine the historical behavior parameters of multiple applications; for each application, set the historical behavior parameters corresponding to the application as preset parameters, and combine the historical behavior parameters of the multiple applications except the application to obtain the historical sequence behavior information of the application.

[0149] Exemplarily, take the case where there are m applications, the historical behavior parameters corresponding to the applications are preference probabilities, and the preset parameter is a preset probability. If the user is a new user and there is no historical behavior data of this user, and thus no preference record of this user, then the preference probabilities of the m applications in the historical sequence behavior information are all equal. If the user is not a new user, then calculate the preference probabilities of the m applications based on the historical behavior data, that is, calculate the probabilities of the user selecting each application in the historical behavior data. For the k-th application among the m applications, set the preference probability of the k-th application to the preset probability, and combine the preference probabilities of the remaining m - 1 applications to determine the historical sequence behavior information w of the k-th application k , that is, for the k-th application, only set the preference probability of the k-th application among the m applications to the preset probability, and keep the other m - 1 preference probabilities unchanged, so as to form the historical sequence behavior information of the k-th application. Among them, the preset parameter can be appropriately set by those skilled in the art according to actual needs, as long as it can effectively calculate the recommendation score of this application. By analogy, m historical sequence behavior information can be obtained

[0150] In the embodiments of the present application, for each application, set the behavior data corresponding to this application as the preset parameter, and combine the behavior data except this application in the historical behavior data to obtain the historical sequence behavior information of this application, which is convenient for subsequent calculation of similarity using the historical sequence behavior information of this application and improves the accuracy of similarity

[0151] S3033. Perform similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the similarity of each application

[0152] In the embodiments of the present application, perform similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample. Since the sample fusion information and the sample sequence behavior information in the training samples are in one-to-one correspondence, according to the above two similarity processes, the similarity of each application can be obtained, improving the accuracy of similarity

[0153] In some embodiments Figure 7S3033 may include the following steps: performing similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application to determine the first similarity information of each application for each training sample; performing similarity processing on the information to be analyzed and fused with the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; and obtaining the similarity of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample.

[0154] Exemplarily, taking m applications as an example, for the k-th application among the m applications, determine the sample sequence behavior information of each training sample in the sample sequence behavior information, calculate the similarity between the historical sequence behavior information of this application and the sample sequence behavior information of each training sample. If the number of training samples is n, then n first similarity information can be obtained. Calculate the similarity between the information to be analyzed and fused and the sample fusion information of each training sample. If the number of training samples is n, then n second similarity information can be obtained. And based on the n first similarity information and the n second similarity information, obtain the similarity of the k-th application. Repeating the above steps, the similarity of m applications can be obtained.

[0155] In the embodiment of the present application, taking the expression forms of the fusion information and the sequence behavior information as vectors and the similarity information as similarity scores as an example for illustration, exemplarily, the similarity score can be represented by the Pearson correlation coefficient. As shown in formula (1):

[0156]

[0157] Exemplarily, the similarity between vectors x1 and x2 is expressed as sim(x1, x2), and the Pearson correlation coefficient is defined as the covariance of the two vectors divided by the product of their standard deviations. cov(x1, x2) and E[(X1 - μx1)(X2 - μx2)] represent the covariance of the two vectors. In formula (1), X1 and X2 represent the two vectors respectively, μx1 and μx2 represent the scalars (means) of the two vectors respectively, and σ x1 and σ x2 represent the standard deviations of the two vectors respectively.

[0158] In the embodiment of the present application, similarity calculations are performed based on the historical sequence behavior information and the sample fusion information respectively, and the similarity information of the two dimensions is comprehensively considered to obtain the similarity of each application, improving the accuracy of the similarity.

[0159] In some embodiments, when obtaining the similarity of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample, the following steps may be included: obtaining the third similarity of each application for each training sample based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample; for each application, summing the third similarities of each training sample to obtain the similarity of each application.

[0160] Exemplarily, taking the number of training samples as n as an example for illustration, multiplying the first similarity information of the training sample and the second similarity information corresponding to the training sample to obtain the third similarity. Since the number of both the first similarity information and the second similarity information is n, after multiplication, n third similarities can be obtained. One training sample corresponds to one third similarity. Summing the n third similarities to obtain the similarity of each application.

[0161] In the embodiments of the present application, taking the expression form of the fusion information as vectors, the similarity information as similarity scores, the historical sequence behavior information and the sample sequence behavior information as operation sequence vectors, the similarity as a recommendation score, and the sample sequence behavior information corresponding to each application of the training sample as a sequence behavior matrix as examples for illustration. Exemplarily, the recommendation score of the data to be analyzed for the k-th application service in the application service set can be calculated by formula (2).

[0162] score=∑(sim(w k ,v i )*sim(s,t i )) (2)

[0163] In formula (2), score represents the recommendation score, sim(w k ,v i ) represents the similarity score between the operation sequence vector w k of the k-th application service and the i-th operation sequence vector v i in the sequence behavior matrix, sim(s,t i ) represents the similarity score between the fusion vector s to be analyzed and the i-th sample fusion vector ti in the fusion vector set, and ∑ represents summing the n similarity products.

[0164] In the embodiments of the present application, obtaining the similarity of each application based on the similarity information of two dimensions improves the accuracy of the similarity.

[0165] Next, the exemplary application of the embodiments of the present application in an actual application scenario will be described through the collaborative joint learning module and the recommendation result generation module. Taking the expression forms of the implicit information of screen elements, the implicit information of touch actions, and the fusion information as vectors, and the sample sequence behavior information corresponding to the training samples for each application as a sequence behavior matrix, the historical sequence behavior information and the sample sequence behavior information as operation sequence vectors, the similarity information as a similarity score, the similarity as a recommendation score, and the correlation degree of each application as an application recommendation result as an example, the process of generating the application recommendation result may include: determining a fusion vector representing the user's intention according to the mobile phone screen element information and the user's touch actions, and determining the recommendation result with the highest score through a collaborative filtering algorithm.

[0166] In the embodiments of the present application, the collaborative joint learning module is used to determine a fusion vector to be analyzed according to the touch action implicit vector and the screen element implicit vector. Taking the touch action implicit vector and the screen element implicit vector as inputs, a fusion vector representing the user's intention, that is, the fusion vector to be analyzed, is output. The collaborative joint learning module can be trained according to the reverse gradient of the recommendation result.

[0167] It should be noted that the collaborative joint learning module in the embodiments of the present application is a trained model. Both the data to be analyzed and the training data need to undergo collaborative joint learning to obtain a fusion vector. The difference is that the data to be analyzed obtains a fusion vector to be analyzed after collaborative joint learning, and the training samples in the training data obtain a sample fusion vector after collaborative joint learning.

[0168] In the embodiments of the present application, the application recommendation result generation module is used to generate an application recommendation result according to the fusion vector to be analyzed and the preset training data.

[0169] Exemplarily, from the perspective of the overall process of generating an application recommendation result according to the user's touch actions and mobile phone screen element information, the above-mentioned collaborative joint learning module and application recommendation result generation module are respectively introduced, as Figure 8 shown Figure 8 is an optional flowchart for generating a recommendation result provided by the embodiments of the present application. Figure 8 The upper half in the figure is the flowchart of the training data, and the training data can represent the data generated by the user's interaction with the mobile phone. Figure 8 The lower half in the figure is the flowchart of the data to be analyzed.

[0170] In the embodiments of the present application, Figure 8The sample screen element hidden vector and the sample touch action hidden vector can be fused through collaborative joint learning to obtain a sample fusion vector representing the user's intention. The training data includes the sample screen element hidden vector, the sample touch action hidden vector, and the sample service information, and the sample fusion vectors and the sample service information correspond one by one. Exemplarily, if the training data includes n pieces of user interaction data, that is, n training samples, then after the knowledge base embedding and collaborative joint learning are performed, n sample fusion vectors will be obtained. Figure 8 The fusion vector set includes the sample fusion vectors of n pieces of interaction data. The number of sample service information corresponding to the n pieces of interaction data is n, and there will be the same application services among the n sample service information. The set of all application services supported by the application recommendation device is used as the application service set, and the total number of application services it includes is m. Each application service among the m application services is different, and m is much smaller than n.

[0171] It should be noted that for the sample touch action information and the sample screen element information of each training sample in the training data, they are parsed and encoded through the knowledge base embedding method to obtain the screen element hidden vector and the touch action hidden vector. Figure 8 Only the sample screen element hidden vector and the sample touch action hidden vector of each training sample in the training data are shown.

[0172] In the embodiment of the present application, the training data includes n training samples, and one training sample includes a sample fusion vector and its corresponding sample service information. According to the n sample fusion vectors and their corresponding n sample service information, the sample sequence behavior information of each application is determined, that is, Figure 8 The sequence behavior matrix with the dimension of n×m in the upper right corner. Figure 8 Rij in it represents the preference probability of the i-th sample fusion vector for the j-th application service, and can also be understood as the interaction score of the i-th interaction data for the j-th application service. The interaction score is determined by the number of sample service information corresponding to the sample fusion vector in the training data, and can also be understood as the preference probability (i.e., the selection frequency) of the user for each application service under the same interaction data. The higher the preference probability, the larger the Rij value.

[0173] Exemplarily, assume that there are only two application services, that is, j = 1 or 2. The i-th interaction data selects the first application service 3 times and the second application service 1 time. Then Ri1 is 3 / (1 + 3) = 0.75, and correspondingly, Ri2 is 0.25. Performing the above calculations on all interaction data in the fusion vector set, a sequence behavior matrix with the dimension of n×m can be obtained. Taking the row vector of each row in the sequence behavior matrix as an operation sequence vector, n operation sequence vectors can be obtained, and the i-th operation sequence vector is denoted as v i .

[0174] In an embodiment of the present application, for the data to be analyzed, touch action information and screen element information are obtained, parsed and encoded by a knowledge base embedding method to obtain a screen element hidden vector and a touch action hidden vector. The screen element hidden vector and the touch action hidden vector are jointly learned to obtain a fusion vector to be analyzed, denoted as s. Among them, the dimension of the fusion vector s to be analyzed is the same as that of the i-th sample fusion vector.

[0175] In an embodiment of the present application, for the data to be analyzed, in Figure 8 the combined service module, for the k-th application service among the m application services, an operation sequence vector of the data to be analyzed is generated according to the user's historical behavior data, denoted as w k , and thus m operation sequence vectors can be generated. The dimension of the operation sequence vector w k is the same as the dimension of each row vector in the above sequence behavior matrix, that is, the same as the dimension of the i-th operation sequence vector v i , which is 1×m. When the combined service module generates the operation sequence vector of the data to be analyzed, exemplarily, for the k-th application service among the m application services, the preference probabilities of the m applications are calculated according to the user's historical behavior data, the preference probability of the k-th application service is set to a preset peak value (i.e., a preset parameter), and combined with the preference probabilities of the remaining m - 1 application services, the operation sequence vector w k of the k-th application service is determined. By analogy, m operation sequence vectors can be obtained.

[0176] In an embodiment of the present application, the recommendation result generation module can generate the most likely application recommendations using a collaborative filtering algorithm. Exemplarily, for the k-th application service among the m application services, it is necessary to calculate the operation sequence vector w k of the data to be analyzed and the similarity scores with the operation sequence vectors v i corresponding to each sample fusion vector in the fusion vector set. There are n operation sequence vectors corresponding to the n sample fusion vectors, so n similarity scores can be obtained. It is also necessary to calculate the similarity scores between the fusion vector s to be analyzed and each sample fusion vector (denoted as t i ) in the fusion vector set. The fusion vector set includes n sample fusion vectors, so n similarity scores can be obtained.

[0177] In the embodiments of the present application, the similarity scores of the above-mentioned n operation sequence vectors and the similarity scores of the n fusion vectors are multiplied correspondingly and then summed to obtain the recommendation score of the k-th application service. Repeat the above steps to calculate the recommendation scores of the m application services in the application service set to be analyzed, and the recommendation scores of the m application services can be obtained. Select the application service with the highest recommendation score to obtain the application recommendation result, realizing the application recommendation result by combining the mobile phone screen element information and the user's touch actions.

[0178] The embodiments of the present application propose an application recommendation method that combines mobile phone screen element information and user touch actions, comprehensively considering the specific touch actions of the user and the element information corresponding to the screen content with which the user is interacting. By extracting the implicit vectors representing the intentions of the screen element information and the user's touch actions and applying them to the intelligent application recommendation based on the collaborative filtering algorithm, the strong interaction needs of the user are speculated, and then corresponding application recommendations are made, which can meet the immediate needs of the user. Moreover, by combining the touch modality information and the screen element information of the user interface, intelligent recommendation services are provided to the user using the dual modality information strongly related to the user, improving the accuracy of the application recommendation result.

[0179] In the embodiments of the present application, by combining the strong active needs of the user contained in the user's touch actions and the corresponding screen element information in the intelligent interaction scenario, the active intentions of the user are fully considered. Compared with the related technologies in which the user passive receives intelligent recommendation services, an intelligent recommendation service triggered by the user's active intention is provided, improving the accuracy of the recommendation result. By identifying the user's intention and providing personalized and customized services to the user, the user's product experience is improved.

[0180] In some embodiments, the above Figure 2 S204 in can also be implemented in the following manner. Generate service prompt information based on the applications that meet the preset conditions; recommend and display the service prompt information on the user interface.

[0181] In the embodiments of the present application, service prompt information is generated based on the applications that meet the preset conditions. The service prompt information can be a list of application software of different types, and the service prompt information is recommended and displayed on the user interface for the user to select. The service prompt information can be displayed at any position on the user interface in the form of a floating window or a button. The embodiments of the present application do not limit the form and position of the service prompt information.

[0182] Exemplarily, as Figure 9 and Figure 10 shown, Figure 9 and Figure 10 The present application embodiments provide an exemplary schematic diagram of the manifestation form of an application recommendation result. Figure 9 and Figure 10The intelligent recommendation service floating window in it is a form of service prompt information. In Figure 9 it, it is recognized that the current user interface (UI) displays a sharing link of a certain shopping application sent by another user (General Manager Liu) received by the user. At the same time, it is detected that the user has a strong interaction intention: the user long-presses the sharing link and clicks to copy. The application recommendation method of the embodiment of the present application can accurately analyze the user's potential interaction intention: the user wants to open the sharing link in the shopping application. Therefore, the application recommendation device makes a recommendation service and displays the application recommendation result in the form of a floating window below the mobile phone screen. When the user clicks on control 1 of the intelligent service recommendation floating window, they can jump to the corresponding shopping application software A to open the link.

[0183] Exemplarily, in Figure 10 it, the application recommendation device recognizes that the current user interface has received a screenshot sharing of a certain video website provided by a social conversation from Brother Wang. At the same time, it is detected that the user has a strong interaction intention: the user long-presses the link and clicks to view the original image. The application recommendation method of the embodiment of the present application can recognize the user's potential intention: to open the video in the video application. Therefore, the application recommendation device makes a recommendation service and displays the application recommendation result in the form of a floating window below the mobile phone screen. When the user clicks on control 2 of the intelligent service recommendation floating window, they can jump to the corresponding video interface of the video application to view the video.

[0184] In the embodiment of the application, service prompt information is also generated based on applications that meet preset conditions; on the user interface, the service prompt information is recommended and displayed, realizing the fast and effective forwarding of the data selected by the user in the user interface.

[0185] The embodiment of the present application also provides an application recommendation method, as Figure 11 shown, Figure 11 is an optional step flow chart of another application recommendation method provided by the embodiment of the present application. The application recommendation method includes the following steps:

[0186] S1101. Obtain touch operation information for the user interface and obtain screen element information corresponding to the user interface.

[0187] S1102. Perform joint learning based on the screen element information and the touch operation information, and perform correlation processing with the preset sample fusion information and the sample sequence behavior information of each preset application to obtain the correlation degree between the screen element information and multiple applications.

[0188] S1103. Recommend and display the applications whose correlation degree meets the preset conditions.

[0189] In the embodiments of the present application, the above-mentioned multiple applications can be applications of the same type or applications of different types, and the embodiments of the present application do not limit this.

[0190] In some embodiments, the above S1102 may further include the following steps: encoding the screen element information to obtain screen element hidden information; encoding the touch operation information to obtain touch action hidden information; performing collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain information to be analyzed and fused; obtaining the historical sequence behavior information of each application determined based on historical behavior data; performing similarity processing based on the historical sequence behavior information of each application and the sample sequence behavior information, and performing similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the similarity of each application.

[0191] In some embodiments, when performing similarity processing based on the historical sequence behavior information of each application and the sample sequence behavior information, and performing similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the similarity of each application, it can be implemented in the following manner. Performing similarity processing based on the historical sequence behavior information of each application and the sample sequence behavior information to determine the first similarity information of each application for each training sample; performing similarity processing on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; obtaining the similarity of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample.

[0192] To implement the application recommendation method in the embodiments of the present application, the embodiments of the present application further provide an application recommendation device, as Figure 12 shown, Figure 12 is a schematic structural diagram of an application recommendation device provided in the embodiments of the present application. The application recommendation device 120 includes: a first acquisition unit 1201, configured to acquire touch operation information acting on the user interface; and based on the touch operation information, acquire screen element information corresponding to the user interface; a first association unit 1202, configured to acquire the association degree between the screen element information and multiple applications based on the touch operation information and the screen element information, and the multiple applications are applications of the same type; a first display unit 1203, configured to recommend and display the applications whose association degree meets a preset condition.

[0193] In some embodiments, the touch operation information includes: a touch operation and operation content;

[0194] The first acquisition unit 1201 is further configured to acquire a touch operation acting on the user interface; and display the operation content corresponding to the touch operation.

[0195] In some embodiments, the touch operation information includes: a touch operation and operation content. The first acquisition unit 1201 is further configured to acquire a touch operation applied to the user interface; based on the touch operation, acquire screen element information corresponding to the user interface; and based on the touch operation and the screen element information, display corresponding operation content.

[0196] In some embodiments, the first association unit 1202 is further configured to, in response to a selection of the operation content, obtain the association degrees between the screen element information corresponding to the user interface and multiple applications.

[0197] In some embodiments, the touch operation information includes: a touch operation and operation content;

[0198] The first acquisition unit 1201 is further configured to, under the action of a selection operation on the user interface, display operation content corresponding to the selected content; within a preset time range, acquire a touch operation on the operation content; and the touch operation includes a selection operation.

[0199] In some embodiments, the first association unit 1202 is further configured to, based on the screen element information and the touch operation information, associate with preset sample fusion information and preset sample sequence behavior information of each application on each application, so as to obtain the association degrees between the screen element information and each application of multiple applications.

[0200] In some embodiments, the first association unit 1202 includes a first screen element hidden information encoding module, a first touch action hidden information encoding module, and a first association degree processing module;

[0201] The first screen element hidden information encoding module is configured to encode the screen element information to obtain screen element hidden information;

[0202] The first touch action hidden information encoding module is configured to encode the touch operation information to obtain touch action hidden information;

[0203] The first association degree processing module is configured to perform association of each application based on the screen element hidden information, the touch action hidden information, the sample fusion information, and the sample sequence behavior information of each application, so as to obtain the association degrees of each application.

[0204] In some embodiments, the screen element information includes: structural knowledge corresponding to the user interface;

[0205] The first screen element hidden information encoding module is further configured to encode the structural knowledge to obtain screen element hidden information.

[0206] In some embodiments, the screen element information includes: structural knowledge, text-based knowledge, visual knowledge corresponding to the user interface, and the set of current application services related to the user interface; the set of current application services is determined by performing service parsing on the user interface.

[0207] The first screen element hidden information encoding module is further configured to encode the structural knowledge, text-based knowledge, and visual knowledge respectively to obtain structural information, text information, and visual information; encode the set of current application services to obtain service bias information; splice at least one of the text information, visual information, and service bias information, and the structural information to obtain the screen element hidden information.

[0208] In some embodiments, the structural knowledge includes at least one of: control type, control function, and control structure distribution; the text-based knowledge includes at least one of: character recognition information and special symbol recognition information, and the original text information; the visual knowledge includes at least one of: image content recognition information and image source information, and the original image information.

[0209] In some embodiments, the first screen element hidden information encoding module is further configured to perform vector quantization encoding on the structural knowledge to obtain structural information; perform vector quantization encoding on at least one of the character recognition information and special symbol recognition information to obtain text content understanding encoding information; and perform stacked auto-encoding on the original text information to obtain high-dimensional text information; splice the high-dimensional text information and the text content understanding encoding information to obtain text information; perform information encoding on at least one of the image content recognition information and image source information to obtain image content understanding encoding information; and perform stacked auto-encoding on the original image information to obtain high-dimensional image information; splice the high-dimensional image information and the image content understanding encoding information to obtain visual information.

[0210] In some embodiments, the touch operation information includes: operation content and touch operation.

[0211] The first touch action hidden information encoding module is further configured to encode the touch operation and the operation content respectively to obtain touch operation information and operation content information; splice the touch operation information and the operation content information to obtain the touch action hidden information.

[0212] In some embodiments, the first acquisition unit 1201 is further configured to perform control information recognition on the user interface or the area to be recognized to obtain structural knowledge; perform text content recognition on the user interface or the area to be recognized to obtain text-based knowledge; perform image content recognition on the user interface or the area to be recognized to obtain visual knowledge; perform service parsing on the user interface to determine the set of current application services; at least one of the text-based knowledge, visual knowledge, and the set of current application services, and the structural knowledge are used as the screen element information.

[0213] In some embodiments, the first acquisition unit 1201 is further configured to obtain, through an accessibility service interface, the structural information of the control tree corresponding to the user interface or the area to be recognized; perform control information recognition on the structural information of the control tree to obtain structural knowledge.

[0214] In some embodiments, the first association unit 1202 further includes a first collaborative joint learning module;

[0215] The first collaborative joint learning module is configured to perform collaborative joint learning on the implicit information of the screen elements and the implicit information of the touch actions to obtain the information to be analyzed and fused;

[0216] The first acquisition unit 1201 is further configured to obtain the historical sequence behavior information of each application determined based on the historical behavior data;

[0217] The first association degree processing module is further configured to perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the similarity of each application.

[0218] In some embodiments, the first association degree processing module is further configured to determine, based on the historical behavior data, the historical behavior parameters of multiple applications; for each application, set the historical behavior parameters corresponding to the application as preset parameters, and combine the historical behavior parameters of the remaining applications in the multiple applications except this application to obtain the historical sequence behavior information of this application.

[0219] In some embodiments, the first association degree processing module is further configured to perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application to determine the first similarity information of each application for each training sample; perform similarity processing on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample, obtain the similarity of each application.

[0220] In some embodiments, the first association degree processing module is further configured to obtain, based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample, the third similarity of each application for each training sample; for each application, sum the third similarities of each training sample to obtain the similarity of each application.

[0221] In some embodiments, the application recommendation device 120 further includes a training unit. The training unit is configured to determine the sample screen element hidden information, the sample touch action hidden information, and the sample service information corresponding to each training sample according to the preset training data; perform collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain the preset sample fusion information of each training sample; and determine the sample sequence behavior information of each preset application according to the sample service information of each training sample.

[0222] In some embodiments, the first display unit 1203 is further configured to recommend and display, on the user interface, the application with the highest similarity to each application; or recommend and display, on the user interface, a preset number of applications with the highest similarity to each application.

[0223] In some embodiments, an application includes: an application type, and / or, an application service.

[0224] In some embodiments, the first display unit 1203 is further configured to generate a service prompt message based on an application that meets a preset condition; and recommend and display the service prompt message on the user interface.

[0225] In some embodiments, the first acquisition unit 1201 is further configured to, if the touch operation information indicates that its operation content meets a preset trigger recommendation condition, acquire the screen element information of the entire interface corresponding to the user interface; the preset trigger recommendation condition indicates an expected operation intention.

[0226] In some embodiments, the first acquisition unit 1201 is further configured to determine a current area to be recognized at the touch position where the touch operation information acts; and recognize the current area to be recognized to determine the screen element information.

[0227] In some embodiments, the application recommendation device 120 includes a recognition unit. The recognition unit is configured to, if no screen element information is recognized in the current area to be recognized, determine the next area to be recognized, and perform recognition in the next area to be recognized until screen element information exists or the entire interface is recognized; the next area to be recognized is larger than the current area to be recognized.

[0228] To implement the application recommendation method of the embodiments of the present application, the embodiments of the present application further provide another application recommendation device, as Figure 13 shown Figure 13A structural schematic diagram of an application recommendation device provided by an embodiment of the present application. The application recommendation device 130 includes: a second acquisition unit 1301, configured to acquire touch operation information for a user interface and acquire screen element information corresponding to the user interface; a second association unit 1302, configured to perform joint learning based on the screen element information and the touch operation information, and perform an association process with preset sample fusion information and preset sample sequence behavior information of each application to obtain the association degree between the screen element information and multiple applications; a second display unit 1303, configured to recommend and display applications whose association degree meets a preset condition.

[0229] In some embodiments, the second association unit 1302 includes a second screen element hidden information encoding module, a second touch action hidden information encoding module, a second collaborative joint learning module, and a second association degree processing module;

[0230] The second screen element hidden information encoding module is configured to encode the screen element information to obtain screen element hidden information;

[0231] The second touch action hidden information encoding module is configured to encode the touch operation information to obtain touch action hidden information;

[0232] The second collaborative joint learning module is configured to perform collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain information to be analyzed and fused;

[0233] The second acquisition unit 1301 is further configured to acquire historical sequence behavior information of each application determined based on historical behavior data;

[0234] The second association degree processing module is configured to perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the similarity of each application.

[0235] In some embodiments, the second association degree processing module is configured to perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application to determine the first similarity information of each application for each training sample; perform similarity processing on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; and obtain the similarity of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample.

[0236] It should be noted that when the application recommendation device provided in the above embodiments performs application recommendation, only the division of the above program units is used for illustration. In actual applications, the above processing can be allocated to different program units according to needs, that is, the internal structure of the device is divided into different program units to complete all or part of the above-described processing. In addition, the application recommendation device provided in the above embodiments and the embodiments of the application recommendation method belong to the same concept. For the specific implementation process and beneficial effects, please refer to the method embodiments, which will not be elaborated here. For the technical details not disclosed in the embodiments of this device, please refer to the description of the method embodiments of this application for understanding.

[0237] In the embodiments of the present application, Figure 14 is a schematic structural diagram of the application recommendation device proposed in the embodiments of the present application. As Figure 14 shown, the device 140 proposed in the embodiments of the present application may further include a processor 1401, a memory 1402 storing executable instructions of the processor 1401. In some embodiments, the application recommendation device 140 may further include a communication interface 1403, and a bus 1404 for connecting the processor 1401, the memory 1402, and the communication interface 1403.

[0238] In the embodiments of the present application, the bus 1404 is used to connect the communication interface 1403, the processor 1401, and the memory 1402 and enable mutual communication between these components.

[0239] In the embodiments of the present application, the above-mentioned processor 1401 is configured to obtain touch operation information acting on the user interface; based on the touch operation information, obtain screen element information corresponding to the user interface; based on the touch operation information and the screen element information, obtain the association degree between the screen element information and multiple applications, where the multiple applications are applications of the same type; recommend and display the applications whose association degree meets the preset conditions.

[0240] In the embodiments of the present application, the above-mentioned processor 1401 is further configured to obtain touch operation information for the user interface, and obtain screen element information corresponding to the user interface; perform joint learning based on the screen element information and the touch operation information, and perform association processing with the preset sample fusion information and the sample sequence behavior information of each preset application to obtain the association degree between the screen element information and multiple applications, where the multiple applications are applications of the same type; recommend and display the applications whose association degree meets the preset conditions.

[0241] In an embodiment of the present application, the above-mentioned processor 1401 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic devices used to implement the above-mentioned processor functions may also be others, and the embodiments of the present application do not make specific limitations.

[0242] In the application recommendation device 140, the memory 1402 may be connected to the processor 1401. The memory 1402 is used to store executable program codes and data, and the program codes include computer operation instructions. The memory 1402 may include high-speed RAM memory and may also include non-volatile memory, for example, at least two disk memories. In practical applications, the above-mentioned memory 1402 may all be volatile memory, such as Random-Access Memory (RAM); or non-volatile memory, such as Read-Only Memory (ROM), flash memory, Hard Disk Drive (HDD), or Solid-State Drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 1401.

[0243] In addition, in this embodiment, each functional module may be integrated in a processing unit, may exist separately physically for each unit, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.

[0244] When the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0245] An embodiment of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the application recommendation method of any of the above embodiments.

[0246] Exemplarily, the program instructions corresponding to an application recommendation method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to an application recommendation method in the storage medium are read or executed by an electronic device, the application recommendation method of any of the above embodiments can be implemented.

[0247] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) that include computer-usable program codes.

[0248] The present application is described with reference to the schematic flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the schematic flowcharts and / or block diagrams, and the combination of flows and / or blocks in the schematic flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified function in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0249] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more of the flowcharts and / or boxes of the implementation process schematic. Figure 1 one or more of the processes and / or boxes Figure 1 of the functions specified in one or more of the boxes.

[0250] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the flowcharts and / or boxes of the implementation process schematic. Figure 1 one or more of the processes and / or boxes Figure 1 of the functions specified in one or more of the boxes.

[0251] The above are merely preferred embodiments of the present application and are not intended to limit the scope of protection of the present application.

Claims

1. A method for application recommendation, characterized in that, The method includes: Obtaining touch operation information acting on the user interface; Based on the touch operation information, obtaining screen element information corresponding to the user interface; Based on the touch operation information and the screen element information, obtaining the association degrees between the screen element information and multiple applications, where the multiple applications are applications of the same type; Recommending and displaying the applications whose association degrees meet a preset condition; Wherein, the obtaining the association degrees between the screen element information and multiple applications based on the touch operation information and the screen element information includes: Encoding the screen element information to obtain screen element hidden information; Encoding the touch operation information to obtain touch action hidden information; wherein, the touch action hidden information is obtained by encoding the operation content and touch operation included in the touch operation information; Performing collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain information to be analyzed and fused; Obtaining historical sequence behavior information of each application determined based on historical behavior data; Performing similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application, and performing similarity processing based on the information to be analyzed and fused and sample fusion information of each training sample, to obtain the association degrees of the respective applications; Wherein, before obtaining the association degrees of the respective applications, the method further includes: Determining sample screen element hidden information, sample touch action hidden information, and sample service information corresponding to each training sample according to preset training data; Performing collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain the preset sample fusion information of each training sample; Determining the sample sequence behavior information of the preset respective applications according to the sample service information of each training sample.

2. The method according to claim 1, characterized in that The touch operation information includes: touch operation and operation content; the obtaining the touch operation information acting on the user interface includes: Obtaining the touch operation acting on the user interface; Displaying the operation content corresponding to the touch operation.

3. The method according to claim 1, characterized in that, The touch operation information includes: touch operation and operation content; the obtaining the touch operation information acting on the user interface includes: Obtaining the touch operation acting on the user interface; Based on the touch operation, obtaining the screen element information corresponding to the user interface; Based on the touch operation and the screen element information, displaying the corresponding operation content.

4. The method according to claim 2 or 3, characterized in that, The obtaining the association degrees between the screen element information and multiple applications based on the touch operation information and the screen element information further includes: In response to the selection of the operation content, obtaining the association degrees between the screen element information corresponding to the user interface and the multiple applications.

5. The method according to claim 1, wherein The touch operation information includes: touch operation and operation content; the obtaining the touch operation information acting on the user interface includes: Under the action of a selection operation on the user interface, displaying the operation content corresponding to the selected content; Within a preset time range, obtain the touch operation for the operation content; the touch operation includes the selection operation.

6. The method according to claim 1, characterized in that, The screen element information includes: structural knowledge corresponding to the user interface; Encoding the screen element information to obtain screen element hidden information includes: Encoding the structural knowledge to obtain the screen element hidden information.

7. The method according to claim 1, characterized in that The screen element information includes: structural knowledge, text-based knowledge, visual knowledge corresponding to the user interface, and a set of current application services related to the user interface; the set of current application services is determined by performing service parsing on the user interface; Encoding the screen element information to obtain screen element hidden information includes: Encoding the structural knowledge, the text-based knowledge, and the visual knowledge respectively to obtain structural information, text information, and visual information; Encoding the set of current application services to obtain service bias information; Concatenating at least one of the text information, the visual information, and the service bias information, and the structural information to obtain the screen element hidden information.

8. The method according to claim 7, wherein The structural knowledge includes at least one of: control type, control function, and control structure distribution; The text-based knowledge includes at least one of: character recognition information and special symbol recognition information, and text original information; The visual knowledge includes at least one of: image content recognition information and image source information, and original image information.

9. The method according to claim 7 or 8, characterized in that, Encoding the structural knowledge, the text-based knowledge, and the visual knowledge respectively to obtain structural information, text information, and visual information includes: Performing vector quantization encoding on the structural knowledge to obtain the structural information; Performing vector quantization encoding on at least one of character recognition information and special symbol recognition information to obtain text content understanding encoding information; and performing stacked autoencoding on the text original information to obtain text high-dimensional information; Concatenating the text high-dimensional information and the text content understanding encoding information to obtain the text information; Performing information encoding on at least one of image content recognition information and image source information to obtain image content understanding encoding information; and performing stacked autoencoding on the original image information to obtain image high-dimensional information; Concatenating the image high-dimensional information and the image content understanding encoding information to obtain the visual information.

10. The method according to claim 1, wherein The touch operation information includes: the operation content and the touch operation; Encoding the touch operation information to obtain touch action hidden information includes: Encoding the touch operation and the operation content respectively to obtain touch operation information and operation content information; Concatenating the touch operation information and the operation content information to obtain the touch action hidden information.

11. The method according to claim 1, characterized in that, Obtaining the screen element information corresponding to the user interface includes: Performing control information recognition on the user interface or the area to be recognized to obtain structural knowledge; Performing text content recognition on the user interface or the area to be recognized to obtain text-based knowledge; Perform image content recognition on the user interface or the area to be recognized to obtain visual knowledge; Perform service parsing on the user interface to determine the current application service set; At least one of the text-based knowledge, the visual knowledge, and the current application service set, and the structural knowledge serve as the screen element information.

12. The method according to claim 11, wherein The performing control information recognition on the user interface or the area to be recognized to obtain structural knowledge includes: Obtain the structural information of the control tree corresponding to the user interface or the area to be recognized through the accessibility service interface; Perform control information recognition on the structural information of the control tree to obtain the structural knowledge.

13. The method according to claim 1, wherein The obtaining the historical sequence behavior information of each application determined based on historical behavior data includes: Determine the historical behavior parameters of multiple applications based on the historical behavior data; For each application, set the historical behavior parameters corresponding to the application to preset parameters, and combine the historical behavior parameters of the multiple applications except this application to obtain the historical sequence behavior information of this application.

14. The method according to claim 1, wherein The performing similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application, and performing similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the correlation degree of each application includes: Perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application to determine the first similarity information of each application for each training sample; Perform similarity processing on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; Obtain the correlation degree of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample.

15. The method according to claim 14, wherein The obtaining the correlation degree of each application based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample includes: Obtain the third similarity of each application for each training sample based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample; For each application, sum the third similarities of each training sample to obtain the correlation degree of each application.

16. The method according to claim 1, wherein The recommending and displaying the applications whose correlation degree meets the preset conditions includes: On the user interface, recommend and display the application with the highest correlation degree among each application; Or, On the user interface, recommend and display a preset number of applications with the highest correlation degree among each application.

17. According to the method of claim 1 or 16, characterized in that An application includes: application type, and / or, application service.

18. The method according to claim 1, characterized in that, The recommending and displaying the applications whose correlation degree meets the preset conditions includes: Generate service prompt information based on the applications that meet the preset conditions; On the user interface, recommend and display the service prompt information.

19. The method according to claim 1 or 2, characterized in that The obtaining the screen element information corresponding to the user interface based on the touch operation information includes: If the touch operation information indicates that its operation content meets the preset trigger recommendation condition, obtain the screen element information of the entire interface corresponding to the user interface; the preset trigger recommendation condition indicates the expected operation intention.

20. The method according to claim 1 or 2, characterized in that, The obtaining of the screen element information corresponding to the user interface based on the touch operation information includes: Determine the current area to be recognized at the touch position where the touch operation information acts; Recognize the current area to be recognized to determine the screen element information.

21. The method according to claim 20, wherein, After determining the current area to be recognized at the touch position where the touch operation information acts, the method further includes: If no screen element information is recognized in the current area to be recognized, determine the next area to be recognized, and recognize in the next area to be recognized until there is screen element information or the entire interface is recognized; the next area to be recognized is larger than the current area to be recognized.

22. A method for application recommendation, characterized in that The method includes: Obtain touch operation information for the user interface and obtain the screen element information corresponding to the user interface; Perform joint learning based on the screen element information and the touch operation information, and perform association processing with the preset sample fusion information and the sample sequence behavior information of each preset application to obtain the association degree between the screen element information and multiple applications; Recommend and display the applications whose association degree meets the preset conditions; Among them, the performing of joint learning based on the screen element information and the touch operation information, and performing association processing with the preset sample fusion information and the sample sequence behavior information of each preset application to obtain the association degree between the screen element information and multiple applications includes: Encode the screen element information to obtain screen element hidden information; Encode the touch operation information to obtain touch action hidden information; wherein, the touch action hidden information is obtained by encoding the operation content and touch operation included in the touch operation information; Perform collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain the information to be analyzed and fused; Obtain the historical sequence behavior information of each application determined based on historical behavior data; Perform similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the association degree of each application; Among them, before obtaining the association degree of each application, the method further includes: Determine the sample screen element hidden information, sample touch action hidden information, and sample service information corresponding to each training sample according to the preset training data; Perform collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain the preset sample fusion information of each training sample; Determine the sample sequence behavior information of each preset application according to the sample service information of each training sample.

23. The method according to claim 22, wherein Performing similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application, and performing similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the correlation degree of each application, including: Performing similarity processing based on the historical sequence behavior information and the sample sequence behavior information of each application to determine the first similarity information of each application for each training sample; Performing similarity processing on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the second similarity information corresponding to each training sample; Based on the first similarity information of each application for each training sample and the second similarity information corresponding to each training sample, obtaining the correlation degree of each application.

24. An application recommendation device, characterized in that, The device includes: A first acquisition unit, configured to acquire touch operation information acting on a user interface; and based on the touch operation information, acquire screen element information corresponding to the user interface; A first association unit, configured to acquire the correlation degree between the screen element information and multiple applications based on the touch operation information and the screen element information, where the multiple applications are applications of the same type; The first association unit is further configured to encode the screen element information to obtain screen element hidden information; encode the touch operation information to obtain touch action hidden information; where the touch action hidden information is obtained by encoding the operation content and touch operation included in the touch operation information; perform collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain information to be analyzed and fused; acquire the historical sequence behavior information of each application determined based on historical behavior data; perform similarity processing based on the historical sequence behavior information and sample sequence behavior information of each application, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the correlation degree of each application; A training unit, configured to determine sample screen element hidden information, sample touch action hidden information, and sample service information corresponding to each training sample according to preset training data; perform collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain the preset sample fusion information of each training sample; determine the sample sequence behavior information of the preset each application according to the sample service information of each training sample; A first display unit, configured to recommend and display applications whose correlation degree meets a preset condition.

25. An application recommendation device, characterized in that, The device includes: A second acquisition unit, configured to acquire touch operation information for a user interface, and acquire screen element information corresponding to the user interface; A second association unit, configured to perform joint learning based on the screen element information and the touch operation information, and perform association processing with the preset sample fusion information and the sample sequence behavior information of the preset each application to acquire the correlation degree between the screen element information and multiple applications; The second association unit is further configured to encode the screen element information to obtain screen element hidden information; encode the touch operation information to obtain touch action hidden information, where the touch action hidden information is obtained by encoding the operation content and touch operation included in the touch operation information; perform collaborative joint learning on the screen element hidden information and the touch action hidden information to obtain information to be analyzed and fused; obtain historical sequence behavior information of each application determined based on historical behavior data; perform similarity processing based on the historical sequence behavior information of each application and the sample sequence behavior information, and perform similarity processing based on the information to be analyzed and fused and the sample fusion information of each training sample to obtain the association degree of each application; Wherein, before obtaining the association degree of each application, the device further includes: determining sample screen element hidden information, sample touch action hidden information, and sample service information corresponding to each training sample according to preset training data; performing collaborative joint learning on the sample screen element hidden information and the sample touch action hidden information to obtain the preset sample fusion information of each training sample; determining the sample sequence behavior information of the preset each application according to the sample service information of each training sample; The second display unit is configured to recommend and display an application whose association degree meets a preset condition.

26. An application recommendation device, characterized in that, The device includes a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the method according to any one of claims 1 to 21, or the method according to claim 22 or 23.

27. A computer-readable storage medium, characterized in that, It stores executable instructions, which are used to implement the method according to any one of claims 1 to 21, or the method according to claim 22 or 23 when executed by a processor.

Images