Artificial intelligence-based recommendation processing methods, devices, and electronic equipment

Abstract

This application provides a recommendation processing method, apparatus, electronic device, and computer-readable storage medium based on artificial intelligence. The method includes: acquiring an image of an object to be recommended, wherein the image is captured when the object is viewing information in an environment; performing recognition processing on the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment; acquiring target recommendation information for the object to be recommended and generating multiple candidate display methods for the target recommendation information; determining a target display method from the multiple candidate display methods that matches at least one of the state features and the environmental features; and displaying the target recommendation information according to the target display method. This application can improve recommendation accuracy by using images captured in real time.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to an AI-based recommendation processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0003] Information recommendation is an important application of artificial intelligence. The ranking module in a recommendation system typically ranks items based on click-through rates predicted by machine learning models, prioritizing those with higher ratings. Various efforts have been made to improve the accuracy of click-through rate predictions for machine learning models, such as building large amounts of feature data during the feature engineering stage to allow the models to learn effectively. However, even with a very high degree of matching between the instant recommendation information and the recommended items, situations still arise where no interactive behavior is generated. Summary of the Invention

[0004] This application provides an artificial intelligence-based recommendation processing method, apparatus, electronic device, and computer-readable storage medium, which can improve recommendation accuracy by utilizing real-time captured images.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an artificial intelligence-based recommendation processing method, including:

[0007] Acquire an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment;

[0008] The image is processed to obtain at least one of the state features of the object to be recommended and the environmental features of the environment;

[0009] Obtain target recommendation information for the object to be recommended, and generate multiple candidate display methods for the target recommendation information;

[0010] From the plurality of candidate display methods, a target display method that is adapted to at least one of the state feature and the environment feature is determined;

[0011] The target recommendation information is displayed according to the target display method.

[0012] This application provides an artificial intelligence-based recommendation processing device, comprising:

[0013] A shooting module is used to acquire an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment;

[0014] The recognition module is used to process the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment.

[0015] The recommendation module is used to obtain target recommendation information for the object to be recommended, and generate multiple candidate display methods for the target recommendation information;

[0016] The display module is configured to determine, from the plurality of candidate display modes, a target display mode that is adapted to at least one of the state features and the environment features;

[0017] The display module is also used to display the target recommendation information according to the target display method.

[0018] In the above scheme, the recommendation module is further configured to: obtain recommendation information for multiple candidates for the object to be recommended; and determine target recommendation information that matches at least one of the state feature and the environment feature from the multiple candidate recommendation information.

[0019] In the above scheme, the recommendation module is further configured to: perform the following processing on each candidate recommendation information: obtain the content features of the candidate recommendation information and the profile features of the object to be recommended; perform a first feature cross-processing on the content features of the candidate recommendation information, the profile features of the object to be recommended, and at least one of the state features and the environment features to obtain a first feature cross-processing result; perform logistic regression processing on the first feature cross-processing result to obtain a first recommendation index of the candidate recommendation information; and determine the candidate recommendation information corresponding to the first recommendation index that exceeds the first recommendation index threshold as the target recommendation information.

[0020] In the above scheme, the recommendation module is further configured to: select multiple candidate recommendation information corresponding to a first recommendation indicator that exceeds a first recommendation indicator threshold; select at least one recommendation information that satisfies the diversity condition from the selected multiple candidate recommendation information as the target recommendation information; wherein the diversity condition specifies the maximum number of target recommendation information belonging to the same category.

[0021] In the above scheme, the recommendation module is further configured to: generate at least one candidate template of the target recommendation information; generate at least one candidate summary of the target recommendation information; generate at least one candidate cover of the target recommendation information; combine the at least one candidate template, the at least one candidate summary, and the at least one candidate cover in different ways to obtain the multiple candidate display methods; wherein, the candidate summary includes at least one of the following: the title of the target recommendation information, the recommendation reason of the target recommendation information, and the introduction of the target recommendation information.

[0022] In the above scheme, the recommendation module is further configured to: perform at least one of the following processes: extract key content of the target recommendation information and generate the candidate summary based on the key content; determine interactive objects that have interactive behavior with the target recommendation information and generate the candidate summary based on the interactive objects; determine related information whose correlation with the target recommendation information exceeds a correlation threshold and generate the candidate summary based on the related information.

[0023] In the above scheme, the recommendation module is further configured to: when the target recommendation information is a recommended video, segment the recommended video into multiple shots, wherein each shot includes multiple consecutive video frames; perform the following processing for each shot: perform clustering processing on the multiple video frames of the shot to obtain multiple clusters under the shot; determine the video frame in each cluster that is closest to the corresponding cluster center, and determine the closest video frame as the candidate cover.

[0024] In the above scheme, the recommendation module is further configured to: perform the following processing for each video frame of the recommended video: determine the position of each pixel in the video frame; form a segmentation unit from any two consecutive video frames, and perform the following processing for each segmentation unit of the recommended video: determine the grayscale difference of pixels at the same position in the two video frames of the segmentation unit, and average the grayscale differences of pixels at multiple positions to obtain the difference of the segmentation unit; determine that the segmentation unit corresponding to the difference greater than the difference threshold has a shot boundary, and divide the two video frames of the segmentation unit into different shots.

[0025] In the above scheme, the recommendation module is further configured to: form a video frame set from the multiple consecutive video frames; randomly select N video frames from the video frame set, use the image features corresponding to the N video frames as the initial cluster centers of multiple cluster sets, and remove the N video frames from the video frame set, where N is the number of candidate covers corresponding to the shot, and N is an integer greater than or equal to 2; initialize the number of iterations of the clustering process to M, and establish an empty set corresponding to each cluster, where M is an integer greater than or equal to 2; and perform the following processing in each iteration of the clustering process: perform a more... The new process involves performing cluster center generation based on the updated processing results to obtain new cluster centers for each cluster. When the new cluster centers differ from the initial cluster centers, the video frames corresponding to the initial cluster centers are added back to the video frame set, and the initial cluster centers are updated based on the new cluster centers. The set of each cluster obtained after M iterations is determined as the clustering processing result, or the set of each cluster obtained after m iterations is determined as the clustering processing result. Wherein, the cluster centers of multiple clusters obtained after m iterations are the same as those of multiple clusters obtained after m-1 iterations, where m is an integer variable and its value satisfies 2≤m≤M.

[0026] In the above scheme, the recommendation module is further configured to: perform the following processing for each cluster: average the image features of each video frame of the cluster to obtain the cluster center; and determine the video frame closest to the cluster center based on the distance between the image features of each video frame of the cluster and the cluster center.

[0027] In the above scheme, the recognition module is further configured to perform at least one of the following processes: perform state recognition processing on the image to obtain the state features of the object to be recommended; perform environment recognition processing on the image to obtain the environmental features of the environment.

[0028] In the above scheme, the display module is configured to perform the following processing for each candidate display method: extract text features and image features of the candidate display method; perform a second feature cross-processing on the text features and image features of the candidate display method, the portrait features, and at least one of the state features and the environment features to obtain a second feature cross-processing result; perform logistic regression processing on the second feature cross-processing result to obtain a second recommendation index for the candidate display methods of the target recommendation information. The candidate display method corresponding to the second recommendation index that exceeds the second recommendation index threshold is determined as the target display method.

[0029] This application provides an artificial intelligence-based recommendation processing method, including:

[0030] Acquire an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment;

[0031] The image is processed to obtain at least one of the state features of the object to be recommended and the environmental features of the environment;

[0032] Obtain recommendation information from multiple candidates for the object to be recommended;

[0033] From the plurality of candidate recommendation information, determine target recommendation information that matches at least one of the state feature and the environment feature;

[0034] Display the target recommendation information.

[0035] This application provides an artificial intelligence-based recommendation processing device, comprising:

[0036] A shooting module is used to acquire an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment;

[0037] The recognition module is used to process the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment.

[0038] The recommendation module is used to obtain recommendation information from multiple candidates for the object to be recommended;

[0039] The recommendation module is further configured to determine, from the plurality of candidate recommendation information, target recommendation information that is compatible with at least one of the state feature and the environment feature;

[0040] The display module is used to display the target recommendation information.

[0041] This application provides an electronic device, including:

[0042] Memory, used to store executable instructions;

[0043] When the processor executes the executable instructions stored in the memory, it implements the AI-based recommended processing method provided in the embodiments of this application.

[0044] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the AI-based recommendation processing method provided in this application.

[0045] The embodiments of this application have the following beneficial effects:

[0046] The system captures images of the user viewing information in their environment, obtaining at least one of the user's state and environmental features. Then, it selects a target display method from multiple candidate display methods that matches at least one of the state and environmental features, and displays the target recommendation information accordingly. Since the target display method is related to at least one of the state and environmental features, and these features directly characterize the user's performance and environment during the real-time recommendation process, the target display method can be accurately derived, effectively increasing the user's click-through rate and thus improving recommendation accuracy and efficiency. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the recommendation logic of the recommendation processing method provided in related technologies;

[0048] Figure 2 This is a schematic diagram of the architecture of the AI-based recommendation processing system provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0050] Figures 4A-4D This is a flowchart illustrating the recommendation processing method based on artificial intelligence provided in an embodiment of this application;

[0051] Figure 5 This is a flowchart illustrating the recommended processing method provided in the embodiments of this application;

[0052] Figure 6 This is a flowchart illustrating the recommended processing method provided in the embodiments of this application;

[0053] Figure 7 This is a schematic diagram of keyframe extraction provided in this application.

[0054] Figure 8 This is an image processing flowchart provided in an embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the display processing provided in an embodiment of this application;

[0056] Figure 10 This is a product interface diagram of the AI-based recommendation processing method provided in the embodiments of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0059] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0061] In the implementation of this application, the collection and processing of relevant data should be strictly in accordance with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0062] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0063] 1) Features: Features are data representations that can effectively reflect the different characteristics of a problem.

[0064] 2) Feature extraction: The feature extraction process can transform raw data into a more effective representation based on the problem to be solved;

[0065] 3) Feature Engineering: Feature engineering is the process of using relevant knowledge in the data domain to create features that enable the algorithm to achieve optimal performance;

[0066] 4) User profile: A user profile is a user model abstracted based on information such as a user's social attributes, lifestyle habits, and consumption behavior;

[0067] 5) Personalized Recommendation: Personalized recommendation is a technology that pushes different content to different users based on user profiles;

[0068] 6) Personalized display: This is a technology that displays the same content to users in different ways based on user profiles.

[0069] See Figure 1 , Figure 1 This is a schematic diagram of the recommendation logic of the recommendation processing method provided in related technologies. It extracts user features, such as age, residence, gender, occupation, school, favorite celebrities, frequently viewed information types, etc., and builds user profiles based on user features. It also extracts content features, such as content category, content keywords, corresponding works, content popularity, etc. The content features and user features are used as input features of the recommendation model. The machine learning recommendation model predicts the click-through rate based on the input features, generates recommended content for users based on the click-through rate prediction results, and finally displays the recommended content to users.

[0070] When personalizing video recommendations for users, it's necessary to match the user's historical viewing behavior profile with the content features of the recommended videos to determine the most suitable ones. Therefore, user profile features are crucial for recommendation systems, significantly improving user interest and video click-through rates. Typically, we extract useful features from large amounts of training data and then use machine learning methods to train a model. The effectiveness of this model largely depends on the effectiveness of the features extracted from the raw data through feature engineering.

[0071] In implementing the embodiments of this application, the applicant discovered that during the recommendation process, both the recommended video and how it is presented affect whether users view the content. The related technology has the following drawbacks: different users may be recommended the same content for different reasons, but the same title, cover image, and other display methods are shown to different users. Therefore, users may not perceive what is interesting about the recommended content and will not click to enter the details page of the recommended information to view the detailed content and related information. Furthermore, user profiles are long-term behavioral representations, and user profiles cannot reflect the user's current mood and viewing environment when viewing recommended information. The recommended information or its display method may not match the user's current mood and viewing environment, thus causing user confusion.

[0072] To address the aforementioned technical problems, embodiments of this application provide an artificial intelligence-based recommendation processing method, apparatus, electronic device, and computer-readable storage medium. This system automatically generates video summaries for different users based on their current facial expressions and user profiles. Therefore, users are more receptive to content pushed by the recommendation system, thereby improving the effectiveness of recommendations.

[0073] The recommended processing method provided in this application embodiment can be implemented by various electronic devices, such as by a terminal or server alone, or by a terminal and server working together.

[0074] See Figure 2 , Figure 2 This is a schematic diagram of the architecture of an AI-based recommendation processing system provided in this application embodiment. The terminal 400 is connected to the server 200 through the network 300, which can be a wide area network, a local area network, or a combination of both.

[0075] In some embodiments, the function of the AI-based recommendation processing system is implemented based on server 200 and terminal 400. During user interaction with terminal 400, in response to receiving a refresh operation from the user, terminal 400 acquires the user's image, which is captured when the user is viewing information in the environment. Terminal 400 sends the image and recommendation request to server 200, which performs image recognition processing to obtain at least one of the user's state characteristics and environmental characteristics. Server 200 acquires target recommendation information for the user and generates multiple candidate display methods for the target recommendation information. From the multiple candidate display methods, a target display method that matches at least one of the state characteristics and environmental characteristics is determined. Server 200 returns the target display method and target recommendation information to terminal 400, whereby the target recommendation information is displayed according to the target display method.

[0076] In some embodiments, when the recommendation processing system is applied to a video recommendation scenario, in response to the terminal 400 receiving a refresh operation from the user, the terminal 400 acquires an image of the object to be recommended, wherein the image is captured when the object to be recommended is watching a video in the environment; the terminal 400 sends the image to the server 200, which performs recognition processing on the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment; the server 200 acquires a target recommended video for the object to be recommended and generates multiple candidate display modes for the target recommended video; the server determines a target display mode that matches at least one of the state features and environmental features from the multiple candidate display modes; the server 200 returns the target display mode and the target recommended video to the terminal 400, and the target recommended video is displayed on the terminal 400 according to the target display mode.

[0077] In other embodiments, when the recommendation processing method provided in this application is implemented by the terminal alone, in the various application scenarios described above, in response to the terminal 400 receiving the user's refresh operation, the terminal 400 obtains an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment, the image is processed for recognition to obtain at least one of the state features of the object to be recommended and the environmental features of the environment, the target recommendation information for the object to be recommended is obtained, and multiple candidate display methods for the target recommendation information are generated, the target display method that matches at least one of the state features and environmental features is determined from the multiple candidate display methods, and the target recommendation information is displayed according to the target display method.

[0078] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart TV, smart in-vehicle device, etc. Terminal 400 may have a client application, such as a video client, browser client, information streaming client, image capture client, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0079] Next, the structure of the electronic device for implementing the AI-based recommendation processing method provided in the embodiments of this application will be described. As before, the electronic device provided in the embodiments of this application can be... Figure 2 Server 200 or Terminal 400. See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device is described using terminal 400 as an example. Figure 3 The terminal 400 shown includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 440.

[0080] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0082] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0083] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0084] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0085] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0086] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0087] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0088] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0089] In some embodiments, the AI-based recommendation processing apparatus provided in this application can be implemented in software. Figure 3 An AI-based recommendation processing device 455-1 stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a camera module 4551-1, a recognition module 4552-1, a recommendation module 4553-1, and a display module 4554-1. Figure 3 An AI-based recommendation processing device 455-2 stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: a shooting module 4551-2, a recognition module 4552-2, a recommendation module 4553-2, and a display module 4554-2. These modules are logically connected and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0090] The following will describe the AI-based recommendation processing method provided in this application embodiment, with reference to the exemplary application and implementation of the terminal 400 provided in the embodiments of this application.

[0091] See Figure 4A , Figure 4A This is a flowchart illustrating the AI-based recommendation processing method provided in the embodiments of this application, which will be combined with... Figure 4A The steps shown are explained.

[0092] In step 101, the image of the object to be recommended is obtained.

[0093] As an example, the target audience for recommendation is the user using the terminal. Taking a video app as an example, the target audience for recommendation is the user using the video app on the terminal. The image is taken when the target audience is viewing information in the environment. For example, if user A is viewing information on the homepage of a video app on the subway, then the image is taken when user A is viewing information on the homepage of the video app in the subway environment. What is captured is either a video or a photo, that is, the image is a photo or a video frame from a video.

[0094] In step 102, the image is processed to obtain at least one of the state features of the object to be recommended and the environmental features of the environment.

[0095] In some embodiments, the image recognition processing in step 102 to obtain at least one of the state features of the object to be recommended and the environmental features of the environment can be achieved by performing at least one of the following technical solutions: performing state recognition processing on the image to obtain the state features of the object to be recommended; performing environmental recognition processing on the image to obtain the environmental features of the environment.

[0096] As an example, see Figure 8 , Figure 8 This is an image processing flowchart provided in an embodiment of this application. The prerequisite for image capture is user authorization. For example, in response to the authorization operation of the object to be recommended, the recommendation client permanently obtains the permission to take pictures of the user, or the recommendation client obtains the permission to take pictures of the user while the user is using the recommendation client, or the recommendation client obtains the permission to take pictures of the user when a recommendation request is triggered while the user is using the recommendation client. After the image is acquired, based on the expression recognition model (mood recognition model) and the scene recognition model (environment recognition model), the state features representing the user's mood state and the environmental features of the environment in which the user is browsing the recommendation APP can be obtained. The expression recognition model and the scene recognition model can be neural network models, which are trained through machine learning.

[0097] As an example, in the training process of an expression recognition model, image samples and pre-labeled states are input into the expression recognition model. The expression recognition model predicts the predicted state of the image samples. The error between the image samples and the pre-labeled states is used as the first loss. The parameters of the expression recognition model are updated with the goal of minimizing the first loss, thus obtaining the expression recognition model. In the training process of a scene recognition model, image samples and pre-labeled environments are input into the scene recognition model. The scene recognition model predicts the predicted environment of the image samples. The error between the image samples and the pre-labeled environments is used as the second loss. The parameters of the scene recognition model are updated with the goal of minimizing the second loss, thus obtaining the scene recognition model.

[0098] In step 103, target recommendation information for the object to be recommended is obtained, and multiple candidate display methods for the target recommendation information are generated.

[0099] In some embodiments, see Figure 4B , Figure 4B This is a flowchart illustrating the AI-based recommendation processing method provided in this application embodiment. Step 103, obtaining target recommendation information for the object to be recommended, can be achieved through steps 1031-1032, combining... Figure 4BSteps 1031-1032 are described below.

[0100] In step 1031, recommendation information for multiple candidates for the object to be recommended is obtained.

[0101] As an example, the candidate recommendation information can be at least one of the following: text, video, or image, and the candidate recommendation information is obtained through recall processing, ranking processing, or re-ranking processing.

[0102] In step 1032, target recommendation information that matches at least one of the state features and environmental features is determined from multiple candidate recommendation information.

[0103] In some embodiments, the above-mentioned determination of target recommendation information that matches at least one of the state features and environmental features from multiple candidate recommendation information can be achieved through the following technical solution: performing the following processing on each candidate recommendation information: obtaining the content features of the candidate recommendation information and the profile features of the object to be recommended; performing a first feature cross-processing on the content features of the candidate recommendation information, the profile features of the object to be recommended, and at least one of the state features and environmental features to obtain a first feature cross-processing result; performing logistic regression processing on the first feature cross-processing result to obtain a first recommendation index of the candidate recommendation information; and determining the candidate recommendation information corresponding to the first recommendation index that exceeds the threshold of the first recommendation index as the target recommendation information.

[0104] As an example, profile features are single or combined features related to user profile data. Single features can be user age or user occupation, while combined features can be combinations of different dimensions of user characteristics, such as a combination of user age and user occupation. Content features are single or combined features related to recommendation information. Single features can be the publisher of the recommendation information or the category of the recommendation information, while combined features can be combinations of different dimensions of the recommendation information. State features and environment features are obtained by recognizing images. State features represent the user's mood state, such as happy or sad, while environment features represent the environment in which the user is browsing the recommendation app, such as the environment of a subway or a library. The first feature cross processing is either Cartesian cross or factorization machine cross. During the first feature cross processing, the above features are processed in different ways. In the same way, the feature parts used in each combination are different or completely different to form multiple combined features of recommendation information. The first recommendation index factor of each combined feature of recommendation information is used as the weight parameter. The first recommendation index of recommendation information is obtained by weighted summation of each combined feature of recommendation information. The first recommendation index factor of the combined feature is the product of the first recommendation index correlation factors of the features included in the combined feature. The first recommendation index correlation factor is an updatable parameter during the training of the recommendation model. Logistic regression is performed on the cross-processing results of the first feature. Logistic regression can be linear processing or a combination of linear processing and logistic regression equation. For linear processing, the first recommendation index can be obtained by linearly combining the cross-processing results of each first feature. For example, click rate, see formula (1):

[0105]

[0106] Here, w0 is the bias value for bias processing, and w... i The result of cross-processing each first feature x i The corresponding first recommendation index factors.

[0107] In some embodiments, the linear combination can be substituted into the logistic regression equation (2) based on the above linear processing. In this way, the linear combination of features is used as the independent variable. Since the range of the independent variable is from negative infinity to positive infinity, the logistic regression equation is used to map the independent variable to (0, 1). The result obtained after substituting into the logistic regression equation is used as the first recommendation indicator, such as the click-through rate.

[0108]

[0109] Here, z is a linear combination of the features shown in formula (1), and g is the first recommendation index.

[0110] In some embodiments, the above-mentioned determination of the candidate recommendation information corresponding to the first recommendation index that exceeds the first recommendation index threshold as the target recommendation information can be achieved by the following technical solution: selecting multiple candidate recommendation information corresponding to the first recommendation index that exceeds the first recommendation index threshold; selecting at least one recommendation information that satisfies the diversity condition from the selected multiple candidate recommendation information as the target recommendation information; wherein, the diversity condition specifies the maximum number of target recommendation information belonging to the same category.

[0111] As an example, selecting at least one recommendation that meets the diversity condition from multiple candidate recommendations as the target recommendation can be achieved through the following technical solution: Multiple candidate recommendations form a first information set; the recommendation with the highest first recommendation index in the first information set is transferred to a second information set as the first piece of information in the second information set; when the number of recommendations in the second information set is less than an information number threshold, the semantic distance between the recommendation with the highest first recommendation index in the first information set and the first piece of information is determined; when the semantic distance is greater than a semantic distance threshold, the recommendation with the highest first recommendation index is transferred from the first information set to the second information set. The information is then updated to be the first piece of information in the second information set. The process of transferring information can be based solely on the first recommendation metric. For example, for 10,000 pieces of recommendation information, the first recommendation metric is predicted for each piece of information to obtain the corresponding first recommendation metric. Then, these 10,000 pieces of recommendation information are arranged from high to low according to the recommendation metric. The top 200 pieces of recommendation information with the highest recommendation metric can be selected as the head recommendation information. These pieces of recommendation information are then transferred to the second information set. The number of recommendation information in the first and second information sets can be set according to actual needs. Here, the information number threshold is used as the minimum value of the number of recommendation information in the second information set.

[0112] In some embodiments, when the number of recommended information in the second information set is less than the information number threshold, the deduplication function threshold here is the number of information in the second information set, that is, the number of information to be obtained after deduplication. If the number of information in the second information set is less than the information number threshold, it means that the number of information in the second information set has not reached the preset value, that is, the first information set still needs to be deduplicated.

[0113] In some embodiments, when the semantic distance is greater than a semantic distance threshold, the recommendation information with the highest first recommendation index is deleted from the first information set and added to the second information set, becoming the first piece of information in the second information set. The semantic distance between the recommendation information with the highest first recommendation index in the first information set and the first piece of information just updated to the second information set is then determined. This allows for the further filtering of recommendation information from the first information set that is dissimilar to the previous recommendation information transferred to the second information set, and these dissimilar recommendations are then transferred to the second information set. Here, similarity and dissimilarity are defined by a semantic distance threshold. When the semantic distance between two pieces of information is greater than the threshold, the two pieces of information are determined to be dissimilar. When the semantic distance between two pieces of information is not greater than the threshold, the two pieces of information are determined to be similar, i.e., belonging to the same category, such as both belonging to the game category, or both belonging to the category of celebrity A, etc. The category hierarchy can be determined according to the actual situation.

[0114] In some embodiments, see Figure 4C , Figure 4C This is a flowchart illustrating the AI-based recommendation processing method provided in this application embodiment. The multiple candidate display methods for generating target recommendation information in step 103 can be implemented through steps 1033-1036, combining... Figure 4C Steps 1033-1036 are described below.

[0115] In step 1033, at least one candidate template for the target recommendation information is generated.

[0116] In some embodiments, candidate templates can be generated based on target recommendation information. For example, if the target recommendation information is a new video, then candidate templates corresponding to the new video are generated, and the cover image occupies a larger proportion in the candidate templates corresponding to the new video compared to other candidate templates.

[0117] In step 1034, at least one candidate summary of the target recommendation information is generated.

[0118] In some embodiments, generating at least one candidate summary of the target recommendation information in step 1034 can be achieved by performing at least one of the following processes: extracting key content of the target recommendation information and generating candidate summaries based on the key content; identifying interactive objects that have interactive behavior with the target recommendation information and generating candidate summaries based on the interactive objects; identifying associated information whose relevance to the target recommendation information exceeds a relevance threshold and generating candidate summaries based on the associated information.

[0119] As an example, the process involves extracting key content from the target recommendation information and generating candidate summaries based on this content. For instance, for movie A, candidate summaries are generated based on the actor information of movie A. It also involves identifying interactive objects that have interacted with the target recommendation information and generating candidate summaries based on these interactive objects. For example, if user A and user B have commented on movie A, candidate summaries are generated based on user A and user B to represent that user A and user B have commented on movie A. Finally, it involves identifying related information whose relevance to the target recommendation information exceeds a relevance threshold and generating candidate summaries based on this related information. For instance, if the relevance between advertisement A and movie A exceeds a relevance threshold, candidate summaries are generated based on advertisement A to recommend advertisement A to users using these candidate summaries.

[0120] In step 1035, at least one candidate cover for the target recommendation information is generated;

[0121] In some embodiments, when the target recommendation information is a recommended video, generating at least one candidate cover for the target recommendation information in step 1035 can be achieved by the following technical solution: dividing the recommended video into multiple shots, wherein each shot includes multiple consecutive video frames; performing the following processing for each shot: performing clustering processing on the multiple video frames of the shot to obtain multiple clusters under the shot; determining the video frame in each cluster that is closest to the corresponding cluster center, and determining the closest video frame as the candidate cover.

[0122] In some embodiments, the above-mentioned segmentation of the recommended video into multiple shots can be achieved through the following technical solution: For each video frame of the recommended video, the following processing is performed: determining the position of each pixel in the video frame; forming a segmentation unit from any two consecutive video frames, and performing the following processing for each segmentation unit of the recommended video: determining the grayscale difference of pixels at the same position in the two video frames of the segmentation unit, and averaging the grayscale differences of pixels at multiple positions to obtain the difference of the segmentation unit; determining that the segmentation unit corresponding to the difference greater than the difference threshold has a shot boundary, and dividing the two video frames of the segmentation unit into different shots.

[0123] As an example, the video is divided into multiple shots, mainly to find the boundaries of the shots. A shot depicts an event or continuous action in the same scene. Therefore, when the shot changes, the changes between video frames will be relatively large, while the differences between video frames within the same shot will be relatively small. Therefore, the difference between two adjacent video frames can be used to determine whether two video frames belong to the same shot. The difference between two video frames can be represented by the grayscale difference, see formula (3):

[0124]

[0125] Where Dis (difference between segmentation units) is the grayscale difference between video frame I1 and video frame I2, M is the number of pixels in video frame I1 and video frame I2, x and y are the positions of the corresponding pixels in video frame I1 and video frame I2, I1(x,y)-I2(x,y) is the grayscale difference between pixels at the same position in the two video frames, and the difference between the two frames can also be calculated by comparing the histograms and edges of the two frames.

[0126] In some embodiments, the above-mentioned clustering processing of multiple video frames of a shot to obtain multiple clusters under the shot can be implemented by the following technical solution: forming a video frame set from multiple consecutive video frames; randomly selecting N video frames from the video frame set, using the image features of the corresponding N video frames as the initial cluster centers of the multiple cluster sets, and removing the N video frames from the video frame set, where N is the number of candidate covers for the corresponding shot, and N is an integer greater than or equal to 2; initializing the number of iterations of the clustering processing to M, and establishing an empty set corresponding to each cluster, where M is an integer greater than or equal to 2; executing the following in each iteration of the clustering processing: The following processing is performed: The set of each cluster is updated, and cluster center generation is performed based on the update result to obtain new cluster centers for each cluster. When the new cluster centers differ from the initial cluster centers, the video frames corresponding to the initial cluster centers are added back to the video frame set, and the initial cluster centers are updated based on the new cluster centers. The set of each cluster obtained after M iterations is determined as the clustering result, or the set of each cluster obtained after m iterations is determined as the clustering result. Wherein, the cluster centers of multiple clusters obtained after m iterations are the same as those of multiple clusters obtained after m-1 iterations, where m is an integer variable and its value satisfies 2≤m≤M.

[0127] As an example, the above process of updating the set of each cluster and generating cluster centers based on the update results can be achieved through the following technical solution: For each video frame in the video frame set, perform the following processing: determine the similarity between the image features of the video frame and the initial cluster centers of each cluster; determine the initial cluster center corresponding to the maximum similarity as belonging to the same cluster as the video frame, and transfer the video frame to the set of clusters corresponding to the initial cluster center with the maximum similarity, where the initial cluster center with the maximum similarity is the initial cluster center corresponding to the maximum similarity; average the image features of each video frame in the set of each cluster to obtain the new cluster centers of each cluster.

[0128] Continuing the example above, given 30 video frames and N = 2, the goal of clustering is to divide these 30 video frames into two clusters. Each cluster has a corresponding set, and each set contains the video frames belonging to that cluster. First, two video frames are randomly selected based on their image features as the initial cluster centers for each of the two clusters. For the remaining 28 video frames, the similarity between each video frame and the two initial cluster centers is calculated, for example, using L2 distance to evaluate similarity. For instance, for video frame E, its image features are closer to the initial cluster center a, so video frame E is assigned to the set of the cluster corresponding to initial cluster center a. After performing the assignment operation for all 28 video frames, a new cluster center is recalculated for each cluster. If the new cluster centers of two clusters are the same... Alternatively, if the similarity between the new cluster center and the initial cluster center is greater than the similarity threshold, each set can be directly determined as the clustering result. If the new cluster centers of the two clusters are different, and the similarity between the new cluster center and the initial cluster center is not greater than the similarity threshold, then the initial cluster center is updated using the new cluster center, and the allocation operation is performed again for video frames other than those corresponding to the cluster center. For example, after the original initial cluster center 'a' is replaced by the new cluster center, the video frame corresponding to the original initial cluster center 'a' belongs to a video frame other than those corresponding to the cluster center, and needs to participate in the allocation process again until the new cluster centers of the two clusters are the same, or the similarity between the new cluster center and the initial cluster center is greater than the similarity threshold, or the specified number of iterations has been completed.

[0129] In some embodiments, the above-mentioned determination of the video frame closest to the corresponding cluster center in each cluster can be achieved by the following technical solution: performing the following processing for each cluster: averaging the image features of each video frame in the cluster to obtain the cluster center; determining the video frame closest to the cluster center based on the distance between the image features of each video frame in the cluster and the cluster center.

[0130] As an example, for a certain cluster A, there are 10 video frames. The image features of the 10 video frames of cluster A are averaged to obtain the cluster center. Based on the distance between the image features of each video frame of cluster A and the cluster center, the video frame closest to the cluster center is determined. For example, if the image features of video frame A are closest to the cluster center, then video frame A is determined as the video frame closest to the cluster center.

[0131] In step 1036, at least one candidate template, at least one candidate summary, and at least one candidate cover are combined in different ways to obtain multiple candidate display methods.

[0132] As an example, the candidate summary includes at least one of the following: the title of the target recommendation information, the reason for recommending the target recommendation information, and the introduction of the target recommendation information. Each of the multiple candidate display methods includes a candidate template, a candidate summary, and a candidate cover, and the multiple candidate display methods are different.

[0133] In some embodiments, at least one candidate summary and at least one candidate cover can be generated, and then the at least one candidate summary and at least one candidate cover can be combined in different ways to obtain multiple candidate display methods.

[0134] In step 104, a target display mode that is adapted to at least one of the state features and environmental features is determined from a plurality of candidate display modes.

[0135] In some embodiments, determining the target display mode that matches at least one of the state features and environment features from multiple candidate display modes in step 104 can be achieved through the following technical solution: performing the following processing for each candidate display mode: extracting text features and image features of the candidate display mode; performing a second feature cross-processing on the text features, image features, portrait features, and at least one of the state features and environment features of the candidate display mode to obtain a second feature cross-processing result; performing logistic regression processing on the second feature cross-processing result to obtain a second recommendation index for the candidate display modes of the target recommendation information. The candidate display mode corresponding to the second recommendation index that exceeds the second recommendation index threshold is determined as the target display mode.

[0136] As an example, profile features are single or combined features related to user profile data. Single features can be user age or user occupation, while combined features can be combinations of different dimensions of the user, such as a combination of user age and user occupation. State features and environment features are obtained by recognizing images. State features represent the user's mood state, such as happy or sad, while environment features represent the environment in which the user is browsing the recommendation app, such as the environment of a subway or a library. The first feature cross-processing is either Cartesian cross-processing or factorization machine cross-processing. In the first feature cross-processing, the above features are combined in different ways, and the feature components used in each combination are different or completely different, in order to form a recommendation. The multiple combined features of the recommendation information are weighted by the second recommendation index factor of each combined feature of the recommendation information. The weighted summation of each combined feature of the recommendation information is used to obtain the second recommendation index of the recommendation information. The second recommendation index factor of the combined feature is the product of the second recommendation index correlation factors of the features included in the combined feature. The second recommendation index correlation factor is an updatable parameter during the training of the recommendation model. Logistic regression is performed on the cross-processing results of the second feature. Logistic regression can be linear processing or a combination of linear processing and logistic regression equation. For linear processing, the second recommendation index can be obtained by linearly combining the cross-processing results of each second feature. For example, click rate, see formula (4):

[0137]

[0138] Here, w0 is the bias value for bias processing, and w... i The result of cross-processing each second feature x i The corresponding second recommendation index factors.

[0139] In some embodiments, the linear combination can be substituted into the logistic regression equation (5) based on the above linear processing. In this way, the linear combination of features is used as the independent variable. Since the range of the independent variable is from negative infinity to positive infinity, the logistic regression equation is used to map the independent variable to (0, 1). The result obtained after substituting into the logistic regression equation is used as the second recommendation indicator, such as the click-through rate.

[0140]

[0141] Here, z is a linear combination of the features shown in formula (4), and g is the second recommendation index.

[0142] In step 105, the target recommendation information is displayed according to the target display method.

[0143] See Figure 4D , Figure 4DThis is a flowchart illustrating the AI-based recommendation processing method provided in the embodiments of this application, which will be combined with... Figure 4D The steps shown are explained.

[0144] In step 201, an image of the object to be recommended is acquired, wherein the image is captured when the object to be recommended is viewing information in the environment.

[0145] As an example, an image is a photograph or video frame taken when the object to be recommended is viewing information in its environment.

[0146] In step 202, the image is processed for recognition to obtain at least one of the state features of the object to be recommended and the environmental features of the environment.

[0147] In step 203, recommendation information for multiple candidates for the object to be recommended is obtained;

[0148] In step 204, target recommendation information that matches at least one of the state features and environmental features is determined from multiple candidate recommendation information;

[0149] In step 205, target recommendation information is displayed.

[0150] As an example, the implementation of steps 201-202 can refer to the implementation of steps 101-102, and the implementation of steps 203-205 can refer to the implementation of steps 1031-1032.

[0151] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0152] In some embodiments, when the recommendation processing system is applied to a video recommendation scenario, the terminal acquires an image of the object to be recommended, wherein the image is captured when the object is watching a video in the environment; the terminal sends the image to the server, which performs recognition processing on the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment; the server acquires a target recommendation video for the object to be recommended and generates multiple candidate display methods for the target recommendation video; the server determines a target display method that matches at least one of the state features and environmental features from the multiple candidate display methods; the server returns the target display method and the target recommendation video to the terminal, and the target recommendation video is displayed on the terminal according to the target display method. See [link to documentation]. Figure 10 , Figure 10This is a product interface diagram of the AI-based recommendation processing method provided in this application embodiment. Responding to a user's trigger operation on target display information 501, a playback page 502 of the target recommendation information is displayed. The display method of target display information 501 is personalized; different covers, titles, etc., can be displayed for different users. Through the recommendation processing method provided in this application embodiment, the terminal captures the user's expression and environment, and combines this with the user profile to personalize the display of recommended content. Personalized covers, personalized content descriptions, personalized titles, and personalized recommendation reasons are displayed for different users. For example, for video A, if the user's expression is happy, a brightly colored cover with a humorous title is displayed; if the user's expression is unhappy, a darkly colored cover with a simple title is displayed. For example, when recommended to advanced players, the title emphasizes the excitement of the gameplay; when recommended to casual players, the content description highlights the fun of the content. For example, when recommended to players using hero A, the cover image can capture hero A's performance in the video; when recommended to players using hero B, the cover image should highlight hero B, thus attracting users to click and watch video A.

[0153] See Figure 5 , Figure 5 This is a flowchart illustrating the recommendation processing method provided in this application embodiment. First, feature engineering is used to extract user profiles and video content features. Then, user photos are acquired in real time (after user authorization), and state and environmental features (features of user photos) are extracted from the user photos. Historical data is used to train a recommendation model. Based on the recommendation model, recommended videos corresponding to the user are obtained. Historical data is used to train a video display model. The video display model is used to generate personalized display content based on the features of user photos, user profiles, and recommended videos, and the personalized display content (cover image, title, introduction, recommendation reasons, etc.) is pushed to the user.

[0154] The core idea of ​​this application's embodiments is to improve the click-through rate and accuracy of video recommendations by automatically generating recommended video display methods. See [link to relevant documentation]. Figure 6 , Figure 6This is a flowchart illustrating the recommendation processing method provided in this application. First, a personalized recommended video for each user is obtained through a recommendation system. For the video text data, multiple text summaries (titles, descriptions, etc.) can be generated from different perspectives. These can be used as candidate summaries and can be automatically generated using machine learning methods. For the same video, information in a certain dimension can be highlighted. For example, for a movie-related video, a content summary can be generated by editing based on the following dimensions: a) Information dimension: for example, highlighting the actors in the movie; b) User dimension: for example, highlighting that the video has been viewed by multiple friends; c) Dissemination dimension: for example, highlighting the popularity of the related movie. For the cover of the display mode, images or videos are extracted from the recommended videos as candidate covers. The user's user profile and current photo are obtained. Based on the current photo, state features and environmental features are obtained. Then, a personalized display mode is generated through a video display model (machine learning model). When using a machine learning model to select the final display mode, machine learning algorithms other than logistic regression models, such as neural networks and random forests, can be selected, mainly based on the data distribution and the actual scenario.

[0155] In some embodiments, see Figure 7 , Figure 7 This is a schematic diagram of keyframe extraction provided in this application. For candidate covers of display modes, multiple keyframes are extracted from the video as candidates for the cover. This requires the use of video keyframe extraction technology. Using the idea of ​​clustering, the video is first divided into multiple shots. Then, image features (e.g., color, histogram, shape, motion, etc.) are extracted for each frame in the shot. Then, the K-Means algorithm is used to divide all video frames under the shot into K classes. Finally, the video frame closest to the cluster center in each class is selected as the keyframe of that class. Dividing the video into multiple shots is mainly to find the boundaries of the shots. A shot depicts an event or continuous action in the same scene. Therefore, when the shot changes, the changes between video frames will be relatively large, while the differences between video frames within the same shot will be relatively small. Therefore, the difference between two adjacent video frames can be used to determine whether two video frames belong to the same shot. The difference between two video frames can be represented by the grayscale difference, see formula (6):

[0156]

[0157] Where Dis is the grayscale difference between video frame I1 and video frame I2, M is the number of pixels in video frame I1 and video frame I2, and x and y are the positions of the corresponding pixels in video frame I1 and video frame I2. The difference between the two frames can also be calculated by comparing the histograms and edges of the two frames.

[0158] In some embodiments, other keyframe extraction methods can also be used when extracting keyframes from a video. For example, a video content-based method can be used to extract the scene of an actor as a keyframe. Different content recognition models can be trained for different types of videos, and then keyframes can be extracted through the content recognition model.

[0159] In some embodiments, see Figure 8 , Figure 8 This is an image processing flowchart provided in an embodiment of this application. The prerequisite for image capture is user authorization. For example, in response to the authorization operation of the object to be recommended, the recommendation client permanently obtains the permission to take pictures of the user, or the recommendation client obtains the permission to take pictures of the user while the user is using the recommendation client, or the recommendation client obtains the permission to take pictures of the user when a recommendation request is triggered while the user is using the recommendation client. After the image is acquired, based on the expression recognition model and the scene recognition model, the state features representing the user's mood state and the environmental features of the environment in which the user is browsing the recommendation APP can be obtained to form the photo features of the video currently being watched by the user. The expression recognition model and the scene recognition model can be neural network models, which are trained through machine learning.

[0160] In some embodiments, see Figure 9 , Figure 9 This is a schematic diagram of the display processing provided in the embodiments of this application. Based on a machine learning model, the most suitable text and images can be selected from candidates of text and images as the displayed content according to the user's state features, environmental features, and user profile. The logistic regression model used in the recommendation system can be reused to extract profile features, state features, and environmental features from the user, extract text features from the candidate text (such as bag-of-words features, article features, etc.), and extract image features such as color and histogram from the candidate images. Then, the above features are cross-processed, and the logistic regression model is used to predict whether the user will click on a certain display mode of the recommended video. The output of the logistic regression model is the probability that the user will click on a certain display mode of the recommended video. Therefore, for each candidate display mode of the recommended video, the probability of whether the user will click on it can be obtained. The display mode with the highest probability is selected as the final result. The training process of the logistic regression model is as follows: First, training data D is collected. The training data D consists of multiple sets of data, such as (x 1 y 1 ), (x 2 y 2 ), ..., (x N y N), where x represents a multi-dimensional feature vector (including user-side features and video-side features), y represents whether the user clicks on the candidate display method of the recommended video, 1 for clicking and 0 for not clicking, the training objective is to minimize the loss function, the loss function is shown in formula (7):

[0161]

[0162] Where P(y=|x;θ) is the predicted click rate, x is a multidimensional feature vector, and the goal of training is to find a set of parameters θ that can minimize the loss function, and then apply this set of parameters to predict new data.

[0163] The personalized video display scheme based on user photos proposed in this application can better attract users, thereby improving the click-through rate and accuracy of video recommendations.

[0164] The following description continues to illustrate the exemplary structure of the AI-based recommendation processing devices 455-1 and 455-2 provided in the embodiments of this application as software modules. In some embodiments, such as Figure 3 As shown, the software modules in the AI-based recommendation processing device 455-1 stored in the memory 450 may include: a capture module 4551-1, used to acquire an image of the object to be recommended, wherein the image is captured when the object to be recommended is viewing information in the environment; a recognition module 4552-1, used to perform recognition processing on the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment; a recommendation module 4553-1, used to acquire target recommendation information for the object to be recommended and generate multiple candidate display methods for the target recommendation information; a display module 4554-1, used to determine the target display method that matches at least one of the state features and environmental features from the multiple candidate display methods; and the display module 4554-1 is also used to display the target recommendation information according to the target display method.

[0165] In some embodiments, the recommendation module 4553-1 is further configured to: obtain recommendation information for multiple candidates of the object to be recommended; and determine target recommendation information that is adapted to at least one of the state features and environmental features from the multiple candidate recommendation information.

[0166] In some embodiments, the recommendation module 4553-1 is further configured to: perform the following processing for each candidate recommendation information: obtain the content features of the candidate recommendation information and the profile features of the object to be recommended; perform a first feature cross-processing on the content features of the candidate recommendation information, the profile features of the object to be recommended, and at least one of the state features and environment features to obtain a first feature cross-processing result; perform logistic regression processing on the first feature cross-processing result to obtain a first recommendation index of the candidate recommendation information; and determine the candidate recommendation information corresponding to the first recommendation index that exceeds the threshold of the first recommendation index as the target recommendation information.

[0167] In some embodiments, the recommendation module 4553-1 is further configured to: select multiple candidate recommendation information corresponding to a first recommendation index that exceeds a first recommendation index threshold; select at least one recommendation information that satisfies the diversity condition from the selected multiple candidate recommendation information as the target recommendation information; wherein the diversity condition specifies the maximum number of target recommendation information belonging to the same category.

[0168] In some embodiments, the recommendation module 4553-1 is further configured to: generate at least one candidate template for the target recommendation information; generate at least one candidate summary for the target recommendation information; generate at least one candidate cover for the target recommendation information; and combine at least one candidate template, at least one candidate summary, and at least one candidate cover in different ways to obtain multiple candidate display methods; wherein the candidate summary includes at least one of the following: the title of the target recommendation information, the reason for recommending the target recommendation information, and the introduction of the target recommendation information.

[0169] In some embodiments, the recommendation module 4553-1 is further configured to: perform at least one of the following processes: extract key content of the target recommendation information and generate candidate summaries based on the key content; determine interactive objects that have interactive behavior with the target recommendation information and generate candidate summaries based on the interactive objects; determine associated information whose association degree with the target recommendation information exceeds the association degree threshold and generate candidate summaries based on the associated information.

[0170] In some embodiments, the recommendation module 4553-1 is further configured to: when the target recommendation information is a recommended video, segment the recommended video into multiple shots, wherein each shot includes multiple consecutive video frames; perform the following processing for each shot: perform clustering processing on the multiple video frames of the shot to obtain multiple clusters under the shot; determine the video frame in each cluster that is closest to the corresponding cluster center, and determine the closest video frame as a candidate cover.

[0171] In some embodiments, the recommendation module 4553-1 is further configured to: perform the following processing for each video frame of the recommended video: determine the position of each pixel in the video frame; form a segmentation unit from any two consecutive video frames, and perform the following processing for each segmentation unit of the recommended video: determine the grayscale difference of pixels at the same position in the two video frames of the segmentation unit, and average the grayscale differences of pixels at multiple positions to obtain the difference of the segmentation unit; determine that the segmentation unit corresponding to the difference greater than the difference threshold has a shot boundary, and divide the two video frames of the segmentation unit into different shots.

[0172] In some embodiments, the recommendation module 4553-1 is further configured to: form a video frame set from multiple consecutive video frames; randomly select N video frames from the video frame set, use the image features of the corresponding N video frames as the initial cluster centers of multiple cluster sets, and remove the N video frames from the video frame set, where N is the number of candidate covers for the corresponding shot, and N is an integer greater than or equal to 2; initialize the number of iterations of the clustering process to M, and establish an empty set corresponding to each cluster, where M is an integer greater than or equal to 2; and perform the following processing in each iteration of the clustering process: for each cluster set An update process is performed, and based on the update result, a cluster center generation process is executed to obtain a new cluster center for each cluster. When the new cluster center is different from the initial cluster center, the video frame corresponding to the initial cluster center is added back to the video frame set, and the initial cluster center is updated based on the new cluster center. The set of each cluster obtained after M iterations is determined as the clustering result, or the set of each cluster obtained after m iterations is determined as the clustering result. Among them, the cluster centers of multiple clusters obtained after m iterations are the same as those of multiple clusters obtained after m-1 iterations, where m is an integer variable and its value satisfies 2≤m≤M.

[0173] In some embodiments, the recommended module 4553-1 is further configured to: perform the following processing for each cluster: average the image features of each video frame of the cluster to obtain the cluster center; and determine the video frame closest to the cluster center based on the distance between the image features of each video frame of the cluster and the cluster center.

[0174] In some embodiments, the recognition module 4552-1 is further configured to perform at least one of the following processes: perform state recognition processing on the image to obtain the state features of the object to be recommended; perform environment recognition processing on the image to obtain the environment features of the environment.

[0175] In some embodiments, the display module 4554-1 is configured to perform the following processing for each candidate display method: extracting text features and image features of the candidate display method; performing a second feature cross-processing on the text features, image features, portrait features, and at least one of state features and environment features of the candidate display method to obtain a second feature cross-processing result; performing logistic regression processing on the second feature cross-processing result to obtain a second recommendation index for the candidate display methods of the target recommendation information. The candidate display method corresponding to the second recommendation index that exceeds the second recommendation index threshold is determined as the target display method.

[0176] In some embodiments, such as Figure 3 As shown, the software modules stored in the AI-based recommendation processing device 455-2 in the memory 450 may include: a capture module 4551-2 for acquiring an image of the object to be recommended, wherein the image is captured when the object is viewing information in the environment; a recognition module 4552-2 for performing recognition processing on the image to obtain at least one of the state features of the object to be recommended and the environmental features of the environment; a recommendation module 4553-2 for acquiring multiple candidate recommendation information for the object to be recommended; the recommendation module 4553-2 is also used to determine the target recommendation information that matches at least one of the state features and environmental features from the multiple candidate recommendation information; and a display module 4554-2 for displaying the target recommendation information.

[0177] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the artificial intelligence-based recommended processing method described above in this application.

[0178] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, the processor will execute the AI-based recommendation processing method provided in this application, for example... Figures 4A-4D The example shown is an AI-based recommendation processing method.

[0179] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0180] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0181] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0182] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0183] In summary, this application's embodiments obtain images of the user viewing information in their environment, thereby acquiring at least one of the user's state characteristics and environmental characteristics. Subsequently, a target display method that matches at least one of the state characteristics and environmental characteristics is determined from multiple candidate display methods, and the target recommendation information is displayed according to the target display method. Since the target display method is related to at least one of the state characteristics and environmental characteristics, and the state characteristics and environmental characteristics can directly characterize the user's performance and environment during the real-time recommendation process, the target display method can be accurately derived, effectively improving the user's click-through rate, thereby improving recommendation accuracy and efficiency.

[0184] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A recommendation processing method based on artificial intelligence, characterized by, The method comprises: acquiring an image of a to-be-recommended object, wherein the image is captured when the to-be-recommended object views information in an environment; performing identification processing on the image to obtain state features of the to-be-recommended object and environment features of the environment; performing first feature cross processing on content features of candidate recommended information, portrait features of the to-be-recommended object, and the state features and the environment features to obtain a first feature cross processing result; performing logistic regression processing on the first feature cross processing result to obtain a first recommendation index of the candidate recommended information, and determining the candidate recommended information corresponding to the first recommendation index that exceeds a first recommendation index threshold as target recommended information; generating a plurality of candidate display modes of the target recommended information; for each of the candidate display modes, performing the following processing: extracting text features and image features of the candidate display mode, and performing second feature cross processing on the text features and image features of the candidate display mode, the portrait features, and the state features and the environment features to obtain a second feature cross processing result; performing logistic regression processing on the second feature cross processing result to obtain a second recommendation index of the candidate display mode of the target recommended information; determining the candidate display mode corresponding to the second recommendation index that exceeds a second recommendation index threshold as a target display mode; wherein the first feature cross processing is Cartesian cross or factorization machine cross, and the second feature cross processing is Cartesian cross or factorization machine cross; displaying the target recommended information in the target display mode.

2. The method of claim 1, wherein, The determining of the candidate recommended information corresponding to the first recommendation index that exceeds the first recommendation index threshold as the target recommended information comprises: selecting a plurality of candidate recommended information corresponding to the first recommendation index that exceeds the first recommendation index threshold; selecting at least one recommended information that satisfies a diversity condition from the selected plurality of candidate recommended information as the target recommended information; wherein the diversity condition specifies a maximum number of target recommended information belonging to the same category.

3. The method of claim 1, wherein, The generating of the plurality of candidate display modes of the target recommended information comprises: generating at least one candidate template of the target recommended information; generating at least one candidate abstract of the target recommended information; generating at least one candidate cover of the target recommended information; combining the at least one candidate template, the at least one candidate abstract, and the at least one candidate cover in different ways to obtain the plurality of candidate display modes; wherein the candidate abstract comprises at least one of the following: a title of the target recommended information, a recommendation reason of the target recommended information, and an introduction of the target recommended information.

4. The method of claim 3, wherein, The generating of the at least one candidate abstract of the target recommended information comprises: performing at least one of the following processing: extracting key content of the target recommended information, and generating the candidate abstract according to the key content; determining an interactive object that has an interactive behavior with the target recommended information, and generating the candidate abstract according to the interactive object; determine association information associated with the target recommendation information and having a degree of association exceeding a degree of association threshold, and generate the candidate summary based on the association information.

5. The method of claim 3, wherein, When the target recommendation information is a recommended video, the generating of the at least one candidate cover of the target recommendation information comprises: segmenting the recommended video into a plurality of shots, wherein each of the shots comprises a plurality of continuous video frames; performing the following processing on each of the shots: performing clustering processing on the plurality of video frames of the shot to obtain a plurality of clusters under the shot; determining a video frame closest to a corresponding cluster center in each of the clusters, and determining the closest video frame as the candidate cover.

6. The method of claim 5, wherein, The segmenting of the recommended video into a plurality of shots comprises: performing the following processing on each video frame of the recommended video: determining the position of each pixel point of the video frame; grouping any two continuous video frames into a segmentation unit, and performing the following processing on each segmentation unit of the recommended video: determining the gray scale difference of the pixel points at the same position of the two video frames of the segmentation unit, and performing average processing on the gray scale differences of the pixel points at a plurality of positions to obtain the difference value of the segmentation unit; determining that the segmentation unit corresponding to the difference value greater than a difference value threshold has a shot boundary, and dividing the two video frames of the segmentation unit into different shots.

7. The method of claim 5, wherein, The clustering processing on the plurality of video frames of the shot to obtain a plurality of clusters under the shot comprises: grouping the plurality of continuous video frames into a video frame set; randomly selecting N video frames from the video frame set, taking the image features corresponding to the N video frames as initial cluster centers of a plurality of cluster sets, and removing the N video frames from the video frame set, wherein N is the number of candidate covers corresponding to the shot, and N is an integer greater than or equal to 2; initializing the number of iterations of the clustering processing as M, and establishing an empty set corresponding to each cluster, wherein M is an integer greater than or equal to 2; performing the following processing in each iteration process of the clustering processing: updating each cluster set, and performing cluster center generation processing based on the update result to obtain a new cluster center of each cluster, when the new cluster center is different from the initial cluster center, adding the video frame corresponding to the initial cluster center to the video frame set again, and updating the initial cluster center based on the new cluster center; determining each cluster set obtained after M iterations as the clustering processing result, or determining each cluster set obtained after m iterations as the clustering processing result; wherein the cluster centers of the plurality of clusters obtained after m iterations are the same as the cluster centers of the plurality of clusters obtained after m-1 iterations, m is an integer variable and takes a value satisfying 2≤m≤M.

8. The method of claim 1, wherein, The recognition processing on the image to obtain at least one of the state feature of the object to be recommended and the environment feature of the environment comprises: performing at least one of the following processing: performing state recognition processing on the image to obtain the state feature of the object to be recommended; performing environment recognition processing on the image to obtain the environment feature of the environment. 9.A recommendation processing apparatus based on artificial intelligence, characterized by The method comprises the following steps: a photographing module is configured to acquire an image of a to-be-recommended object, wherein the image is obtained by photographing when the to-be-recommended object views information in an environment; an identification module is configured to perform identification processing on the image to obtain state features of the to-be-recommended object and environment features of the environment; a recommendation module is configured to perform first feature cross processing on content features of candidate recommendation information, portrait features of the to-be-recommended object, and the state features and the environment features to obtain a first feature cross processing result; performing logistic regression processing on the first feature cross processing result to obtain a first recommendation index of the candidate recommendation information, and determining the candidate recommendation information corresponding to the first recommendation index that exceeds a first recommendation index threshold as target recommendation information; generating a plurality of candidate display modes of the target recommendation information; a display module is configured to perform the following processing for each candidate display mode: extracting text features and image features of the candidate display mode; performing second feature cross processing on the text features and image features of the candidate display mode, the portrait features, and the state features and the environment features to obtain a second feature cross processing result; performing logistic regression processing on the second feature cross processing result to obtain a second recommendation index of the candidate display mode of the target recommendation information; and determining the candidate display mode corresponding to the second recommendation index that exceeds a second recommendation index threshold as a target display mode; wherein the first feature cross processing is Cartesian cross or factorization machine cross, and the second feature cross processing is Cartesian cross or factorization machine cross; the display module is further configured to display the target recommendation information in the target display mode.

10. An electronic device, comprising: The method comprises the following steps: a memory is configured to store executable instructions; a processor is configured to execute the executable instructions stored in the memory to implement the recommendation processing method based on artificial intelligence according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer executable instructions or the computer program are executed by the processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer-executable instructions or a computer program, characterized in that, ​

Images