Image recommendation method and electronic equipment

Abstract

The invention provides an image recommendation method and electronic equipment, relates to the technical field of terminals, and aims to solve the problems of low efficiency, poor quality and poor relevance of an image selection mode. The method comprises the following steps: receiving first content input by a user and a first text of the first content; displaying a first recommendation result according to the image feature of the to-be-recommended image corresponding to the first content and the text feature of the first text; the first recommendation result comprises a first recommendation image; updating the image features and the text features in response to an updating operation of the user on the first recommendation result; displaying a second recommendation result according to the updated image features and the updated text features; the second recommendation result comprises a second recommendation image, and the second recommendation result is different from the first recommendation result.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image recommendation method and electronic device. Background Technology

[0002] With the rapid development of self-media, creators can publish videos or text-based content through content creation platforms. When publishing content, creators usually need to select a cover image to attract users. However, current methods for selecting cover images suffer from problems such as inefficiency, poor quality, and weak relevance to the content title. Summary of the Invention

[0003] This application provides an image recommendation method and electronic device. The terminal intelligently recommends images based on image features and text features, which improves image selection efficiency, enhances the quality and relevance of cover images, and strengthens the user experience.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, an image recommendation method is provided, comprising: receiving first content and first text of the first content input by a user; displaying a first recommendation result based on image features of an image to be recommended corresponding to the first content and text features of the first text; the first recommendation result including a first recommended image; updating the image features and text features in response to an update operation by the user on the first recommendation result; displaying a second recommendation result based on the updated image features and updated text features; the second recommendation result including a second recommended image, and the second recommendation result being different from the first recommendation result.

[0006] In this application, the terminal determines a first recommendation result based on first content and first text of the first content, according to the image features of the image to be recommended corresponding to the first content and the text features of the first text. By analyzing the image features and text features, a comprehensive understanding of user needs can be achieved, ensuring that the recommendation results match user needs, thereby improving recommendation efficiency, quality, and relevance. Furthermore, the image features and text features can be updated based on user actions, enabling more accurate recommendations, enhancing user experience, and increasing user satisfaction.

[0007] According to the first aspect, the first content can be a user-created multimedia file. The first content can include one or more multimedia files such as video, images, text, and audio. For example, the first content can be creative content, learning materials, meeting materials, etc.

[0008] In some examples, the first text of the first content can be text information describing the first content. For example, the content title, a brief description, or tags of the first content. Alternatively, if the first content includes text, that text can be used as the first text of the first content.

[0009] In some examples, the first text of the first content can also be the keyword information of the first content. For example, keywords can be extracted from the content title, the description, or the tags of the first content to obtain the first text of the first content. Alternatively, if the first content includes text, keywords can be extracted from that text to obtain the first text of the first content.

[0010] The number of images to be recommended can be multiple, or it can be one.

[0011] According to the first aspect, or any implementation of the first aspect above, displaying the first recommendation result based on the image features of the image to be recommended corresponding to the first content and the text features of the first text includes: obtaining the image features of the image to be recommended and the text features of the first text; determining the relevance between the image to be recommended and the first text based on the image features of the image to be recommended and the text features of the first text, and determining the first recommended image based on the relevance; determining the click-through rate of the first recommended image based on the image features of the first recommended image and the text features of the first text; and determining the first recommendation result based on the relevance between the first recommended image and the first text and the click-through rate of the first recommended image.

[0012] In this application, relevance and click-through rate are calculated based on image features and text features. Based on the relevance and click-through rate, the user is guided to select a cover image. Thus, the terminal can assist the user in selecting an image that is highly relevant to the first text and has a high potential click-through rate as the cover image, thereby improving the quality of the cover image and the click-through rate of the first content, and also improving the user experience.

[0013] According to the first aspect, or any implementation of the first aspect above, the image features of the image to be recommended and the text features of the first text are obtained, including: obtaining the image to be recommended corresponding to the first content; encoding the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended; encoding the first text to obtain the original text features and the original high-dimensional features of the first text; determining the original image features as the image features of the image to be recommended, and determining the original text features as the text features of the first text.

[0014] In this application, core information of images and text can be accurately captured based on image and text features. Based on the acquired features, subsequent feature analysis and correlation calculations can be performed to recommend highly relevant images. Furthermore, image features contain comprehensive information about the image, and text features contain comprehensive information about the text, which is beneficial for subsequent iterative feature updates. High-dimensional image features can introduce more dimensions to represent the image, and high-dimensional text features can introduce more dimensions to represent the text, enhancing the expressive power and richness of the features. This facilitates accurate capture of correlations between features during subsequent iterative updates, thereby improving the accuracy of recommendations.

[0015] According to the first aspect, or any implementation of the first aspect above, obtaining the image features of the image to be recommended and the text features of the first text includes: obtaining the image to be recommended corresponding to the first content; encoding the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended; encoding the first text to obtain the original text features and the original high-dimensional features of the first text; updating the original image features and the original high-dimensional features of the image to be recommended to obtain the updated image features of the image to be recommended; updating the original text features and the original high-dimensional features of the text to obtain the updated text features of the first text; and determining the updated image features as the image features of the image to be recommended and the updated text features as the text features of the first text.

[0016] In some examples, where the first content includes images and / or videos, the terminal determines the image to be recommended based on the first content input by the user.

[0017] Specifically, when the first content includes video, the terminal can perform frame extraction on the video and use one or more images obtained after the frame extraction as images to be recommended.

[0018] Specifically, when the first content includes images, the terminal can select one or more images from the first content as images to be recommended.

[0019] Optionally, if the first content includes video and / or images, the terminal may also select one or more images from the local gallery as images to be recommended.

[0020] In other examples, where the first content includes text and / or audio, the terminal can select one or more images from its local gallery as recommended images.

[0021] In other examples, the terminal can interact with the server to obtain the image to be recommended corresponding to the first content.

[0022] Image features are feature vectors containing comprehensive information about an image, while high-dimensional image features are feature vectors containing the underlying semantics or logic of the image. Text features are feature vectors containing comprehensive information about the text, while high-dimensional text features are feature vectors containing the underlying semantics or logic of the text.

[0023] In some examples, an image encoding module is used to encode the image to be recommended. The image encoding model outputs the original image features and the original high-dimensional features of the image to be recommended.

[0024] In some examples, a text encoding module is used to encode the first text, which outputs the original text features and the original high-dimensional features of the first text.

[0025] In some examples, the original image features and high-dimensional features of the image to be recommended, as well as the original text features and high-dimensional features of the first text, are saved.

[0026] According to the first aspect, or any implementation of the first aspect above, updating the original image features and the original high-dimensional image features to obtain the updated image features of the image to be recommended includes: in the first update process, obtaining the first image features and the first high-dimensional image features based on the original image features and the original high-dimensional image features; in the i-th update process, obtaining the i-th image features and the i-th high-dimensional image features based on the i-1-th image features and the i-1-th high-dimensional image features obtained after the (i-1)-th update of the original image features and the original high-dimensional image features; 2≤i, i is a positive integer; obtaining the updated image features, the updated image features are the k-th image features obtained after the k-th update, i≤k, k is a positive integer.

[0027] According to the first aspect, or any implementation of the first aspect above, in the first update process, obtaining the first image feature and the first image high-dimensional feature based on the original image feature and the original image high-dimensional feature includes: in the first update process, determining the residual feature of the original image feature, the weight of the original image feature, and the weight of the original image high-dimensional feature; determining the first reference image feature and the first reference image high-dimensional feature based on the original image feature and the weight of the original image feature; determining the first image feature based on the residual feature of the original image feature and the first reference image feature; and determining the first reference image high-dimensional feature as the first image high-dimensional feature.

[0028] According to the first aspect, or any implementation of the first aspect above, updating the original text features and the original high-dimensional text features to obtain the updated text features of the first text includes: in the first update process, obtaining the first text features and the first high-dimensional text features based on the original text features and the original high-dimensional text features; in the i-th update process, obtaining the i-th text features and the i-th high-dimensional text features based on the (i-1)-th text features and the (i-1)-th high-dimensional text features obtained after the (i-1)-th update of the original text features and the original high-dimensional text features; 2≤i, i is a positive integer; determining the updated text features, the updated text features are the k-th text features obtained after the k-th update, i≤k, k is a positive integer.

[0029] According to the first aspect, or any implementation of the first aspect above, in the first update process, obtaining the first text feature and the first high-dimensional feature based on the original text feature and the original high-dimensional feature includes: in the first update process, determining the residual feature of the original text feature, the weight of the original text feature, and the weight of the original high-dimensional feature; determining the first reference text feature and the first reference text high-dimensional feature based on the original text feature and the weight of the original text feature; determining the first text feature based on the residual feature of the original text feature and the first reference text feature; and determining the first reference text high-dimensional feature as the first high-dimensional feature.

[0030] In some examples, a residual feature iterative optimization model is used to update the image features of the image to be recommended and the text features of the first text. The residual feature iterative optimization model can be connected to the image encoding model described above, and placed after the image encoding model. Alternatively, the residual feature iterative optimization model can be connected to the text encoding model described above, and placed after the text encoding model.

[0031] In some examples, image / text features are updated based on residual features and a self-attention mechanism.

[0032] Understandably, residual features represent the parts of a feature that need optimization or adjustment, allowing subsequent models to specifically optimize the shortcomings of the feature, perform more precise feature optimization, and improve feature quality. The self-attention mechanism generates the weight of each feature by calculating the correlation between the feature and the high-dimensional feature. Based on this correlation, it changes the attention given to the feature and the high-dimensional feature, thereby adjusting the weights corresponding to the feature and the high-dimensional feature. Based on these weights, it updates and optimizes the feature and the high-dimensional feature, enhancing attention to relevant features and reducing attention to irrelevant features.

[0033] In this application, residual features and a self-attention mechanism enhance the focus on key features, further optimizing the representation of image features. The updated image features more accurately reflect the key information of the image to be recommended, thereby improving the quality, richness, and accuracy of image features. For high-dimensional image features, the self-attention mechanism enhances attention to relevant features to accurately update image features. Correspondingly, the updated text features more accurately reflect the key information of the first text, thereby improving the quality, richness, and accuracy of text features. For high-dimensional text features, the self-attention mechanism enhances attention to relevant features to accurately optimize text features.

[0034] In some examples, image features or text features are updated iteratively. Iterative updates should be understood as each update being based on the features after the previous update.

[0035] In some examples, the termination condition for updating image or text features includes any of the following: the number of updates meets a preset number, or the updated data meets a preset convergence condition. The preset number of updates can be a pre-set number of iterations. For example, the update ends after the preset number of updates is reached. The preset convergence condition can be determined in advance based on experience or experimentation; for example, the update ends when the difference between the data after the (x-1)th update and the data after the xth update is less than a preset threshold. Here, x is a positive integer. The preset threshold can be pre-set.

[0036] In some examples, the image features of the image to be recommended after the update, as well as the image features of the first text after the update, are saved.

[0037] According to the first aspect, or any implementation of the first aspect above, determining the click-through rate of the first recommended image based on the image features of the first recommended image and the text features of the first text includes: determining the click-through rate features of the first recommended image based on the image features of the first recommended image and the text features of the first text, wherein the click-through rate features include statistical data of user behavior over one or more periods; and determining the click-through rate of the first recommended image based on the image features of the first recommended image, the text features of the first text, and the click-through rate features of the first recommended image, wherein the click-through rate includes the click-through rate over one or more periods.

[0038] In some examples, similarity is calculated between the image features of each image to be recommended and the text features of the first text. The first recommended image is then determined based on the results of multiple similarity calculations. The similarity calculation can be a cosine similarity calculation.

[0039] In some examples, a click-through rate (CTR) feature extraction model is used to determine the CTR features of each first recommended image, and then a CTR prediction model is used to predict the CTR of each first recommended image. The time period can be long-term, medium-term, short-term, or other different periods.

[0040] This application combines image features, text features, and click-through rate (CTR) features to comprehensively analyze the performance of the first recommended image, which helps improve the accuracy of recommendations. Furthermore, by predicting the CTR of the first recommended image, images that users may be interested in can be recommended more accurately, improving user satisfaction. CTR can also assist users in analyzing the posting effectiveness of each first recommended image, thereby optimizing their image selection strategy, increasing CTR, and ultimately improving conversion rates.

[0041] According to the first aspect, or any implementation of the first aspect above, determining the first recommendation result based on the relevance between the first recommended image and the first text and the click-through rate of the first recommended image includes: determining the sorting order of the first recommended images based on the relevance between the first recommended image and the first text and the click-through rate of the first recommended image; displaying the first recommendation result according to the sorting order, wherein the first recommendation result includes the first recommended images arranged in the sorting order.

[0042] In some examples, a ranking model is used to determine the ranking order of the first recommended images.

[0043] In this application, images with high relevance and high click-through rate are ranked first, allowing users to prioritize their selection and making image recommendations more intelligent and adaptive. The recommended images prioritize high relevance and high click-through rate, thus improving the quality and efficiency of image recommendations.

[0044] In some examples, if the user is satisfied with the first recommended result, they can select a cover image from the first recommended result without performing an update operation.

[0045] According to the first aspect, or any of the above implementations of the first aspect, the first recommendation result also includes the click-through rate of the first recommended image.

[0046] According to the first aspect, or any implementation of the first aspect above, the second recommendation result is different from the first recommendation result, specifically: the first recommendation image is the same as the second recommendation image, but the sorting order of the first recommendation image is different from the sorting order of the second recommendation image; or, the first recommendation image is different from the second recommendation image.

[0047] In this application, the terminal can also update image and text features based on user actions, better capturing the user's latest interests and needs to ensure that the recommended results better match the user's current preferences, thereby improving the accuracy and personalization of the recommendations. Timely adjustment of recommendation results based on user updates enhances the user experience and improves the flexibility of image recommendations, optimizing the recommendation strategy. Furthermore, the first recommendation result differs from the second, providing users with more choices and perspectives, increasing the diversity of the recommendation results.

[0048] According to the first aspect, or any implementation of the first aspect above, the image to be recommended includes one or more of the following: user-uploaded images, images extracted from user-uploaded video frames, and images in the local image library.

[0049] According to the first aspect, or any implementation of the first aspect above, when the image to be recommended is an image in the local image library, the image features of the image to be recommended are obtained from the local cache of the terminal. The image features of the image to be recommended are pre-encoded and updated image features.

[0050] In this application, images in the local image library are pre-processed, and the processing results are cached. This can significantly improve the recommendation response speed and reliability, while saving computational resources. Users can obtain recommendation results more quickly, increasing user satisfaction and user experience.

[0051] Secondly, this application provides an image recommendation device, which includes a processor and a memory coupled to the processor. The memory is used to store computer-readable instructions, and when the processor reads the computer-readable instructions from the memory, the image recommendation device causes the image recommendation device to perform the method as described in the first aspect and any embodiment of the first aspect.

[0052] Thirdly, this application provides an electronic device comprising: a processor and a memory coupled to the processor, the memory being used to store computer-readable instructions, wherein when the processor reads the computer-readable instructions from the memory, the electronic device causes the electronic device to perform a method as described in the first aspect and any embodiment of the first aspect.

[0053] Fourthly, this application provides a chip system including at least one processor and at least one interface circuit. The at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor. The at least one processor executes the instructions and performs the method as described in the first aspect and any embodiment of the first aspect.

[0054] Fifthly, this application provides a computer-readable storage medium including a computer program (also referred to as instructions or code) that, when executed on an electronic device, causes the electronic device to perform the method as described in the first aspect and any embodiment of the first aspect.

[0055] In a sixth aspect, this application provides a computer program product comprising: a computer program or instructions that, when executed on a computer, cause the computer to perform a method as described in the first aspect and any of the embodiments described in the first aspect.

[0056] The technical effects corresponding to any implementation method of aspects two through six, as well as any aspect, can be found in the first aspect and the technical effects corresponding to any implementation method of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0057] Figure 1A A schematic diagram illustrating the scene of selecting a cover image for the creation platform provided in this application embodiment;

[0058] Figure 1B This is a schematic diagram of an iterative method provided in an embodiment of this application;

[0059] Figure 1C This is a schematic flowchart of a click-through rate prediction method provided in an embodiment of this application;

[0060] Figure 1D This is a schematic flowchart of an image generation method provided in an embodiment of this application;

[0061] Figure 2 A schematic diagram of the hardware structure of the terminal provided in the embodiments of this application;

[0062] Figure 3 A flowchart illustrating the image recommendation method provided in this application embodiment;

[0063] Figure 4 A schematic diagram of the image recommendation scenario provided in this application embodiment;

[0064] Figure 5 This application provides an example of an image recommendation scenario. Figure 2 ;

[0065] Figure 6 This application provides an example of an image recommendation scenario. Figure 3 ;

[0066] Figure 7 Flowchart of the image recommendation method provided in the embodiments of this application Figure 2 ;

[0067] Figure 8 This is a schematic diagram of the system architecture of the terminal provided in the embodiments of this application;

[0068] Figure 9 A flowchart illustrating the processing of the feature update module provided in this application embodiment;

[0069] Figure 10 Timing diagram 1 for the image recommendation method provided in the embodiments of this application;

[0070] Figure 11 Timing of the image recommendation method provided in the embodiments of this application Figure 2 ;

[0071] Figure 12 Flowchart of the image recommendation method provided in the embodiments of this application Figure 3 ;

[0072] Figure 13 This is a schematic diagram of the structure of the image recommendation device provided in the embodiments of this application;

[0073] Figure 14 This is a schematic diagram of the chip system provided in an embodiment of this application. Detailed Implementation

[0074] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two).

[0075] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0076] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0077] With the rapid development of self-media, creators can publish their own videos, text, images, or documents through content creation platforms. When publishing content on these platforms, it's generally necessary to select an image as the cover image to attract other users to view it.

[0078] In some examples, such as Figure 1A As shown, users can select content to be uploaded, such as videos, images, or documents, on the terminal. The terminal then determines the image corresponding to the content being created.

[0079] For example, if the content being created is a video, the terminal can perform frame-by-frame processing on the video selected by the user. The image corresponding to the content is the image after frame-by-frame processing, and the terminal can display the image after frame-by-frame processing so that the user can select a cover image from it.

[0080] If the content to be created is a combination of text and images, the corresponding image can be one from the text or images. The terminal can display the images from the text or images so that the user can select a cover image. Furthermore, for scenarios where the content to be uploaded is a video or a combination of text and images, the terminal can also respond to user input by redirecting to the local image library, where the user can manually select an image as the cover image for the content to be uploaded.

[0081] If the content being created is a document, the terminal can directly display the local image library, allowing the user to select an image from the terminal's local image library as the cover image.

[0082] The terminal can also receive the title of the content created by the user. Afterwards, the terminal can publish the user-uploaded content, the user-selected cover image, and the content title to the creation platform.

[0083] As can be seen in the example above, users need to manually select a cover image, resulting in low efficiency. Furthermore, the cover image selected by the user may be mismatched or irrelevant to the content title. In related technologies, the terminal cannot guide users to select images with higher potential click-through rates.

[0084] In view of this, embodiments of this application provide an image recommendation method. The terminal can obtain the image features of the image to be recommended and the text features of the content title corresponding to the created content, and iteratively update the image features of the image to be recommended and the text features of the content title. Then, the relevance between the image to be recommended and the content title is calculated based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the content title. Furthermore, the click-through rate of the image to be recommended at different times is predicted based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the content title. Afterwards, the terminal can recommend images related to the content title based on relevance and click-through rate. In the image recommendation method provided by embodiments of this application, the terminal can intelligently recommend images that better meet user expectations, improving the selection efficiency of cover images corresponding to the created content. Moreover, the terminal guides users to select cover images based on relevance and click-through rate, thereby assisting users in selecting images highly relevant to the content title and with potentially high click-through rates as cover images, improving the quality of cover images and the click-through rate of published content, and enhancing the user experience.

[0085] Related technologies provide an iterative scheme, such as Figure 1B As shown, the method obtains image features of the reference image and text features of the prompt text, determines the random noise image as the initial image, and iteratively updates the initial image based on the image features and text features to generate the target image. However, this technique does not update the image features and text features; the image features used in each iteration are the same as the text features, which fails to capture the details of the reference image and prompt text, affecting the image generation effect.

[0086] Related technologies also provide a click-through rate prediction scheme, such as Figure 1CAs shown, the target features corresponding to resources are determined based on the resource interaction record table. Based on the bipartite graph corresponding to the resource interaction records, multimodal representations corresponding to user representation, item representation, and target features are determined. Click-through rate (CTR) sequences are calculated based on these multimodal representations, and recommendation results are generated from the CTR sequences. However, the multimodal markers in this technology are representation vectors resulting from the fusion of image and text features. If the fusion is improper, such as failing to extract effective information, it can lead to the loss of effective information, thus affecting the accuracy of the recommendations.

[0087] Related technologies also provide an image generation method, such as Figure 1D As shown, an image set is formed by selecting images related to the text information from a library. A base image is determined based on this image set (e.g., selecting one image as the base image) and input into the image generation model. The base image is iteratively updated region by region until a complete target image is generated. However, during the iterative training process, updates are performed only based on image information, without considering the text information, leading to a mismatch between the target image and the text information. Furthermore, the updates are performed locally, affecting the overall image quality.

[0088] This application provides an image recommendation method that can be applied to a terminal. A terminal can also be referred to as terminal equipment, electronic equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc.

[0089] For example, the terminal can be a mobile phone, tablet computer, handheld computer, netbook, and various other devices such as personal digital assistant (PDA) and artificial intelligence (AI) devices.

[0090] Figure 2 A schematic diagram of the structure of a terminal provided in an embodiment of this application is shown.

[0091] like Figure 2 As shown, the terminal 200 may include a processor 210, a memory 220, an antenna 1, an antenna 2, a mobile communication module 230, a wireless communication module 240, a display screen 250, etc.

[0092] Processor 210 may include one or more processing units, such as application processors (APs), system-on-chips (SoCs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of terminal 200. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0093] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0094] In some embodiments, processor 210 may include one or more interfaces. These one or more interfaces can be used to connect processor 210 to memory 220, etc.

[0095] In some embodiments of this application, processor 210 can be used to acquire image features of the image to be recommended corresponding to the created content and text features of the content title. Processor 210 is also used to iteratively update the image features of the image to be recommended and the text features of the content title. Processor 210 is also used to calculate the relevance between the image to be recommended and the content title based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the content title. Processor 210 is also used to predict the click-through rate of the image to be recommended at different times based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the content title. Processor 210 is also used to determine the images related to the content title and their sorting order based on the relevance and click-through rate. Processor 210 is also used to trigger display screen 250 to display the images related to the content title based on the images related to the content title and their sorting order. For details, please refer to the following description.

[0096] The memory 220 can be used to store computer executable program code, which includes instructions. The processor 210 executes various functional applications and data processing of the terminal 200 by running the instructions stored in the memory 220. The memory 220 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of the terminal 200 (such as audio data, phonebook, etc.). In addition, the memory 220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0097] The wireless communication function of terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 230, wireless communication module 240, modem processor and baseband processor, etc.

[0098] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0099] The mobile communication module 230 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the terminal 200. The mobile communication module 230 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 230 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0100] In some embodiments, at least some functional modules of the mobile communication module 230 may be disposed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 230 and at least some modules of the processor 210 may be disposed in the same device.

[0101] The wireless communication module 240 can provide solutions for wireless communication applications on the terminal 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 240 can be one or more devices integrating at least one communication processing module. The wireless communication module 240 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 210. The wireless communication module 240 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0102] In some embodiments, antenna 1 of terminal 200 is coupled to mobile communication module 230, and antenna 2 is coupled to wireless communication module 240, enabling terminal 200 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0103] For example, the terminal 200 provided in this embodiment can send messages or instructions to other terminals through antenna 1, antenna 2, mobile communication module 230, wireless communication module 240, etc. Alternatively, the terminal provided in this embodiment can receive information or instructions sent by other terminals through antenna 1, antenna 2, mobile communication module 230, wireless communication module 240, etc.

[0104] Terminal 200 implements display functions through a GPU, display screen 250, and application processor. The GPU is a microprocessor for image processing, connecting the display screen 250 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0105] The display screen 250 is used to display images, videos, etc. The display screen 250 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 200 may include one or N displays 250, where N is a positive integer greater than 1.

[0106] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal 200. In other embodiments of this application, the terminal 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0107] The image recommendation method in this application embodiment can be applied to any application scenario where content is published. The content can include one or more multimedia files such as videos, images, text, and audio. Videos can be user-created videos, edited works, instructional videos, film and television videos, game videos, music videos, short films, etc. Documents can be news articles, strategy guides, lifestyle sharing, technical sharing, daily records, etc. Images can be photographs, paintings, works of art, etc. Audio can be music, lecture audio, radio dramas, audiobooks, etc.

[0108] It is understandable that the application scenarios for publishing creative content can cover different fields, such as education, entertainment, lifestyle, business, technology, art, and news. For example, users can publish teaching videos, textbooks, exercise books, learning summaries, etc. Users can also publish film and television clips, game sharing videos, variety show short videos, music videos, etc. Users can also share life logs, recipes, personal blogs, interior design, etc. Users can also publish product demonstrations, advertisements, promotional posters, etc. Users can also publish product review reports, technical reviews, etc. Users can also publish paintings, photographs, design works, and written content, etc. Users can also publish news reports, feature articles, and current affairs commentaries, etc. This application embodiment does not limit the application scenarios for publishing creative content.

[0109] The image recommendation method of this application will be described in detail below with reference to specific embodiments and accompanying drawings. The embodiments below can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0110] Figure 3 This is a flowchart illustrating an image recommendation method provided in an embodiment of this application. Figure 3 As shown, the method may include:

[0111] S301. The terminal receives the user's input of the created content and the title of the created content.

[0112] The content created can be a multimedia file created by the user. It can also be a multimedia file downloaded by the user from the network or sent to that terminal from another terminal. For example, the content created can include one or more multimedia files such as video, images, text, and audio.

[0113] The content title is the name or description of the content created by the user. It is usually used to reflect the theme and focus of the content, attract the attention of other users, and arouse their curiosity so that they can continue to read or watch the content corresponding to the content title.

[0114] In this embodiment of the application, the user (or creator) can upload creative content through the creation platform and enter the title of the creative content in the creation platform.

[0115] Correspondingly, the terminal receives the content and title of the content entered by the user on the creation platform.

[0116] In some examples, users can input their content and title through a creation platform provided by a creation application (APP) installed on their device. For example, a creation APP could be a video APP, music APP, writing platform APP, news APP, or lifestyle sharing APP.

[0117] In other examples, users can use their terminal's browser to open the authoring platform's website, where they can enter their content and title.

[0118] For example, in response to a user's touch on the video app's icon, the terminal launches the video app and displays its main interface. The user can then click on a creation control (or publishing control, etc.) within the video app, and in response, the terminal displays the video app's creation interface. In this creation interface, the user can select videos and / or images to edit and create videos. The user can also enter information such as a title in the creation interface. Accordingly, the terminal obtains the created content and its title based on the user's actions.

[0119] For example, in response to a user's voice command to create a news article in a news app via a voice assistant, the terminal displays the news app's creation interface. In one implementation, the user can manually input images and text in the creation interface and format them to generate the content. The user can also manually input information such as the content title in the creation interface. Accordingly, the terminal obtains the created content and its title based on the user's actions. In another implementation, the user can manually input images and voice in the creation interface. The terminal receives the user's voice input and converts it into text. Based on the user's voice input, the terminal obtains the text and title of the created content. For example, if the user inputs voice in the creation content module or content title module of the creation interface, the terminal uses its voice-to-text function to convert the user's voice input in different modules into corresponding text, thereby obtaining the text and title of the created content. In yet another implementation, the user can input voice commands in the creation interface, and the terminal obtains the created content and its title in response to the user's voice commands. For example, a voice command could be, "Create a tutorial for making dish A. First, you need ingredient XXX. Second, the cooking process is XXX." The terminal responds to this voice command by converting it into text. The terminal can then retrieve an image of dish A from the converted text and generate the content based on the image and the text "A tutorial for making dish A. First, you need ingredient XXX. Second, the cooking process is XXX." The terminal can also set "A tutorial for making dish A" as the content title.

[0120] It is understood that the terminal can obtain the created content and its title in response to user manual operations or voice commands. This application embodiment does not limit the specific implementation method by which the terminal obtains the created content and its title.

[0121] S302. The terminal obtains the image to be recommended corresponding to the created content.

[0122] The number of images to be recommended can typically be multiple.

[0123] In this embodiment of the application, when the created content includes images and / or videos, the terminal can determine the images to be recommended based on the created content uploaded by the user.

[0124] In some examples, when the user-uploaded content is a video, the terminal can extract multiple frames from the video as images to be recommended. This process of extracting images from a video can be called frame extraction.

[0125] Specifically, the terminal can perform frame extraction using uniform sampling. For example, it can select one frame at regular intervals as the image to be recommended. Alternatively, the terminal can perform frame extraction based on whether the images in the video are keyframes, where keyframes can be determined based on factors such as image clarity or completeness. For example, it can use complete multi-frame images from the video as the image to be recommended. Alternatively, the terminal can perform frame extraction based on specific events, such as motion detection events or scene change events. For example, the terminal can detect motion changes in the video, capture images where the motion changes significantly, and use the captured multi-frame images as the image to be recommended.

[0126] Optionally, when the terminal performs frame extraction on the video, the terminal can also analyze the images and content titles obtained from the frame extraction, and determine the images that match the content titles among the extracted images as the images to be recommended.

[0127] In other examples, when the user-uploaded content is an image, the terminal can select multiple images from the user-uploaded images as images to be recommended.

[0128] Specifically, the terminal can identify all user-uploaded images as potential recommendation images. Alternatively, the terminal can analyze the user-uploaded images and content titles, and identify images that match the content titles as potential recommendation images.

[0129] In other examples, when the user-uploaded content consists of videos and images, the terminal can perform frame extraction on the video and use the extracted images and the user-uploaded images as recommended images, or select multiple images from the extracted images and user-uploaded images as recommended images. For example, the terminal can analyze the extracted images and user-uploaded images against the content title and determine the image that matches the content title as the recommended image.

[0130] Optionally, when the content being created includes images and / or videos, the terminal can also select images from its local image library as images to be recommended.

[0131] In this embodiment of the application, when the created content includes text and / or audio, the terminal can select an image from the local image library as the image to be recommended.

[0132] Specifically, in the above implementation of selecting images from the terminal's local image library as recommended images, the terminal can determine all images in the local image library as recommended images. Alternatively, the terminal can analyze the images and content titles in the local image library and determine the images in the local image library that match the content titles as recommended images.

[0133] Optionally, the terminal can extract keywords from the content title and check whether the image contains visual elements related to those keywords. If so, the image is identified as a candidate for recommendation. For example, if the keyword in the content title includes "hot pot," the terminal checks whether the image contains visual elements related to hot pot, such as the pot itself, ingredients, and broth. Images containing these hot pot-related visual elements are then identified as candidates for recommendation. Alternatively, the terminal can analyze the image content using image recognition software to determine image tags. The image tags and content title are then compared to determine if the image matches the content title. Alternatively, contextual analysis of the image and content title can be performed; if the image content supports or enhances the description of the content title, the image is determined to match the content title. This application does not limit the specific implementation method by which the terminal determines whether the image and content title match.

[0134] Optionally, the terminal can also interact with the server to obtain the image to be recommended corresponding to the created content. That is, the terminal can send the user-uploaded created content (or created content and content title) to the server, which will then obtain the image to be recommended corresponding to the created content. After obtaining the image to be recommended, the server can send it back to the terminal. The specific implementation of the server obtaining the image to be recommended corresponding to the created content can be found in the section on the specific implementation of the terminal obtaining the recommended image corresponding to the created content, and will not be elaborated upon here.

[0135] It is understood that the embodiments of this application do not limit the specific implementation method of obtaining the image to be recommended.

[0136] S303. The terminal obtains the image features of the image to be recommended, as well as the text features of the content title.

[0137] In this embodiment, the terminal encodes the image to be recommended to obtain its image features. Additionally, the terminal can also obtain the high-dimensional features of the image to be recommended by encoding it. The image features and high-dimensional features of the image to be recommended can also be described as a pair of image features.

[0138] In this embodiment, the terminal encodes the content title to obtain its text features. Additionally, the terminal can also obtain the high-dimensional text features of the content title by encoding it. The text features and high-dimensional text features of the content title can also be described as a text feature pair.

[0139] Image features are feature vectors containing comprehensive information about an image, while high-dimensional image features are feature vectors containing the underlying semantics or logic of the image. Text features are feature vectors containing comprehensive information about the text, while high-dimensional text features are feature vectors containing the underlying semantics or logic of the text.

[0140] Specifically, the terminal can use an image encoding model to encode the image to be recommended. For example, the image to be recommended is input into the image encoding model, which processes the image and outputs the image features and high-dimensional features of the image. The terminal can use a text encoding model to encode the content title. For example, the content title is input into a text encoding model, which processes the content title and outputs the text features and high-dimensional features of the text.

[0141] Among them, the image encoding model and the text encoding model can be deep learning models, such as the Transformer model.

[0142] The image encoding model and text encoding model mentioned above can be obtained through training and inference with a large model.

[0143] For example, the image coding model training process includes the following steps: The server collects a large number of labeled images as training data, and selects a Transformer model as the training model. The training data is input into the training model for training. The training process typically includes forward propagation, loss calculation, and backpropagation to optimize the weights and structure of the training model. After training is complete, the image coding model is obtained.

[0144] It is understandable that the training process for text encoding models is the same as that for image encoding models, so it will not be repeated here.

[0145] The terminal can obtain the image encoding model from the server. Based on the image encoding model, the terminal can encode the image, infer and extract the image features and the high-dimensional image features of the image to be recommended.

[0146] In some embodiments, the terminal may pre-encode images from a local image library to obtain image features and high-dimensional features of the images in the local image library. The terminal may also store the image features and high-dimensional features of the images in the local image library. For example, when the terminal system's computing resources are idle, the terminal encodes images from a local image library, obtains the encoded features (such as image features and high-dimensional features of the images), and stores this information in a local cache.

[0147] Understandably, if the image to be recommended obtained in S302 includes images from the local image library, and the image to be recommended is an encoded image from the local image library, the terminal can directly obtain the image features and high-dimensional features of the image to be recommended from the local cache. If the image to be recommended is an unencoded image from the local image library, the terminal can encode the image to be recommended using an image encoding model to obtain the image features and high-dimensional features of the image to be recommended.

[0148] For example, if the user-uploaded content is a video, the terminal performs frame extraction on the video, and then extracts the resulting image 1 and... Figure 2 The image is identified as the image to be recommended; taking the title of the content received by the terminal as text 1 as an example. Image 1 is inferred using an image coding model to obtain image features f10 and high-dimensional image features h10. Image 2 is inferred using an image coding model to obtain image features f20 and high-dimensional image features h20. Text 1 is inferred using a text coding model to obtain text features t0 and high-dimensional text features g0.

[0149] The example above illustrates this by using a video image obtained through frame extraction. In other examples, the recommended image may also include an image from the terminal's local image library, such as image 3. In this case, if image 3 is an encoded image from the local image library, the image features and high-dimensional features of image 3 can be directly retrieved from the local cache.

[0150] For example, taking image coding models as an example, the image coding model is a Transformer model, which includes an attention layer, a normalization layer, and a linear layer. The attention layer is responsible for capturing the relationships and contextual information between different regions in the image. Through a self-attention mechanism, it focuses on important regions of the image and ignores unimportant regions. The normalization layer is used to integrate the output of the attention layer, typically mapping the output to a higher-dimensional space. The normalization layer can integrate and enhance features, such as determining the structure and content of the image. The linear layer is used to perform a linear transformation on the features processed by the attention and normalization layers, typically mapping the features to the final output space and transforming the feature representation.

[0151] It's understandable that the linear layer outputs image features, encompassing the overall features of the image. The attention layer and the normalization layer output high-dimensional image features, containing features from the lower to higher levels of the image. Image features and high-dimensional image features are determined in the same inference process. High-dimensional image features are those processed and output during inference, containing underlying logic, and are not the last layer in the image coding model. Image features are those output by the last layer (i.e., the normalization layer), representing the final inference output. Image features and high-dimensional image features are corresponding.

[0152] The structure of the text encoding model is the same as that of the image encoding model, so it will not be described again here.

[0153] Understandably, in this step, the terminal encodes the image to be recommended to obtain its image features and high-dimensional image features. The terminal also encodes the content title to obtain its text features and high-dimensional text features. Based on these acquired features, the terminal can subsequently analyze the features and calculate relevance. Furthermore, image features contain comprehensive information about the image, and text features contain comprehensive information about the text, which is beneficial for subsequent iterative updates. High-dimensional image features can introduce more dimensions to represent the image, and high-dimensional text features can introduce more dimensions to represent the text, improving the expressive power and richness of the features. This facilitates accurate capture of relevance between features during subsequent iterative updates, thereby improving the accuracy of recommendations.

[0154] In this embodiment, after obtaining the image features and high-dimensional features of the image to be recommended, as well as the text features and high-dimensional features of the content title, the terminal can save the image features and high-dimensional features of the image to be recommended, and the text features and high-dimensional features of the content title. For example, the terminal stores the above features in the form of feature pairs in its local cache; that is, the terminal can store the feature pairs of the image to be recommended and the feature pairs of the content title in its local cache.

[0155] Understandably, caching feature pairs of the images to be recommended and feature pairs of the content titles on the terminal allows for the preservation of comprehensive and underlying information about both, ensuring information integrity and preventing loss. Furthermore, in the event of an error, rapid recovery can be achieved based on the saved feature pairs.

[0156] S304. The terminal updates the image features of the image to be recommended and the text features of the content title.

[0157] In this embodiment, the terminal can use a residual feature iterative optimization model to update the image features of the image to be recommended and the text features of the content title.

[0158] In some examples, the terminal can be pre-configured with a feature for intelligent iterative features, which, when enabled, allows the terminal to automatically update the image features of the image to be recommended and the text features of the content title.

[0159] For example, a terminal can input the image features and high-dimensional features of the image to be recommended into a residual feature iterative optimization model. The residual feature iterative optimization model then infers and outputs updated image features and high-dimensional features of the image to be recommended. Similarly, a terminal can input the text features and high-dimensional features of the content title into a residual feature iterative optimization model. The residual feature iterative optimization model then infers and outputs updated text features and high-dimensional features of the content title.

[0160] Specifically, taking updating the image features of an image to be recommended as an example, the terminal can input the image features and the high-dimensional features of the image to be recommended into the residual feature iterative optimization model. The residual feature iterative optimization model infers residual features based on the image features. The residual feature iterative optimization model processes the image features and high-dimensional features of the image to be recommended using a self-attention mechanism to obtain the weights of the image features and the high-dimensional features. The image features are multiplied by their weights to obtain reference image features, and the high-dimensional features are multiplied by their weights to obtain reference high-dimensional features. The reference image features and the residual features are added together to obtain the updated image features of the image to be recommended. The reference high-dimensional features are then determined as the updated high-dimensional features of the image to be recommended.

[0161] Understandably, residual features represent the parts of a feature that need optimization or adjustment, allowing subsequent models to specifically optimize the shortcomings of the feature, perform more precise feature optimization, and improve feature quality. The self-attention mechanism generates the weight of each feature by calculating the correlation between the feature and the high-dimensional feature. Based on this correlation, it changes the attention given to the feature and the high-dimensional feature, thereby adjusting the weights corresponding to the feature and the high-dimensional feature. Based on these weights, it updates and optimizes the feature and the high-dimensional feature, enhancing attention to relevant features and reducing attention to irrelevant features.

[0162] Understandably, for image features, the focus on key features can be enhanced using residual features and self-attention mechanisms, further optimizing the representation of image features. Optimized image features will more accurately reflect the key information of the image to be recommended, thereby improving the quality, richness, and accuracy of image features. For high-dimensional image features, attention to relevant features can be enhanced using self-attention mechanisms to accurately optimize image features.

[0163] Understandably, the text features and high-dimensional text features of the content title, when input into the residual feature iterative optimization model, undergo a similar processing procedure. Specifically, the residual feature iterative optimization model infers residual features based on text features. The model then processes the text features and high-dimensional text features of the content title using a self-attention mechanism, obtaining the weights of the text features and the high-dimensional text features. The text features are multiplied by their weights to obtain reference text features, and the high-dimensional text features are multiplied by their weights to obtain reference high-dimensional text features. The reference text features are then added to the residual features to obtain the updated text features of the content title. Finally, the reference high-dimensional text features are determined as the updated high-dimensional text features of the content title.

[0164] Understandably, for text features, the focus on key features can be enhanced using residual features and self-attention mechanisms to further optimize the text feature representation. The optimized text features will more accurately reflect the key information of the content title, thereby improving the quality, richness, and accuracy of the text features. For high-dimensional text features, the attention to related features can be enhanced using self-attention mechanisms to accurately optimize the text features.

[0165] Among them, the residual feature iterative optimization model can be obtained through training and inference of a large model.

[0166] For example, the training process of a residual feature iterative optimization model includes the following steps: The server collects a large amount of labeled images and text as training data to ensure the model can learn effective features and optimization strategies. A deep learning network containing a residual feature optimization module and a self-attention mechanism is constructed. The training data is input into the deep learning network for training. During forward propagation, the residual feature optimization module is used to infer residual features, weights are generated according to the self-attention mechanism, and the features of the training data are updated based on the residual features and weights. The forward propagation process is then executed again based on the updated features of the training data to obtain new, further updated features. This update step is repeated until the predetermined number of iterations is reached, at which point the backpropagation process begins. During backpropagation, the corresponding loss function is calculated based on the features of the updated training data from each step of the forward propagation process. All loss functions from the forward propagation process are integrated to obtain a comprehensive loss function, and the network parameters are updated based on this comprehensive loss function. After updating the network parameters, the model repeats the forward and backpropagation processes until convergence, resulting in the residual feature iterative optimization model.

[0167] The terminal can obtain the residual feature iterative optimization model from the server, and based on the residual feature iterative optimization model, the terminal can update the image features of the image to be recommended and the text features of the content title.

[0168] In some implementations, the residual feature iterative optimization model can be connected to the aforementioned image coding model, following it. Alternatively, it can be connected to the aforementioned text coding model, following it. The structure of the residual feature iterative optimization model is smaller than that of the image coding or text coding models. Computationally, it is more efficient than the image coding or text coding models, requiring less computation per iteration, thus reducing the computational burden during inference and accelerating model inference. Furthermore, the residual feature iterative optimization model focuses on optimizing the features extracted by the image coding or text coding models, eliminating the need for complex feature extraction and enabling more efficient feature adjustment and optimization. Moreover, based on residual features, the residual feature iterative optimization model can capture subtle feature differences. Iteratively optimizing features based on these differences helps improve model accuracy and obtain more precise features for accurate subsequent correlation calculations.

[0169] In some examples, the terminal inputs the image features of the image to be recommended and the high-dimensional features of the image to be recommended, output by the image encoding model, into the residual feature iterative optimization model, and inputs the text features of the content title and the high-dimensional features of the content title, output by the text encoding model, into the residual feature iterative optimization model.

[0170] Understandably, in this example, the data input to the residual feature iterative optimization model is generated in real time by an image encoding model or a text encoding model, allowing for immediate optimization.

[0171] In other examples, the terminal inputs the image features of the image to be recommended and the high-dimensional features of the image to be recommended into the residual feature iterative optimization model; the terminal inputs the text features of the content title and the high-dimensional text features of the content title into the residual feature iterative optimization model.

[0172] Understandably, in this example, the data input to the residual feature iterative optimization model is statically stored on the terminal and may not be generated in real time. This reduces the need for real-time computation and is suitable for scenarios with limited computing resources.

[0173] It is understandable that iterative updates should be interpreted as each update being based on the features from the previous update. That is, the image features and text features in S303 are the features before the iterative updates. Image features can be understood as the original image features of the image to be recommended, and text features can be understood as the original text features of the content title. Correspondingly, the high-dimensional image features obtained after encoding the image to be recommended can be understood as the original high-dimensional image features, and the text features obtained after encoding the content title can be understood as the original high-dimensional text features.

[0174] Subsequently, the terminal performs a first iterative update based on the original image features and high-dimensional features of the image to be recommended, obtaining the first update result (e.g., image feature 1 and high-dimensional feature 1 of the image to be recommended after the first update). That is, the terminal updates the original image features to image feature 1 and the original high-dimensional features to high-dimensional feature 1. After obtaining the first update result, the terminal can also perform a second iterative update based on the first update result, obtaining the second update result (e.g., image feature 2 and high-dimensional feature 2 of the image to be recommended after the second update). That is, the terminal updates image feature 1 to image feature 2 and high-dimensional feature 1 to high-dimensional feature 2. After obtaining the second update result, the terminal can also perform one or more iterative updates based on the second update result until the update ends, obtaining the update result corresponding to the last update. Similarly, the terminal can also perform one or more iterative updates based on the original text features and high-dimensional features of the content title until the update ends, obtaining the update result corresponding to the last update. The specific implementation method for updating text features is the same as the method for updating image features described above, and will not be repeated here.

[0175] In this application embodiment, the termination condition for updating the image features of the image to be recommended and the text features of the content title includes any one of the following: the number of updates meets a preset number, and the updated data meets a preset convergence condition.

[0176] The preset number of iterations can be a pre-defined number of iterations. For example, the preset number of iterations could be m, where m is a positive integer.

[0177] The preset convergence condition can be determined in advance based on experience or experiments. For example, the update ends when the difference between the data after the (x-1)th update and the data after the xth update is less than a preset threshold value. Here, x is a positive integer. The preset threshold value can be set in advance.

[0178] For example, continuing with the example in S303, based on the above examples of image 1, image 2, and text 1, and taking the above termination condition as the number of updates satisfying a preset number, where the preset number is m, after the terminal obtains the image features f10 and high-dimensional features h10 of image 1, the image features f20 and high-dimensional features h20 of image 2, and the text features t0 and high-dimensional features g0 of text 1, the terminal sends the feature pairs f10 and h10, f20 and h20, and t0 and g0 to the residual feature iterative optimization model respectively. Taking the feature pair f10 and h10 as an example for one update, the residual feature iterative optimization model infers and predicts the residual feature delt_t1 based on f10. F10 and h10 are processed according to the self-attention mechanism, and the weight of f10 is determined to be w10, and the weight of h10 is determined to be l10. The product of f10 and w10 is calculated as f10*w10 = f10' (i.e., obtaining the reference image features of image 1 at the time of the first update). The product of h10 and l10 is calculated as h10*l10 = h10' (i.e., obtaining the high-dimensional features of the reference image of image 1 at the time of the first update). The addition of f10' and delt_t1 is calculated as f10' + delt_t1 = f11 (i.e., obtaining the image features of image 1 after the first update). h10' is determined as h11 (i.e., obtaining the high-dimensional features of image 1 after the first update). If m is greater than 1, the terminal needs to update the feature pair (i.e., f11 and h11) after the first update. The updated f11 and h11 are sent to the residual feature iterative optimization model for the second update to obtain f12 and h12. ... This process continues until the number of iterations meets the preset number m, obtaining the updated f1m and h1m after the mth update.

[0179] Accordingly, the terminal can perform m iterations of the residual feature iterative optimization model to update f20 and h20, obtaining f2m and h2m. The terminal can also perform m iterations of the residual feature iterative optimization model to update t0 and g0, obtaining tm and gm.

[0180] In this embodiment, the terminal can save the updated image features and high-dimensional features of the image to be recommended, as well as the updated text features and high-dimensional features of the content title. For example, the terminal stores the updated features in a local cache.

[0181] In some examples, the terminal can save the image features of the image to be recommended before it is updated (i.e., the original image features mentioned above) and the high-dimensional features of the image (i.e., the original high-dimensional features of the image mentioned above), as well as the image features and high-dimensional features of the image to be recommended after each iteration update. The terminal can also save the text features of the content title before it is updated (i.e., the original text features mentioned above) and the high-dimensional features of the text (i.e., the original high-dimensional features of the text mentioned above), as well as the text features and high-dimensional features of the content title after each iteration update. For example, the updated data can be saved to a local cache.

[0182] For example, based on the example in Image 1 above, the terminal locally caches the data updated in each iteration before and after the update, f10, h10, f11, h11, f12, h12...f1m, h1m.

[0183] In other examples, during the iteration process, the terminal can save the image features of the image to be recommended before it is updated (i.e., the original image features mentioned above) and the high-dimensional features of the image (i.e., the original high-dimensional features of the image mentioned above), as well as the latest image features and high-dimensional features of the image to be recommended after it is updated. The terminal can also save the text features of the content title before it is updated (i.e., the original text features mentioned above) and the high-dimensional features of the text (i.e., the original high-dimensional features of the text mentioned above), as well as the latest text features and high-dimensional features of the text title after it is updated.

[0184] For example, based on the example in Figure 1 above, when the terminal does not perform iterative updates, it locally caches f10 and h10. After the terminal performs the first update, it saves f11 and h11. After the terminal performs the second update, it clears f11 and h11 and saves f12 and h12. ... After the terminal performs the m-th update, the data saved in the cache this time includes f10, h10, f1m, and h1m.

[0185] It is understood that in this embodiment of the application, the terminal continuously saves the original text features and original image features so that when update data is lost, iterative updates can be made based on the original data.

[0186] In some embodiments, the terminal may also pre-update the image features of images in the local image library. The terminal obtains the image features of encoded images in the local image library and the corresponding high-dimensional image features. The image features and the corresponding high-dimensional image features are updated using a residual feature iterative optimization model. The terminal records the updated image features and the updated high-dimensional image features for each iteration.

[0187] It is understandable that when the terminal updates image features and text features, the update frequency should be kept the same or synchronized. Therefore, it is necessary to record the data of each update of the images in the local image library. When the content creation includes images and / or videos, and the images to be recommended are determined based on the content creation and the local image library, the update frequency of the image features of the images to be detected based on the local image library can be consistent with the update frequency of the image features of the images to be detected based on the images and / or videos, and the update frequency of the text features of the content titles. If the update frequency of the image features of the images to be detected based on the images and / or videos, and the update frequency of the text features of the content titles, do not exceed the update frequency of the image features of the images to be detected based on the local image library, only the image features of the images to be detected based on the images and / or videos, and the text features of the content titles, should be updated. If the update frequency of the image features of the images to be detected based on the images and / or videos, and / or the update frequency of the text features of the content titles, are greater than the update frequency of the image features of the images to be detected based on the local image library, the image features of all images to be detected and the text features of the content titles should be updated.

[0188] Understandably, in this step, by iteratively updating image features and text features, the updated image features more accurately represent the key information of the image to be recommended, thus improving the relevance between image features and the image; the updated text features more accurately represent the key information of the text, thus improving the relevance between text features and the content title; this is beneficial for accurately calculating the relevance between the image to be recommended and the content title in the future, thereby improving the quality of image recommendation.

[0189] It is understood that the embodiments of this application do not limit the specific implementation of the terminal updating the image features of the image to be recommended and the text features of the content title.

[0190] S305. The terminal calculates the relevance between the image to be recommended and the content title based on the updated image features and the updated text features, and determines the recommended image based on the relevance.

[0191] The updated image features and updated text features are the output data of the residual feature iterative optimization model after the update is completed. For example, if the update ends with a preset number of iterations m as the termination condition, the updated image features can be f1m and f2m, and the updated text features can be tm. Alternatively, if the update ends with a preset convergence condition as the termination condition, the updated image features and updated text features can be the output data after the x-th iteration.

[0192] The relevance between the image to be recommended and the content title can be represented by similarity. The number of recommended images cannot exceed the number of images to be recommended.

[0193] In this embodiment of the application, as described in the foregoing embodiments, there are generally multiple images to be recommended. For each image to be recommended, the terminal can perform a similarity calculation based on the updated image features of the image to be recommended and the updated text features of the content title, and determine the recommended image based on the multiple similarity calculation results. The similarity calculation can be a cosine similarity calculation.

[0194] For example, the terminal sorts the images to be recommended in descending order based on multiple similarity calculation results, and selects the top n images as recommended images. Here, n is a positive integer. Alternatively, it selects the images whose similarity calculation results are greater than a preset threshold as recommended images.

[0195] For example, based on the example in S304 above, the terminal can calculate the similarity between f1m and tm (e.g., similarity 1) and the similarity between f2m and tm (e.g., similarity 2) based on cosine similarity. The terminal sorts the images to be recommended according to the similarity results. If similarity 2 is greater than similarity 1, it means that the similarity between image 2 and text 1 is higher than the similarity between image 1 and text 1. The sorting order of images related to text 1 from high to low should be image 2, image 1. If n is 2, then the two most relevant images to text 1 should be image 2, image 1.

[0196] It is understandable that the above example uses two images, but in practical applications, the images to be recommended can include many more images.

[0197] Understandably, by calculating the similarity between the updated image features and the updated text features, and then filtering the images to be processed, it can be ensured that the subsequently recommended cover images and content titles are relevant and highly relevant, which is beneficial to improving the quality of subsequent recommendations. Moreover, processing only recommended images can improve the system's processing speed while reducing the computational burden and resource consumption.

[0198] S306. The terminal determines the click-through rate of the recommended image based on the updated image features corresponding to the recommended image and the updated text features of the content title.

[0199] Click-through rate (CTR) includes CTR over one or more periods. A period can be understood as a time span or time interval, that is, the time span after the content is created and published. Different periods correspond to different time spans. For example, taking CTR encompassing multiple periods as an example, the periods can be long-term, medium-term, short-term, etc. Short-term can be a relatively short time span after the recommended image and content title are published, such as a few hours to a few days. Medium-term can be a moderate time span after the recommended image and content title are published, such as a few days to a few weeks. Long-term can be a relatively long time span after the recommended image and content title are published, such as a few weeks to a few months, or even a year or longer.

[0200] In this embodiment, the terminal determines the click-through rate (CTR) feature of the recommended image based on the updated image features corresponding to the recommended image and the updated text features of the content title. The CTR feature includes CTR features for one or more periods. The terminal determines the CTR of the recommended image based on the updated image features corresponding to the recommended image, the updated text features of the content title, and the CTR feature of the recommended image.

[0201] Click-through rate (CTR) features can be predicted as the frequency or probability of users clicking on the recommended image and content title. CTR features for different time periods (long-term, medium-term, short-term) can reflect statistical data on user click behavior on the recommended image and content title over different time spans. For example, long-term CTR features can reflect the long-term appeal and lasting influence of the recommended image and content title. Medium-term CTR features can reflect the appeal and trend of the recommended image and content title over a medium time span. Short-term CTR features can reflect the click situation of the recommended image and content title in the initial stage of publication.

[0202] Specifically, the terminal can use a click-through rate (CTR) feature extraction model to determine CTR features. For example, the terminal can input the updated image features corresponding to the recommended image and the updated text features of the content title into the CTR feature extraction model. The CTR feature extraction model then infers and outputs the CTR features of the recommended image for multiple periods, including long-term, medium-term, and short-term periods.

[0203] In this embodiment of the application, the terminal can save the click-through rate characteristics of each recommended image.

[0204] Among them, the click-through rate feature extraction model can be obtained through training and inference with a large model.

[0205] For example, the training process for a click-through rate (CTR) feature extraction model includes the following steps: The server collects image features and corresponding text features containing CTR data from different time periods (long, medium, and short) as training data to ensure the model can learn effective features and optimization strategies. A Transformer model is selected as the training model. The training data is input into the training model for training. The training process typically includes forward propagation, loss calculation, and backpropagation to optimize the model's weights and structure. After training, the CTR feature extraction model is obtained.

[0206] The terminal can obtain the click-through rate feature extraction model from the server, and then extract the click-through rate features of the recommended images based on this model.

[0207] For example, continuing with the example of S303 above, the terminal inputs the updated image features f2m of image 2 and the updated text features tm of text 1 into the click-through rate (CTR) feature extraction model. The CTR feature extraction model then infers the long-term CTR feature flong2, the medium-term CTR feature fmid2, and the short-term CTR feature fshort2. The terminal also inputs the updated image features f1m of image 1 and the updated text features tm of text 1 into the CTR feature extraction model. The CTR feature extraction model then infers the long-term CTR feature flong1, the medium-term CTR feature fmid1, and the short-term CTR feature fshort1.

[0208] It is understandable that predicting the click-through rate characteristics of recommended images at different times is helpful in understanding users' behavioral patterns toward the recommended images at different times, so as to optimize the recommendation strategy in the future.

[0209] Specifically, the terminal can use a click-through rate (CTR) prediction model to predict the CTR features of recommended images. For example, the terminal can input the updated image features corresponding to the recommended image, the updated text features of the content title, and the CTR features of the recommended image into the CTR prediction model. The CTR prediction model then infers and outputs the CTR of the recommended image for multiple periods, including long-term, medium-term, and short-term periods.

[0210] The click-through rate (CTR) prediction model can be trained and inferred using a large model. The training process for the CTR prediction model is similar to that of the CTR feature extraction model described above. The difference lies in the fact that the training data for the CTR prediction model consists of a large amount of image features and their corresponding text features containing CTR data from different periods.

[0211] For example, continuing with the above example, the terminal inputs the updated image features f2m of image 2, the updated text features tm of text 1, the long-term click-through rate (CTR) feature flong2, the medium-term CTR feature fmid2, and the short-term CTR feature fshort2 into the CTR prediction model. Based on the above information, the CTR prediction model infers the long-term CTR clong2, medium-term CTR cmid2, and short-term CTR cshort2 corresponding to image 2. Correspondingly, the terminal inputs the updated image features f1m of image 1, the updated text features tm of text 1, the long-term CTR feature flong1, the medium-term CTR feature fmid1, and the short-term CTR feature fshort1 into the CTR prediction model. Based on the above information, the CTR prediction model infers the long-term CTR clong1, medium-term CTR cmid1, and short-term CTR cshort1 corresponding to image 1.

[0212] It is understandable that the aforementioned period can be set by the user or pre-configured by the terminal.

[0213] Understandably, by predicting the click-through rate (CTR) of recommended images, the terminal can more accurately recommend images that users might be interested in, thereby improving user satisfaction. Furthermore, CTR also helps users analyze the posting performance of each recommended image, allowing them to optimize their image selection strategy, increase CTR, and ultimately improve conversion rates.

[0214] It is understandable that in S305-S306 above, the terminal first determines the recommended images, and then determines the click-through rate (CTR) of each recommended image. In practical applications, the terminal can also determine the CTR of each image to be recommended, and then combine the relevance of each image to be recommended with the content title and the CTR of each image to be recommended to determine the recommended image. This application does not limit the specific implementation method of determining the recommended image.

[0215] S307. The terminal determines the sorting order of recommended images based on the relevance between the recommended images and the content titles and the click-through rate of the recommended images.

[0216] In this embodiment, the terminal performs a weighted calculation based on the relevance of the recommended image to the content title and the click-through rate of the recommended image to determine the sorting order of the recommended images.

[0217] For example, the terminal predefines weights for click-through rates (CTRs) at different times, such as wlong, wmid, and wshort. The terminal calculates the overall CTR for each recommended image by combining the CTRs of each time period with the corresponding weights. A comprehensive score is then calculated for each recommended image based on its overall CTR and its similarity to the content title. The ranking order of the recommended images is determined based on their comprehensive scores. This ranking order can be from highest to lowest, prioritizing the display of recommended images with higher comprehensive scores later.

[0218] For example, based on the examples of Image 1, Image 2, and Text 1 above, the overall click-through rate for Image 1 is clong1*wlong + cmid1*wmid + cshort1*wshort. The overall score for Image 1 is clong1*wlong + cmid1*wmid + cshort1*wshort + similarity 1. The overall click-through rate for Image 2 is clong2*wlong + cmid2*wmid + cshort2*wshort. The overall score for Image 2 is clong2*wlong + cmid2*wmid + cshort2*wshort + similarity 2. If the overall score for Image 2 is greater than the overall score for Image 1, then the image ranking order is Image 2, then Image 1.

[0219] Optionally, the terminal can also input the relevance of the recommended image to the content title and the click-through rate of the recommended image into the ranking model, and the ranking model will process and output the ranking order of the recommended images.

[0220] It is understandable that the ranking model can be a trained network model. The training process of the ranking model is similar to that of the models mentioned above, and will not be repeated here.

[0221] Optionally, the terminal can also determine the sorting order of recommended images based on a comprehensive consideration of factors such as the relevance of the recommended images to the content titles, the click-through rate of the recommended images, the quality of the recommended images (e.g., the clarity, resolution, and composition of the recommended images), and user preferences (e.g., determining user preferences or users based on historical data). This application does not limit the specific implementation method for determining the sorting order of recommended images.

[0222] It is understandable that the above example uses multiple recommended images as an illustration. If there is only one recommended image, there is no need to determine the sorting order; the recommended image and its click-through rate can be displayed directly.

[0223] S308. The terminal displays the recommendation results, which include recommended images and the click-through rate of the recommended images. The recommended images are displayed in sorted order.

[0224] For example, such as Figure 4 The diagram illustrates a scenario where a terminal recommends images based on user-uploaded content. When a user selects a cover image, a recommendation result window 401 is displayed on the terminal interface. This window includes image 2 and image 1, sorted in the order determined in step S307. The recommendation result window 401 also displays the click-through rate for each image over multiple periods. The window may also include one or more functional controls, such as a cancel control, an update control, and a confirm control. The cancel control cancels the recommendation function and closes the recommendation result window 401. The update control triggers the terminal to update the information in the recommendation result window 401. The confirm control confirms the user-selected cover image. For example, if a user selects image 2 as the cover image based on the information displayed in the recommendation result window 401, the user can first click on image 2 and then click the confirm control. The terminal responds to the user's action by replacing the cover image with image 2.

[0225] For example, such as Figure 5 The diagram illustrates a scenario where a terminal recommends images based on its local image library. After the user enters a title, a recommendation results window 501 is displayed on the terminal interface. This window shows the recommendation results determined based on images in the local image library, such as local image 1 and local image 2, as well as the click-through rate information for each image at different times.

[0226] For example, such as Figure 6 The diagram illustrates a scenario where a terminal recommends images based on user-uploaded content and a local image library. When a user selects a cover image, a recommendation result window 601 is displayed on the terminal interface. This window displays recommendation results determined based on the user-uploaded content and images from the local image library. The recommendation results also include click-through rate information for each image at different times.

[0227] The source of the images included in the terminal's recommendation results (such as images from user-uploaded content or images from the local image library) is determined by different scenarios and / or user operations. For details, please refer to the relevant description in S301.

[0228] Understandably, the terminal determines the ranking of recommended images based on relevance and click-through rate, which can optimize the ranking strategy. Users can determine the relevance of each recommended image to the content title and the potential click-through rate of the recommended image based on the ranking, which can help users select images with higher potential click-through rates as cover images, thus enhancing the accuracy and quality of cover image recommendations.

[0229] It is understood that the above example illustrates the case where the image to be recommended includes multiple images. If the image to be recommended includes only one image, the terminal can identify that image as the recommended image, obtain the image features of the recommended image and the text features of the content title, and update the aforementioned features. The click-through rate of the recommended image is determined based on the updated image features and updated text features. The terminal displays the recommended image and its click-through rate. This application does not limit the specific implementation method of displaying the recommended image.

[0230] Understandably, the above example illustrates the processing flow of the terminal's intelligent iteration function. In this example, the terminal automatically iterates and updates image features and text features to accurately determine the relevance between the detected image and the content title, thereby improving the image quality recommended by the terminal.

[0231] Optionally, in some other examples, the intelligent iteration feature is not enabled on the terminal. For example... Figure 7 The diagram shown is a flowchart of another image recommendation method provided in this application embodiment, including the following steps:

[0232] S701, The terminal receives the user's input of the created content and the title of the created content.

[0233] S702: The terminal obtains the image to be recommended corresponding to the created content.

[0234] S703: The terminal obtains the image features of the image to be recommended, as well as the text features of the content title.

[0235] S704. The terminal calculates the relevance between the image to be recommended and the content title based on image features and text features, and determines the recommended image based on the relevance.

[0236] S705. The terminal determines the click-through rate of the recommended image based on the image features corresponding to the recommended image and the text features of the content title.

[0237] S706. The terminal determines the sorting order of recommended images based on the relevance between the recommended images and the content titles, and the click-through rate of the recommended images.

[0238] S707. The terminal displays the recommendation results, which include recommended images and the click-through rate of the recommended images. The recommended images are displayed in sorted order.

[0239] Understandably, in this example, the terminal determines the recommended image based on the image features of the encoded image to be recommended (i.e., the original image features) and the text features of the content title (i.e., the original text features). The relevance, click-through rate, and ranking order of the recommended image are calculated based on the image features of the recommended image and the text features of the content title. This can accelerate the processing efficiency of image recommendation.

[0240] It is understandable that the specific implementation methods of S701-S707 mentioned above are described above and will not be repeated here.

[0241] It is understood that in this application, the terminal can perform image recommendation based on various models, with feature extraction, feature iteration, and click-through rate prediction completed on the terminal side, thereby improving the efficiency of image recommendation and achieving intelligent recommendation. The terminal can continuously update and iterate image / text features to obtain more accurate features. The terminal can also respond to user operations to update recommendation results to meet different user needs, improve the quality of cover images, optimize cover image selection strategies, and enhance terminal intelligence.

[0242] In other examples, the terminal can also update the recommendation results in response to the user's update action.

[0243] The update operation can be pre-set and can trigger the terminal to update the recommendation results.

[0244] For example, in response to a user's update operation, the terminal updates image features and text features to generate new recommendation results. The specific implementation of the terminal updating recommendation results is described in sections S304-S308 above, and will not be repeated here.

[0245] For example, based on the above Figure 4For example, if a user touches the update control, the terminal responds to the user's update operation by obtaining f1m and h1m corresponding to image 1, f2m and h2m corresponding to image 2, and tm and gm corresponding to text 1. The terminal iteratively updates based on the above data to generate new recommendation results.

[0246] For example, in response to a user's update operation, the terminal updates the image to be recommended to generate new recommendation results. The specific implementation of the terminal updating the recommendation results is described in S302-S308 above, and will not be repeated here.

[0247] like Figure 8 The diagram shown is a system architecture diagram of the terminal in this embodiment of the application. The terminal includes a preprocessing module, an encoding module, a feature update module, a first processing module, a click-through rate processing module, a second processing module, and a display module. The modules in the terminal are interconnected.

[0248] Figure 8 The terminal shown is capable of performing the above. Figure 3 or Figure 7 The terminal in the image recommendation method described above can execute the steps of the image recommendation method in the above embodiments through its respective modules. Therefore, the foregoing implementation methods are also applicable to the terminal provided in this embodiment, and the functions of each module in the terminal will not be described in detail in this embodiment.

[0249] The foregoing embodiments and their beneficial effects are also applicable to this embodiment; therefore, the same parts will not be repeated.

[0250] like Figure 9 The diagram shows the processing flowchart of the feature update module. The image features and high-dimensional features of the image to be recommended, as well as the text features and high-dimensional features of the content title, are input into the feature update module. The feature update module can process the image features using residual features and a self-attention mechanism to obtain the updated image features of the image to be recommended. Similarly, the feature update module can process the text features using residual features and a self-attention mechanism to obtain the updated text features of the content title. The feature update module can also process the high-dimensional image features using a self-attention mechanism to obtain the updated high-dimensional image features of the image to be recommended. Finally, the feature update module can process the high-dimensional text features using a self-attention mechanism to obtain the updated high-dimensional text features of the content title.

[0251] Understandable Figure 8 The system architecture diagram shown is only an example. In practical applications, the terminal may include more or fewer modules, and the embodiments of this application do not limit the division of modules in the terminal.

[0252] It is understandable that the above Figure 8The encoding module, feature update module, first processing module, click-through rate processing module, and second processing module in the terminal shown can also be located on the server.

[0253] Specifically, this application can also provide an image recommendation system, which includes a terminal and a server. The terminal includes a preprocessing module and a display module. The server includes an encoding module, a feature update module, a first processing module, a click-through rate processing module, and a second processing module. The terminal obtains the content title and the image to be processed through the preprocessing module and sends the content title and the image to be processed to the server. The server's encoding module receives the content title and the image to be processed uploaded by the terminal and processes them through the various modules in the server to obtain the recommendation result. The server sends the recommendation result to the terminal's display module, and the display module displays the recommendation result.

[0254] It is understandable that the above Figure 8 The terminal shown can also be understood as an image recommendation system. The embodiments of this application do not limit the specific implementation of the image recommendation system.

[0255] It is understood that the system architecture and application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute the only limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0256] like Figure 10 The diagram shows the timeline of image recommendations performed by various modules in the terminal based on user-uploaded images / videos when the user activates the intelligent iterative feature function. (Above) Figure 3 The image recommendation method shown is also applicable to the terminal provided in this embodiment, and will not be described in detail in this embodiment.

[0257] The preprocessing module receives the user-inputted content and its title. It also retrieves the corresponding image to be recommended. Finally, it sends the image and title to the encoding module.

[0258] The encoding module is used to obtain the image features of the image to be recommended and the text features of the content title. The encoding module is also used to send the image features and text features to the feature update module. The encoding module is further used to send the high-dimensional image features of the image to be recommended and the high-dimensional text features of the content title to the feature update module. Figure 10 (Not shown in the image).

[0259] The feature update module is used to update the image features of the image to be recommended and the text features of the content title. Specifically, the feature update module updates the image features of the image to be recommended and the text features of the content title based on the image features, high-dimensional image features, text features, and high-dimensional text features. Figure 10 (Not shown in the image). The feature update module is also used to send the updated image features and updated text features to the first processing module.

[0260] The first processing module calculates the relevance between the image to be recommended and the content title based on the updated image features and updated text features, and determines the recommended image based on the relevance. The first processing module also sends the updated image features corresponding to the recommended image and the updated text features of the content title to the click-through rate processing module.

[0261] The click-through rate (CTR) processing module determines the CTR of the recommended image based on the updated image features corresponding to the recommended image and the updated text features of the content title. This CTR processing module then sends the CTR of the recommended image to the second processing module. The first processing module sends the relevance between the recommended image and the content title to the second processing module.

[0262] The second processing module determines the sorting order of recommended images based on the relevance of the recommended images to the content titles and the click-through rate of the recommended images. The second processing module then sends the sorting order to the display module.

[0263] The display module is used to display the recommendation results, which include recommended images and their click-through rates. The recommended images are displayed in sorted order.

[0264] like Figure 11 The diagram shows the timing of image recommendations made by each module in the terminal based on the local image library. The example uses all images from the local image library as the images to be processed.

[0265] The preprocessing module is used to acquire images from the local image library and send the images to the encoding module.

[0266] The encoding module is used to acquire the image features and the high-dimensional features of the image. The encoding module is also used to send the image features to the feature update module. Furthermore, the encoding module is used to send the high-dimensional features of the image to the feature update module.

[0267] The feature update module is used to update the image features and high-dimensional features of the image, and save the updated image features and high-dimensional features in the local cache.

[0268] The preprocessing module also receives user-inputted content and its title. It determines that the image to be recommended corresponds to the content and is from the local image library. The preprocessing module also sends the image information to the local cache. The local cache sends the updated image features of the image to be recommended to the first processing module. Finally, the preprocessing module sends the content title to the encoding module.

[0269] The encoding module is also used to obtain the text features of the content title and the corresponding high-dimensional text features, and send them to the feature update module.

[0270] The feature update module updates the text features and high-dimensional text features, and sends the updated text features to the first processing module. The processing flow from the first processing module to the display module is shown in the figure above, and will not be repeated here.

[0271] It is understood that the above example illustrates the technical solution of this application by using a user publishing content on a creation platform. The technical solution of this application can also be applied to other scenarios. For example, when organizing an image collection, a suitable cover can be selected based on the collection name. Similarly, when organizing materials, conferences, or online courses, a cover can be selected based on the theme.

[0272] For example, a terminal, such as a gallery application, can automatically or in response to user actions analyze images in its gallery to generate video clips or collections using a "Highlights" feature. The terminal can recommend cover images for video clips or collections using the image recommendation method provided in this application. For instance, images of categories such as time, animals, food, architecture, plants, games, and geographic information can be extracted from the gallery to form a "Highlights" collection. Alternatively, users can customize collection titles, and the terminal, in response to user actions, creates the collection based on the user-created title and recommends a collection cover image.

[0273] For example, a terminal, such as an educational application, responds to user actions by compiling learning courses or materials in a particular subject into a collection or publication. Alternatively, a terminal, such as a conferencing application, responds to user actions by compiling meeting materials or minutes based on meeting topics and materials, and generates a cover image for the meeting materials or minutes.

[0274] like Figure 12 As shown, another image recommendation method provided in this application embodiment includes the following steps:

[0275] S1201, Receive the first content input by the user and the first text of the first content.

[0276] The first content can be a user-created multimedia file. The first content can include one or more multimedia files such as video, images, text, and audio.

[0277] For example, the first content can be the creative content described above, or it can be learning materials, meeting materials, etc. The first content can be generated by a user based on multimedia files downloaded from the network, or it can be generated by a user based on multimedia files obtained from other terminals, or it can be generated by a user based on locally cached multimedia files. This application does not limit the specific implementation method of the first content or the method of creating the first content.

[0278] The first text of the first content can be text information describing the first content. For example, the title of the first content, a brief description of the first content, and tags for the first content. For instance, if the first content is a tutorial for making dish A, the title of the first content could be: "Dish A Tutorial." The brief description of the first content could be: "Lazy Version of Dish A Tutorial, Come and Try It!" The tags for the first content could be: Dish A, Dish Tutorial, Home-Style Dishes, etc.

[0279] Optionally, if the first content includes text, that text can be used as the first text of the first content.

[0280] Optionally, the first text of the first content can also be keyword information of the first content. For example, keywords can be extracted from the content title, the description, or the tags of the first content to obtain the first text of the first content. Alternatively, if the first content includes text, keywords can be extracted from that text to obtain the first text of the first content.

[0281] For example, the terminal receives first content input by the user and text information describing the first content, and determines the text information as the first text of the first content.

[0282] For another example, the terminal receives first content input by the user and text information describing the first content, extracts keyword information from the text information, and identifies the keyword information as the first text.

[0283] It is understandable that the specific implementation of receiving the first content input by the user and the first text of the first content can be found in S301 above.

[0284] S1202. Display the first recommendation result based on the image features of the image to be recommended corresponding to the first content and the text features of the first text; the first recommendation result includes the first recommended image.

[0285] In this embodiment of the application, the number of images to be recommended is usually multiple. However, the number of images to be recommended can also be one.

[0286] In some examples, the images to be recommended include one or more of the following: user-uploaded images, images extracted from user-uploaded video frames, and images from the local gallery.

[0287] In some examples, displaying a first recommendation result based on the image features of the image to be recommended corresponding to the first content and the text features of the first text includes: obtaining the image features of the image to be recommended and the text features of the first text; determining the relevance between the image to be recommended and the first text based on the image features of the image to be recommended and the text features of the first text, and determining the first recommended image based on the relevance; determining the click-through rate of the first recommended image based on the image features of the first recommended image and the text features of the first text; and determining the first recommendation result based on the relevance between the first recommended image and the first text and the click-through rate of the first recommended image.

[0288] In some examples, obtaining the image features of the image to be recommended and the text features of the first text includes: obtaining the image to be recommended corresponding to the first content; encoding the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended; encoding the first text to obtain the original text features and the original high-dimensional features of the first text; and determining the original image features as the image features of the image to be recommended and the original text features as the text features of the first text.

[0289] In other examples, obtaining the image features of the image to be recommended and the text features of the first text includes: obtaining the image to be recommended corresponding to the first content; encoding the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended; encoding the first text to obtain the original text features and the original high-dimensional features of the first text; updating the original image features and the original high-dimensional features of the image to be recommended to obtain the updated image features of the image to be recommended; updating the original text features and the original high-dimensional features of the text to obtain the updated text features of the first text; and determining the updated image features as the image features of the image to be recommended and the updated text features as the text features of the first text.

[0290] In this embodiment of the application, when the first content includes images and / or videos, the terminal determines the image to be recommended based on the first content input by the user.

[0291] In one possible implementation, if the first content includes video, the terminal can perform frame extraction on the video and use one or more images obtained after the frame extraction as images to be recommended.

[0292] In one possible implementation, if the first content includes images, the terminal can select one or more images from the first content as images to be recommended.

[0293] Optionally, if the first content includes video and / or images, the terminal may also select one or more images from the local gallery as images to be recommended.

[0294] In other embodiments of this application, when the first content includes text and / or audio, the terminal may select one or more images from a local gallery as images to be recommended.

[0295] In some other embodiments of this application, the terminal can interact with the server to obtain the image to be recommended corresponding to the first content.

[0296] It is understandable that the specific implementation of obtaining the image to be recommended can be found in S302 above.

[0297] Image features are feature vectors containing comprehensive information about an image, while high-dimensional image features are feature vectors containing the underlying semantics or logic of the image. Text features are feature vectors containing comprehensive information about the text, while high-dimensional text features are feature vectors containing the underlying semantics or logic of the text.

[0298] In one possible implementation, encoding the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended includes: encoding the image to be recommended using an image encoding module, and the image encoding model outputting the original image features and the original high-dimensional features of the image to be recommended.

[0299] In one possible implementation, encoding the first text to obtain the original text features and the original high-dimensional features of the first text includes: encoding the first text using a text encoding module, which outputs the original text features and the original high-dimensional features of the first text.

[0300] Among them, the image encoding model and the text encoding model can be deep learning models, which are obtained through training and inference of large models.

[0301] In some embodiments of this application, the terminal can obtain trained image encoding model and text encoding model from the server, and the terminal processes the image to be recommended based on the image encoding model and processes the first text based on the text encoding model.

[0302] Optionally, the terminal can also save the original image features and high-dimensional features of the image to be recommended, as well as the original text features and high-dimensional features of the first text.

[0303] In some other embodiments of this application, the terminal may send the image to be recommended and the first text to the server, and the server may process the image to be recommended according to the image encoding model and process the first text according to the text encoding model.

[0304] For example, the terminal determines the image to be recommended corresponding to the first content, and the first text. The terminal sends the image to be recommended and the first text to the server. The server can encode the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended. The server can also encode the first text to obtain the original text features and the original high-dimensional features of the first text.

[0305] For another example, the terminal receives first content and first text of the first content input by the user, and sends the first content and first text to the server. The server determines the image to be recommended based on the first content and / or the first text. The server can encode the image to be recommended to obtain the original image features and the original high-dimensional features of the image to be recommended. The server can also encode the first text to obtain the original text features and the original high-dimensional features of the first text.

[0306] Optionally, the server can also save the original image features and high-dimensional features of the image to be recommended, as well as the original text features and high-dimensional features of the first text.

[0307] It is understandable that the specific implementation methods for encoding the image to be recommended to obtain the original image features and encoding the first text to obtain the original text features are described in S303 above.

[0308] In some examples, after determining the original image features of the image to be recommended and the original text features of the first text, the original image features can be determined as the image features of the image to be recommended, and the original text features can be determined as the text features of the first text.

[0309] In other examples, after determining the original image features of the image to be recommended and the original text features of the first text, the original image features and the original text features can be updated, and the updated image features are determined as the image features of the image to be recommended, and the updated text features are determined as the text features of the first text.

[0310] In some embodiments of this application, the terminal uses a residual feature iterative optimization model to update the image features of the image to be recommended and the text features of the first text.

[0311] In other embodiments of this application, the server uses a residual feature iterative optimization model to update the image features of the image to be recommended and the text features of the first text.

[0312] Among them, the residual feature iterative optimization model can be a deep learning model, which is obtained through training and inference of a large model.

[0313] In some implementations, the residual feature iterative optimization model can be connected to the image coding model described above, following the image coding model. Similarly, the residual feature iterative optimization model can be connected to the text coding model described above, following the text coding model.

[0314] In some examples, the original image features and the original high-dimensional image features are updated to obtain the updated image features of the image to be recommended. This includes: in the first update process, obtaining the first image features and the first high-dimensional image features based on the original image features and the original high-dimensional image features; in the i-th update process, obtaining the i-th image features and the i-th high-dimensional image features based on the (i-1)-th image features and the (i-1)-th high-dimensional image features obtained after the (i-1)-th update of the original image features and the original high-dimensional image features; 2≤i, where i is a positive integer; obtaining the updated image features, which are the k-th image features obtained after the k-th update, where i≤k, where k is a positive integer.

[0315] In some examples, during the first update process, the first image features and the first high-dimensional features of the original image are obtained based on the original image features and the original image high-dimensional features. This includes: during the first update process, determining the residual features of the original image features, the weights of the original image features, and the weights of the original image high-dimensional features; determining the first reference image features and the first reference image high-dimensional features based on the original image features and the weights of the original image features; determining the first image features based on the residual features of the original image features and the first reference image features; and determining the first reference image high-dimensional features as the first image high-dimensional features.

[0316] Understandably, the above example illustrates the specific implementation method for updating image features. The following describes the specific implementation method for updating text features.

[0317] In some examples, updating the original text features and the original high-dimensional text features to obtain the updated text features of the first text includes: in the first update process, obtaining the first text features and the first high-dimensional text features based on the original text features and the original high-dimensional text features; in the i-th update process, obtaining the i-th text features and the i-th high-dimensional text features based on the (i-1)-th text features and the (i-1)-th high-dimensional text features obtained after the (i-1)-th update of the original text features and the original high-dimensional text features; 2≤i, where i is a positive integer; determining the updated text features, where the updated text features are the k-th text features obtained after the k-th update, where i≤k, where k is a positive integer.

[0318] In some examples, during the first update process, the first text feature and the first high-dimensional feature are obtained based on the original text features and the original high-dimensional features, including: during the first update process, determining the residual features of the original text features, the weights of the original text features, and the weights of the original high-dimensional features; determining the first reference text feature and the first reference high-dimensional feature based on the original text features and the weights of the original text features; determining the first text feature based on the residual features of the original text features and the first reference text feature; and determining the first reference high-dimensional feature as the first high-dimensional feature.

[0319] It is understood that in the embodiments of this application, the image features or text features are iteratively updated. Iterative updates should be understood as each update being based on the features after the previous update.

[0320] In the embodiments of this application, the termination condition for updating image features or text features includes any one of the following: the number of updates meets a preset number of times, and the updated data meets a preset convergence condition.

[0321] The preset number of updates can be a pre-defined number of iterations. For example, the update ends after the preset number of updates is reached. If the preset number of updates is m, then m updates are performed. After the m-th update, the m-th image feature or the m-th text feature is obtained. The m-th image feature is then determined as the updated image feature of the image to be recommended, or the m-th text feature is determined as the updated image feature of the first text. m is a positive integer. In this example, k = m.

[0322] The preset convergence condition can be determined in advance based on experience or experiments. For example, the update ends when the difference between the data after the (x-1)th update and the data after the xth update is less than a preset threshold value. Here, x is a positive integer. The preset threshold value can be set in advance. In this example, m = x.

[0323] Optionally, the terminal or server may save the updated image features of the image to be recommended, as well as the updated image features of the first text.

[0324] It is understandable that the specific implementation of updating image features / text features can be found in S303 above.

[0325] After obtaining the updated image features and the updated text features, the updated image features can be determined as the image features of the image to be recommended, and the updated text features can be determined as the text features of the first text.

[0326] After determining the image features of the image to be recommended and the text features of the first text, a similarity calculation is performed on the image features of the image to be recommended and the text features of the first text for each image to be recommended. The first recommended image is determined based on the results of multiple similarity calculations.

[0327] If there are multiple images to be recommended, the first recommended image can be one or more of them. If there is only one image to be recommended, the first recommended image is that image.

[0328] It is understandable that the specific implementation method for determining the first recommended image is described in S305 above.

[0329] In some examples, determining the click-through rate (CTR) of a first recommended image based on image features of a first recommended image and text features of a first text includes: determining CTR features of a first recommended image based on image features of a first recommended image and text features of a first text, wherein the CTR features include statistical data on user behavior over one or more periods; and determining the CTR of a first recommended image based on image features of a first recommended image, text features of a first text, and CTR features of a first recommended image, wherein the CTR includes CTR over one or more periods.

[0330] In some embodiments of this application, the terminal uses a click-through rate feature extraction model to determine the click-through rate features of each first recommended image, and then uses a click-through rate prediction model to predict the click-through rate of each first recommended image.

[0331] In some other embodiments of this application, the server uses a click-through rate feature extraction model to determine the click-through rate features of each first recommended image, and then uses a click-through rate prediction model to predict the click-through rate of each first recommended image.

[0332] It is understandable that the specific implementation method for determining the click-through rate of the first recommended image is described in S306 above.

[0333] In some examples, determining the first recommendation result based on the relevance of the first recommended image and the first text, as well as the click-through rate of the first recommended image, includes: determining the sorting order of the first recommended images based on the relevance of the first recommended image and the first text, as well as the click-through rate of the first recommended image; and displaying the first recommendation result according to the sorting order, wherein the first recommendation result includes the first recommended images arranged in the sorting order.

[0334] In some examples, the first recommendation result also includes the click-through rate of the first recommended image.

[0335] In some embodiments of this application, the terminal uses a ranking model to determine the ranking order of the first recommended images. The terminal generates a first recommendation result based on this ranking order and the first recommended images.

[0336] In other embodiments of this application, the server uses a ranking model to determine the ranking order of the first recommended images. The server sends the ranking order, the first recommended images, and the click-through rate of the first recommended images to the terminal, so that the terminal can generate a first recommendation result based on the above information.

[0337] It is understandable that the specific implementation method for determining the first recommendation result is described in section S307 above.

[0338] In some examples, when the image to be recommended is an image in the local image library, the image features of the image to be recommended are retrieved from the local cache of the terminal. The image features of the image to be recommended are pre-encoded and updated image features.

[0339] Understandably, if the user is satisfied with the first recommended result, then S1203-S1204 need not be executed, and the user can select a cover image from the first recommended result.

[0340] S1203. In response to the user's update operation on the first recommendation result, update the image features and text features.

[0341] It is understandable that the specific implementation of S1203 is described in S304 above.

[0342] S1204. Display the second recommendation result based on the updated image features and the updated text features; the second recommendation result includes the second recommendation image, and the second recommendation result is different from the first recommendation result.

[0343] In some examples, the second recommendation result differs from the first recommendation result, specifically: the first recommended image is the same as the second recommended image, but the sorting order of the first recommended image is different from the sorting order of the second recommended image; or, the first recommended image is different from the second recommended image.

[0344] In some examples, the click-through rate of the second recommended image is also included in the second recommendation results.

[0345] It is understandable that the specific implementation of S1203 can be found in S305-S307 above.

[0346] In this application, the terminal, based on the first content and the first text of the first content, obtains the image features of the image to be recommended and the text features of the first text corresponding to the first content, and iteratively updates the image features of the image to be recommended and the text features of the first text. Then, the relevance between the image to be recommended and the first text is calculated based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the first text. Furthermore, the click-through rate of the image to be recommended at different times is predicted based on the iteratively updated image features of the image to be recommended and the iteratively updated text features of the first text. Afterwards, the terminal can recommend images related to the first text based on relevance and click-through rate. In the image recommendation method provided in this application embodiment, the terminal can intelligently recommend images that better meet user expectations, improving the selection efficiency of the cover image corresponding to the first content. Moreover, the terminal guides the user to select a cover image based on relevance and click-through rate, thereby assisting the user in selecting images highly relevant to the first text and with potentially high click-through rates as cover images, improving the quality of the cover image and the click-through rate of the published content, and enhancing the user experience.

[0347] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0348] This application embodiment can divide the above-described electronic device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0349] Based on the same inventive concept, embodiments of this application provide an image recommendation device. The image recommendation device provided in embodiments of this application is applied to… Figure 2 Terminal 200 is shown. (As shown) Figure 13The diagram shown is a structural schematic of an image recommendation device provided in an embodiment of this application. The image recommendation device can be used to implement the methods described in the above method embodiments. For example, the image recommendation device may specifically include: a processing module 1301, a transceiver module 1302, and a display module 1303.

[0350] The processing module 1301 is used to support the execution of the image recommendation device. Figures 3-12 The processing module 1301 is configured to receive first content and first text of the first content input by the user. The processing module 1301 is further configured to display a first recommendation result based on the image features of the image to be recommended corresponding to the first content and the text features of the first text; the first recommendation result includes a first recommended image. The processing module 1301 is further configured to update the image features and text features in response to the user's update operation on the first recommendation result. The processing module 1301 is further configured to display a second recommendation result based on the updated image features and updated text features; the second recommendation result includes a second recommended image, and the second recommendation result is different from the first recommendation result.

[0351] The transceiver module 1302 is used to support the execution of the image recommendation device. Figures 3-12 Send and receive functions for any item in the list.

[0352] The display module 1303 is used to support the execution of the image recommendation device. Figures 3-12 The display module 1303 is used to display any of the following: [List of recommendations]. In this embodiment, the display module 1303 is used to display the first recommendation result. The display module 1303 is also used to display the second recommendation result.

[0353] Figure 13 The technical effects of the image recommendation device shown can be referred to the technical effects of the method described in the above-described image recommendation method embodiments, and will not be repeated here. Figure 13 The processing module 1301 involved in the image recommendation device shown can be implemented by a processor or processor-related circuit components, and can be a processor or processing module. The display module 1303 can be implemented by display screen-related components.

[0354] This application also provides another image recommendation device, which includes a processor and a memory, the memory being coupled to the processor and used to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the image recommendation device executes the methods described in the above-described image recommendation method embodiments.

[0355] This application also provides a chip system, such as... Figure 14As shown, the chip system 1400 includes at least one processor 1401 and at least one interface circuit 1402. As an example, when the chip system 1400 includes a processor and an interface circuit, the processor can be... Figure 14 The processor 1401 shown in the solid box (or the processor 1401 shown in the dashed box) may have an interface circuit that can be... Figure 14 The interface circuit 1402 is shown in the solid box (or the dashed box). When the chip system 1400 includes two processors and two interface circuits, the two processors include... Figure 14 The processor 1401 shown in the solid box and the processor 1401 shown in the dashed box, the two interface circuits include Figure 14 Interface circuit 1402 is shown in both solid and dashed boxes. No limitations are imposed on this.

[0356] Processor 1401 and interface circuit 1402 can be interconnected via lines. For example, interface circuit 1402 can be used to receive signals. As another example, interface circuit 1402 can be used to send signals to other devices (e.g., processor 1401). Exemplarily, interface circuit 1402 can read instructions stored in memory and send the instructions to processor 1401. When the instructions are executed by processor 1401, the steps in the above embodiments can be performed. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0357] Optionally, there can be one or more processors in the chip system. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory.

[0358] Optionally, the chip system may also include a memory ( Figure 14 (Not shown in the image), the memory may be one or more, and the memory may be integrated with the processor or set separately from the processor. This application does not limit the scope of the memory.

[0359] For example, the memory can be a non-transient processor, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not specifically limit the type of memory or the way the memory and processor are set.

[0360] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0361] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0362] This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the method described in the above-described method embodiments.

[0363] Computer-readable storage media include, but are not limited to, any of the following: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media capable of storing program code.

[0364] This application provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0365] In addition, this application also provides an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the apparatus to perform the methods in the above-described method embodiments.

[0366] In addition, this application also provides a system, which may specifically be a chip, component or module. The system may include a connected processor and a memory. The memory is used to store computer execution instructions. When the system is running, the processor can execute the computer execution instructions stored in the memory to enable the system to perform the methods in the above-described method embodiments.

[0367] In this embodiment, the apparatus, electronic device, computer storage medium, computer program product, or chip are all used to execute the corresponding methods described above. Therefore, the beneficial effects achieved can be referred to the beneficial effects of the corresponding methods described above, and will not be repeated here.

[0368] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, compact disc read-only memory (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an application-specific integrated circuit (ASIC).

[0369] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the division of the above functional modules is only used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0370] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The embodiments can be combined with or referenced to each other without conflict. The apparatus embodiments described above are merely illustrative; for example, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0371] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0372] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0373] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0374] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image recommendation method characterized by, The method comprises: receiving a first content input by a user and a first text of the first content; displaying a first recommendation result according to image features of a to-be-recommended image corresponding to the first content and text features of the first text; the first recommendation result comprises a first recommended image; in response to an update operation of the first recommendation result by the user, updating the image features and the text features; displaying a second recommendation result according to the updated image features and the updated text features; the second recommendation result comprises a second recommended image, and the second recommendation result is different from the first recommendation result.

2. The method of claim 1, wherein, The method comprises: obtaining image features of the to-be-recommended image and text features of the first text; determining the relevance of the to-be-recommended image and the first text according to the image features of the to-be-recommended image and the text features of the first text, and determining the first recommended image according to the relevance; determining the click rate of the first recommended image according to the image features of the first recommended image and the text features of the first text; determining the first recommendation result according to the relevance of the first recommended image and the first text and the click rate of the first recommended image.

3. The method of claim 2, wherein, The method comprises: obtaining the to-be-recommended image corresponding to the first content; encoding the to-be-recommended image to obtain original image features and original image high-dimensional features of the to-be-recommended image; encoding the first text to obtain original text features and original text high-dimensional features of the first text; determining the original image features as the image features of the to-be-recommended image, and determining the original text features as the text features of the first text.

4. The method of claim 2, wherein, The method comprises: obtaining the to-be-recommended image corresponding to the first content; encoding the to-be-recommended image to obtain original image features and original image high-dimensional features of the to-be-recommended image; encoding the first text to obtain original text features and original text high-dimensional features of the first text; updating the original image features and the original image high-dimensional features to obtain updated image features of the to-be-recommended image; updating the original text features and the original text high-dimensional features to obtain updated text features of the first text; determining the updated image features as the image features of the to-be-recommended image, and determining the updated text features as the text features of the first text.

5. The method of claim 4, wherein, The method comprises: in a first update process, obtaining first image features and first image high-dimensional features based on the original image features and the original image high-dimensional features; In the i th updating process, the i th image feature and the i th high-dimensional image feature are obtained based on the i-1 th image feature and the i-1 th high-dimensional image feature obtained after the original image feature and the original high-dimensional image feature are updated for the i-1 th time; 2≤i, the i is a positive integer; An updated image feature is obtained, the updated image feature being the k th image feature obtained after the k th updating, i≤k, the k being a positive integer.

6. The method of claim 5, wherein, In the first updating process, the first image feature and the first high-dimensional image feature are obtained based on the original image feature and the original high-dimensional image feature, comprising: In the first updating process, the residual feature of the original image feature, the weight of the original image feature, and the weight of the original high-dimensional image feature are determined; The first reference image feature and the first reference high-dimensional image feature are determined according to the original image feature and the weight of the original image feature; The first image feature is determined according to the residual feature of the original image feature and the first reference image feature; The first reference high-dimensional image feature is determined as the first high-dimensional image feature.

7. The method according to any one of claims 4-6, characterized in that, The original text feature and the original high-dimensional text feature are updated to obtain the first text updated text feature, comprising: In the first updating process, the first text feature and the first high-dimensional text feature are obtained based on the original text feature and the original high-dimensional text feature; In the i th updating process, the i th text feature and the i th high-dimensional text feature are obtained based on the i-1 th text feature and the i-1 th high-dimensional text feature obtained after the original text feature and the original high-dimensional text feature are updated for the i-1 th time; 2≤i, the i is a positive integer; An updated text feature is determined, the updated text feature being the k th text feature obtained after the k th updating, i≤k, the k being a positive integer.

8. The method of claim 7, wherein, In the first updating process, the first text feature and the first high-dimensional text feature are obtained based on the original text feature and the original high-dimensional text feature, comprising: In the first updating process, the residual feature of the original text feature, the weight of the original text feature, and the weight of the original high-dimensional text feature are determined; The first reference text feature and the first reference high-dimensional text feature are determined according to the original text feature and the weight of the original text feature; The first text feature is determined according to the residual feature of the original text feature and the first reference text feature; The first reference high-dimensional text feature is determined as the first high-dimensional text feature.

9. The method according to any one of claims 2-8, characterized in that, The click rate of the first recommended image is determined according to the image feature of the first recommended image and the text feature of the first text, comprising: The click rate feature of the first recommended image is determined according to the image feature of the first recommended image and the text feature of the first text, the click rate feature comprising statistical data of user behavior in one or more periods; The click rate of the first recommended image is determined according to the image feature of the first recommended image, the text feature of the first text, and the click rate feature of the first recommended image, the click rate comprising the click rate in one or more periods.

10. The method according to any one of claims 2-9, characterized in that, The first recommendation result is determined according to the relevance between the first recommended image and the first text and the click rate of the first recommended image. The first recommendation result is determined according to the relevance between the first recommended image and the first text and the click rate of the first recommended image. The first recommendation result is displayed according to the sorting order, and the first recommended image is arranged according to the sorting order.

11. The method according to any one of claims 1-10, characterized in that, The first recommendation result further includes the click rate of the first recommended image.

12. The method of any one of claims 1-11, wherein, The second recommendation result is different from the first recommendation result, and specifically: The first recommended image is the same as the second recommended image, and the sorting order of the first recommended image is different from the sorting order of the second recommended image. Or, The first recommended image is different from the second recommended image.

13. The method according to any one of claims 1-12, characterized in that, The image to be recommended includes one or more of the following: an image uploaded by a user, an image obtained by extracting frames from a video uploaded by a user, and an image in a local image library.

14. The method of any one of claims 1-13, wherein, In the case where the image to be recommended is an image in the local image library, the image feature of the image to be recommended is obtained from the local cache of the terminal, and the image feature of the image to be recommended is an image feature pre-encoded and updated.

15. An electronic device, comprising: The electronic device includes a processor and a memory, the memory is coupled to the processor, and the memory is used to store computer readable instructions, when the processor reads the computer readable instructions from the memory, the electronic device executes the method of any one of claims 1-14.

16. A chip system, characterized by The electronic device includes at least one processor and at least one interface circuit, the at least one interface circuit is used to perform transceiving functions and send instructions to the at least one processor, the at least one processor executes the instructions, and the at least one processor executes the method of any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a computer program, when the computer program runs on the electronic device, the electronic device executes the method of any one of claims 1-14.

18. A computer program product, characterised in that, The computer program product includes a computer program or instructions, when the computer program or instructions run on the computer, the computer executes the method of any one of claims 1-14.

Images