Image collection recommendation method and electronic equipment

Abstract

The embodiment of the invention provides an image set recommendation method and electronic equipment. The method comprises the steps that at least one image set is acquired; if n first sets exist in at least one image set, respectively acquiring the type of each first set; n is an integer greater than 1, and the first set refers to a set of which the priority type is a first priority; the similar recommended times corresponding to a second set are determined, the second set is any one of the n first sets, and the similar recommended times corresponding to the second set refer to the recommended times of an image set with the same type as the second set; according to the type of each first set and the corresponding similar recommended times, sorting the n first sets to obtain a first sorting result; determining a to-be-recommended image set according to the first sorting result; and recommending the to-be-recommended image set. The method can be recommended to the user after screening and gathering, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an image collection recommendation method and electronic device. Background Art

[0002] With the gradual improvement of the camera performance of electronic devices such as mobile phones and tablets, taking photos through electronic devices has become one of the important ways for people to record their lives. The images taken by users are often used to record the beautiful moments in life or as important memories of users. Therefore, if some images can be selected from the images taken by users to generate an image collection and the image collection is recommended to users, it will increase the emotional resonance of users and bring them a good experience.

[0003] So, how to recommend more attractive image collections to users and provide them with a better experience is a problem that needs to be solved. Summary of the invention

[0004] The present application provides an image collection recommendation method, device, electronic device, chip, computer-readable storage medium and computer program product, which can filter the collection and recommend it to the user, thereby improving the user experience.

[0005] In a first aspect, the present application provides an image collection recommendation method, the method comprising: obtaining at least one image collection; if there are n first collections in at least one image collection, obtaining the type of each first collection respectively; n is an integer greater than 1, and the first collection refers to a collection whose priority type is a first priority; determining the number of times the same type of recommendation corresponding to a second collection has been made, the second collection is any one of the n first collections, and the number of times the same type of recommendation corresponding to the second collection refers to the number of times an image collection of the same type as the second collection has been recommended; sorting the n first collections according to the type of each first collection and the corresponding number of times the same type of recommendation has been made to obtain a first sorting result; determining an image collection to be recommended according to the first sorting result; and recommending the image collection to be recommended.

[0006] Optionally, the level of priority can be determined according to the user's preference for the image collection, and the higher the user's preference, the higher the priority. In this embodiment, the first priority can be the highest priority, and the user has the highest preference for the image collection of the first priority. The image collection of the first priority can be an image collection of a preset type, such as an image collection generated on holidays, etc.

[0007] The collection of images to be recommended is also called a collection to be recommended.

[0008] The collection recommendation method provided in the first aspect of the present application filters collections from image collections and recommends them to users, rather than directly recommending collections to users. This can prevent excessive collection recommendations from disturbing users and improve user experience. When determining the image collection to be recommended, priority is given to filtering from the first-priority collection in at least one image collection. The first-priority collection is more likely to increase user interest and thus improve user experience. In addition, when filtering the image collection to be recommended from the first-priority collection (i.e., the first collection), the n first collections are sorted according to the type of each first collection and the corresponding number of similar recommendations to obtain a first sorting result; the image collection to be recommended is determined according to the first sorting result. In this way, the types of each collection and the number of similar recommendations are fully compared, aiming to filter and recommend collections that are more in line with user expectations and have a higher degree of user preference to users, and the number of similar recommendations is taken into account when recommending, thereby improving user experience.

[0009] In one possible implementation, n first collections are sorted according to the type of each first collection and the corresponding number of similar recommendations to obtain a first sorting result, including: determining the content score of each first collection according to the type of each first collection; determining the frequency suppression score of each first collection according to the corresponding number of similar recommendations of each first collection, the frequency suppression score of the first collection is negatively correlated with the corresponding number of similar recommendations; sorting the n first collections according to the content score and frequency suppression score of each first collection to obtain a first sorting result.

[0010] In this implementation, the collection is scored according to the type of collection to obtain a content score; the frequency suppression score of the collection is calculated according to the number of times the same type of collection has been recommended, and then the n first collections are sorted based on the content score and the frequency suppression score. The content score and the frequency suppression score can quantitatively and appropriately reflect the relative attractiveness of each collection, the user's relative preference and interest in each collection, etc. Therefore, the image collection to be recommended determined based on the first sorting result is more in line with user expectations, more able to attract users, and improve user experience. In addition, the frequency suppression score is negatively correlated with the number of times the same type of collection has been recommended, which prevents the same type of collection from being recommended too many times, prevents users from being disturbed, and further makes the recommended collection types more diverse, further improving the user experience.

[0011] In a possible implementation, n first collections are sorted according to the content scores and frequency suppression scores of each first collection to obtain a first sorting result, including: sorting the n first collections according to the content scores to obtain a content sorting result; sorting the n first collections according to the frequency suppression scores to obtain a frequency sorting result; determining the recommendation score of each first collection according to the sum of the ranking of each first collection in the content sorting result and the ranking in the frequency sorting result; sorting the n first collections according to the recommendation scores to obtain the first sorting result.

[0012] In this implementation, the recommendation score of each first collection is determined based on the sum of the ranking of each first collection in the content sorting result and the ranking of each first collection in the frequency sorting result. The recommendation score can reflect the relative attractiveness of each collection in the n first collections, the relative preference and interest of the user in each collection, etc., and can more accurately and quantitatively reflect the possibility of the collection being recommended, so that the determined collection to be recommended is more in line with the user's preferences, thereby improving the user experience.

[0013] In one possible implementation, the type of the image collection includes multiple preset attributes, and the type of each first collection includes attribute values ​​corresponding to each preset attribute; according to the type of each first collection, the content score of each first collection is determined separately, including: for the second collection, according to the preset correspondence between the attribute value and the score, determining the score corresponding to each attribute value in the type of the second collection; and taking a weighted sum of the scores corresponding to each attribute value to obtain the content score of the second collection.

[0014] In this implementation, multiple attributes of the collection are scored separately according to the attribute values ​​and the preset correspondence, and the scoring results are weighted and summed to obtain the content score of the collection. In this way, the degree to which the collection meets the user's expectations and the user's possible interest level can be evaluated more carefully, accurately and quantitatively from multiple aspects, so that the recommended collection is more in line with the user's expectations and improves the user experience.

[0015] In one possible implementation, multiple preset attributes include a character subject, a generation time, and a time span. The character subject represents the characters included in the images in the collection, the generation time represents the time characteristics of the collection generation, and the time span represents the time difference between the latest shooting time and the earliest shooting time of the images in the collection.

[0016] In a possible implementation, the frequency suppression score of each first collection is determined according to the number of times the same type of recommendation corresponding to each first collection has been made, including: determining the frequency suppression score of each first collection according to the following formula: frequency =1 / log(n class +1); where y frequencyIndicates the frequency suppression score of the first collection, indicating the number of times the same category corresponding to the first collection has been recommended.

[0017] In this implementation, the frequency suppression score corresponding to a smaller number of recommendations is relatively large, and the frequency suppression score corresponding to a larger number of recommendations is small through the above formula. In this way, it is reflected in the final collection recommendation as follows: the image collection types with a large number of recommendations are suppressed to a certain extent. In other words, the image collection types with low frequency recommendations are recommended as much as possible, and the image collection types with high frequency recommendations are suppressed to a certain extent. In this way, it can prevent the same type of image collection from being disturbed by too high a recommendation frequency, thereby improving the user experience; and image collections with a lower recommendation frequency are more likely to attract users, further improving the user experience.

[0018] In one possible implementation, determining a collection of images to be recommended based on a first sorting result includes: taking the collection ranked first in the first sorting result as the third collection, and determining whether the third collection meets a preset repetition rate condition; if the third collection meets the preset repetition rate condition, determining the third collection as the collection of images to be recommended; if the third collection does not meet the preset repetition rate condition, taking the fourth collection as the third collection, and returning to the execution step of determining whether the third collection meets the preset repetition rate condition, until the collection of images to be recommended is determined or n first collections are traversed; the fourth collection refers to a collection whose ranking is adjacent to the third collection in the first sorting result and whose ranking is after the third collection.

[0019] In this implementation, the collections are ranked according to the recommendation scores, and the repetition rate is detected in order according to the rankings, that is, whether the collection meets the preset repetition rate condition, so as to screen out the collection to be recommended. In this way, a collection that better meets the user's expectations can be selected from multiple collections, which increases the user's interest and thus improves the user experience.

[0020] In a possible implementation, the preset repetition rate condition is: there is no fifth collection, the fifth collection is a collection of the same type as the third collection and has been recommended, and the number of the same images in the fifth collection and the third collection is less than a preset threshold.

[0021] In a possible implementation, the time difference between the current moment and the recommended moment of the fifth collection is less than the first preset duration.

[0022] In these implementation methods, when screening collections, a repetition rate check is required, that is, to determine whether the collection meets the preset repetition rate conditions. Only collections that meet the preset repetition rate conditions can be determined as collections to be recommended. This can prevent the recommended collection from having too many duplicate images with recently recommended collections, further increasing user interest and improving user experience.

[0023] In a possible implementation, the method includes: if the third collection does not meet a preset repetition rate condition, deleting the third collection.

[0024] In this implementation, if the preset repetition rate condition is not met, it means that the third collection has a high repetition rate with the image collections of the same type recommended within the most recent first preset time period, that is, there are many identical images, and the user is unlikely to be interested in the image collection, so the image collection is not recommended, and will not be recommended again in the future, so the image collection is deleted. In this way, not only can image collections that the user is not interested in be prevented from being recommended to the user, but also the image collection can be prevented from being included in the screening range when the next collection is recommended, thereby reducing the amount of screening operations and alleviating the algorithm operation load.

[0025] In a possible implementation, the method also includes: if no image collection to be recommended is determined by traversing n first collections, determining whether the time difference between the current moment and the moment of the most recent recommendation of the sixth collection is greater than a preset time interval; if so, selecting the image collection to be recommended from the sixth collection of at least one image collection, the sixth collection being a collection of a second priority type, and the second priority level being lower than the first priority level.

[0026] In this implementation, when the image collection to be recommended cannot be screened out from the collection of the first priority, further screening is performed from the collection of the second priority, and the collection that attracts the user is recommended to the user as much as possible, thereby improving the user experience. When screening from the collection of the second priority, the screening is performed only when it is determined that the time difference between the current moment and the moment when the second priority collection was most recently recommended is greater than a preset time interval, that is, for the collection of the second priority, the time interval between two adjacent recommendations must be greater than or equal to the preset time interval, so that frequent recommendations can be prevented, excessive disturbance to the user can be prevented, and the user experience can be further improved.

[0027] In one possible implementation, the method also includes: if there is a first collection in at least one image collection, determining whether the first collection meets a preset repetition rate condition; if the first collection meets the preset repetition rate condition, determining the first collection as the image collection to be recommended; if the first collection does not meet the preset repetition rate condition, deleting the first collection, and determining whether the time difference between the current moment and the moment when the sixth collection was last recommended is greater than a preset time interval; if so, selecting the image collection to be recommended from the sixth collection in at least one image collection, the sixth collection being a collection whose priority type is a second priority, and the level of the second priority is lower than the first priority.

[0028] In this implementation, when there is only one first-priority collection in at least one image collection, a repetition rate detection is directly performed on the priority collection to prevent the repetition rate between the collection and the images in the recommended collection from being too high, thereby preventing the user experience from being affected.

[0029] In one possible implementation, the method also includes: if the first collection does not exist in at least one image collection, determining whether the time difference between the current moment and the moment when the sixth collection was last recommended is greater than a preset time interval; if so, selecting the image collection to be recommended from the sixth collection in at least one image collection, the sixth collection being a collection with a priority type of the second priority, and the level of the second priority being lower than the first priority.

[0030] In this implementation, when there is no collection of the first priority in at least one image collection, the collection to be recommended is screened from the collection of the second priority, and a collection that attracts the user is recommended to the user as much as possible, thereby improving the user experience.

[0031] In one possible implementation, at least one image collection includes a target collection and a generated collection, the generated collection is an image collection that has been generated and not recommended, and before obtaining at least one image collection, the method also includes: obtaining social information, the social information characterizing the social relationship between the characters contained in the images in the gallery; based on the social information, if it is determined that the images containing the characters in the gallery meet the first collection generation condition, then obtaining images that meet the second condition from the gallery to generate a first candidate collection; deduplicating the images in the first candidate collection to obtain a second candidate collection; and sampling the second candidate collection based on the shooting time and shooting location of the images in the second candidate collection to obtain the target collection.

[0032] The target collection is the collection generated according to the generation conditions. The generated collection is the collection that has not been recommended in history. Social information is the result of social circle analysis.

[0033] In this implementation, according to the social information and the first collection generation condition, the generated image collection can reflect the social relationship between characters, and thus is more in line with the user's actual social situation. In the subsequent collection recommendation, the image collection that better meets the user's expectations can be recommended to the user, thereby increasing the user's interest in the image collection and improving the user experience. In addition, the method provided by this implementation performs deduplication when selecting images from the candidate collection to generate the target collection, to prevent duplicate images from affecting the user experience, and also considers the time and location of the image when selecting the image, so that the images in the target collection cover the user's "memories" in the time and location dimensions to the greatest extent, thereby improving the user experience.

[0034] In a possible implementation, the social information includes the intimacy between each person and the first person, and the first collection generation conditions include: the most recently added images to the gallery include at least one first image; the first image is an image containing the second person, and the intimacy between the second person and the first person is greater than a preset intimacy threshold; the sum of the number of historical images and first images in the gallery is greater than or equal to a preset number, and the historical image refers to the image containing the second person in the gallery before the most recently added image; the time difference between the first shooting moment and the second shooting moment is greater than a second preset duration, and the first shooting moment refers to the latest shooting moment of the historical images; the second shooting moment refers to the earliest shooting moment of at least one first image.

[0035] The collection generated according to the above conditions is called an old friend reunion collection. In this embodiment, by mining the association between the newly added images and the images in the gallery and the social relationships of the characters, an old friend reunion collection is generated, and image collections with strong storytelling, high excitement, and user attraction are recommended to users, thereby increasing the richness of the recommended image collections and improving the user experience.

[0036] In a possible implementation, the image satisfying the second condition is an image containing the second person. That is, an image containing the second person is obtained. The second person is also an "old friend".

[0037] In a possible implementation, the second candidate collection is sampled according to the shooting time and shooting location of the images in the second candidate collection to obtain a target collection, including: grouping the images in the second candidate collection according to the shooting location and shooting time to obtain at least one comprehensive group; grouping the images in each comprehensive group according to the aesthetic score of the images in each comprehensive group to obtain a preliminary screening group and a candidate group corresponding to each comprehensive group, the aesthetic score of the image in the preliminary screening group is greater than a preset score threshold, and the aesthetic score in the candidate group is less than or equal to the preset score threshold; the aesthetic score represents the aesthetic quality of the image; determining the expected number of samples corresponding to each comprehensive group; for a first comprehensive group, if the number of images in the first preliminary screening group is 0, then the first comprehensive group is not sampled; the first comprehensive group is any one of the at least one comprehensive group, and the first preliminary screening group is the first comprehensive group. a preliminary screening group corresponding to the group; if the number of images in the first preliminary screening group is greater than or equal to the first expected sampling number, the first expected sampling number of images are sampled from the first preliminary screening group, and the first expected sampling number is the expected sampling number corresponding to the first comprehensive group; if the number of images in the first preliminary screening group is greater than 0 and less than the first expected sampling number, the first number of images in the first candidate group are selected and added to the first preliminary screening group to obtain a supplementary group; if the number of images in the supplementary group is greater than or equal to the first expected sampling number, the first expected sampling number of images are sampled from the supplementary group; if the number of images in the supplementary group is less than the first expected sampling number, the first comprehensive group is not sampled; the first candidate group is the candidate group corresponding to the first comprehensive group, and the first number is determined according to the number of images in the first candidate group; the images sampled from at least one comprehensive group are aggregated to obtain a target collection.

[0038] Comprehensive grouping is also called comprehensive bucketing.

[0039] In this implementation, when sampling the second candidate collection to generate the target collection, the images are grouped according to the time dimension and the location dimension to obtain the initial screening group (high aesthetic score group) and (low aesthetic score group). When sampling each comprehensive group, sampling is performed from the initial screening group (high aesthetic score group) first. If the sampling is insufficient, the images in the candidate group (low aesthetic score group) are added to the initial screening group (high aesthetic score group) and then sampled. In this way, while the quality of the images in the sampled image collection is guaranteed as much as possible, the number of samples of the images in the group is guaranteed, so that the location distribution and time distribution of the images in the obtained image collection are as close to a uniform distribution as possible, covering the user's "memory" and improving the user experience.

[0040] In one possible implementation, the images in the second candidate collection are grouped according to the shooting location and the shooting time to obtain at least one comprehensive group, including: grouping the images in the second candidate collection according to the shooting location to obtain at least one location group; grouping the images in each location group according to the shooting time to obtain at least one comprehensive group; for the first comprehensive group, determining the first expected sampling number, including: determining the first attenuation weight of the shooting location corresponding to the first location group according to the number of images in the first location group, the first attenuation weight is negatively correlated with the number of images in the first location group; determining the first expected sampling number according to the product of the number of images in the first comprehensive group, the first attenuation weight and the first preset ratio.

[0041] In this implementation, the first attenuation weight can be used to suppress the frequency of occurrence of images of high-frequency locations. High-frequency locations refer to locations where a large number of images are taken. This prevents too many images of the same location from being included in the image collection, which affects the user experience.

[0042] In a possible implementation, the first number is determined according to the product of the number of images in the first candidate group, the first attenuation weight, and the second preset ratio.

[0043] In this implementation, when adding images to the initial screening group, the images of the candidate group are not added as a whole, or are added according to the number of missing images. Instead, the number of images to be added is calculated based on the number of images in the candidate group, the frequency attenuation weight of the location, and the preset ratio. This allows both image quality and distribution uniformity to be taken into account when adding images, preventing too many low-quality images from entering the final image collection, ensuring the distribution uniformity of the final image collection, and improving the user experience.

[0044] In a possible implementation, sampling a first expected sampling number of images from a first preliminary screening group includes: sampling the first expected sampling number of images from the first preliminary screening group according to a comprehensive score of each image in the first preliminary screening group; the comprehensive score is determined based on at least two of an aesthetic score, an interaction score, and a semantic score of the image, the interaction score is determined based on an amount of interaction between a user and the image, the semantic score is determined based on an amount of interaction between the user and images of the same type of semantics, and the aesthetic score represents the aesthetic quality of the image.

[0045] In this implementation, by comprehensively fractionally sampling images, the quality of images in the final target collection is made higher, thereby improving user experience.

[0046] In a possible implementation, before obtaining social information, the method further includes: after adding an image containing a person to the gallery, obtaining a person image heterogeneous graph and user blocking information; the person image heterogeneous graph represents the correspondence between the images in the gallery and the persons contained in the images, and the user blocking information refers to information related to the images in the gallery that is blocked by the user; based on the person image heterogeneous graph and the user blocking information, the images in the gallery are analyzed to obtain social information.

[0047] In a possible implementation, after sampling the second candidate collection according to the shooting time and shooting location of the images in the second candidate collection to obtain the target collection, the method further includes: recording the type of the target collection and the number of images in the target collection.

[0048] In one possible implementation, after determining the image collection to be recommended according to the first sorting result, the method further includes: recording at least one of the type and the total number of recommendations corresponding to the image collection to be recommended, the total number of recommendations being: a value obtained by adding one to the number of times the same type of recommendation corresponding to the image collection to be recommended has been recommended; the number of times the same type of recommendation corresponding to the image collection to be recommended refers to the number of times an image collection of the same type as the image collection to be recommended has been recommended.

[0049] In a second aspect, the present application provides a device, which is included in an electronic device, and the device has the function of implementing the electronic device behavior in the above-mentioned first aspect and the possible implementation of the above-mentioned first aspect. The function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a receiving module or unit, a processing module or unit, etc.

[0050] In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and an interface; the processor, the memory, and the interface cooperate with each other so that the electronic device executes any one of the methods in the technical solution of the first aspect.

[0051] In a fourth aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory to perform the method in the first aspect and any possible implementation thereof.

[0052] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0053] Further optionally, the chip also includes a communication interface.

[0054] In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes any one of the methods in the technical solution of the first aspect.

[0055] In a sixth aspect, the present application provides a computer program product, the computer program product comprising: a computer program code, when the computer program code runs on an electronic device, the electronic device executes any one of the methods in the technical solution of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a schematic diagram of an application scenario of an image collection recommendation method provided in an embodiment of the present application;

[0057] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0058] Figure 3 is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0059] Figure 4 is a flowchart of an image collection recommendation method provided in an embodiment of the present application;

[0060] Figure 5 This is an example of a person-image isomorphic graph provided in an embodiment of the present application;

[0061] Figure 6 This is a schematic diagram of an interface showing a collection to be recommended provided in an embodiment of the present application;

[0062] Figure 7 is a flowchart of another collection recommendation method provided in an embodiment of the present application;

[0063] Figure 8 is a curve diagram of a frequency suppression fraction provided in an embodiment of the present application;

[0064] Fig. 9 is a flowchart of another method for recommending image collections provided in an embodiment of the present application;

[0065] Fig.10 This is a comparison diagram of the time distribution of an example of candidate collection 2 provided in an embodiment of the present application and the images in the old friends reunion that are finally generated;

[0066] Fig.11 This is a comparison diagram of the location distribution of an example of candidate collection 2 provided in an embodiment of the present application and the images in the old friends reunion that are finally generated;

[0067] Fig.12 is a schematic diagram of an image sampling process provided in an embodiment of the present application;

[0068] Fig.13 is a flowchart of an image scoring process provided in an embodiment of the present application;

[0069] Fig.14 This is an example of a character connection diagram provided in an embodiment of the present application;

[0070] Fig.15 This is a social circle diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0072] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0073] References to "one embodiment" or "some embodiments" etc. described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in the specification of the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0074] In order to better understand the embodiments of the present application, the terms or concepts that may be involved in the embodiments are explained below.

[0075] 1. Heterogeneous Graph

[0076] A heterogeneous graph refers to a graph in which the sum of the node type and the edge type is greater than 2. In other words, a graph in which the node type + the edge type > 2.

[0077] 2. Isomorphic graph

[0078] An isomorphic graph is a graph that has only one type of node and one type of edge.

[0079] 3. Pointwise mutual information (PMI)

[0080] Mutual information is used to measure the correlation between two things. The larger the mutual information, the greater the correlation between the two things. The smaller the mutual information, the smaller the correlation between the two things.

[0081] 4. Shooting time information

[0082] The shooting time information of an image refers to information related to the shooting time of the image. Optionally, the shooting time information may include the shooting time, the week information corresponding to the shooting time (also called week information), etc. The shooting time may include year information, month information, day information, hour information, minute information, and second information, etc. It can be understood that the type of information included in the shooting time may be different depending on the timing accuracy.

[0083] For ease of understanding, in the embodiments of the present application, the year information, month information and day information are collectively referred to as date information, and the hour information, minute information and second information are collectively referred to as timing information. For example, the shooting time of image A is 17:18:12 on April 12, 2023, then the year information contained in the shooting time is 2023, the month information is April, the day information is the 12th, the hour information is 17 hours, the minute information is 18 minutes, and the second information is 12 seconds. Among them, the date information is April 12, 2023, and the timing information is 17 hours, 18 minutes and 12 seconds. It can also be understood that the types of information included in the date information and the timing information may be different due to different timing accuracy.

[0084] 5. Temporal distribution of images

[0085] The embodiments of the present application involve the time distribution of images. The time distribution of images represents the distribution of the shooting moments of the images, that is, the distribution of the number of images in different time intervals is determined according to the shooting moments. For example, according to the timing information in the shooting moments, the number of images in the time interval TA is determined to be pic1, and the number of images in the time interval TA2 is pic2.

[0086] 6. Location distribution

[0087] The embodiments of the present application involve the location distribution of images. The location distribution of images represents the distribution of the shooting locations of the images, that is, the distribution of the number of images of different locations determined according to the shooting locations. For example, it is determined that the number of images shot at location L1 is pic3, and the number of images shot at location L2 is pic4.

[0088] 7. Divergence

[0089] Divergence is a data distribution characteristic and a commonly used indicator in statistics to describe group structure. It is used to indicate the degree of deviation of data from the mean. Divergence can characterize the uniformity of data distribution.

[0090] The time distribution divergence refers to the degree of deviation of the time distribution from the mean of the time distribution (i.e., uniform time distribution). The time distribution divergence can reflect the uniformity of the time distribution. The smaller the time distribution divergence, the more uniform the time distribution is. The larger the time distribution divergence, the more uneven the time distribution is, i.e., the more concentrated it is.

[0091] The location distribution divergence refers to the degree of deviation of the location distribution from the mean of the location distribution (i.e., uniform location distribution). The location distribution divergence can reflect the uniformity of the location distribution. The smaller the location distribution divergence, the more uniform the location distribution is. The larger the location distribution divergence, the more uneven the location distribution is, i.e., the more concentrated it is.

[0092] 8. KL divergence

[0093] KL is the abbreviation of Kullback-Leibler. KL divergence is used to measure the difference between two distributions. The larger the KL divergence, the greater the difference between the two distributions. The smaller the KL divergence, the smaller the difference between the two distributions. The value of KL divergence can be [0,1]. When the two distributions are consistent, the KL divergence is 0.

[0094] 9. Co-occurrence relationship

[0095] Co-occurrence refers to co-appearance. In the embodiment of the present application, the co-occurrence relationship refers to the co-occurrence relationship between characters in an image. In other words, the co-appearance of characters in an image is called the co-occurrence relationship between characters.

[0096] 10. Multi-channel recall

[0097] In the data recommendation process, candidate sets can be generated according to different strategies, features or models. The collection generated according to one strategy, feature or model can be called a one-way candidate set. Multi-way recall refers to recalling candidate sets produced by different strategies, features or models. Candidate sets of multi-way recall can be mixed together for subsequent sorting, screening or recommendation.

[0098] Before describing in detail the image collection recommendation method provided in the embodiment of the present application, the application scenario of the method and the structure of the applicable electronic device are first described.

[0099] Optionally, the image collection recommendation method provided in the embodiment of the present application can be applied to a gallery application of an electronic device to implement functions such as "moments" or "wonderful moments" or "memories" of the gallery application. The following is an explanation in conjunction with the interface diagram.

[0100] For example, Figure 1 Schematic diagram of an application scenario of an image collection recommendation method provided in an embodiment of the present application. Taking a mobile phone as an example, Figure 1 As shown in FIG. 1 (a), the main interface of the mobile phone includes an icon 101 of a gallery application. The user clicks the icon 101, and in response to the user's click, the mobile phone opens the gallery application and displays Figure 1 The interface shown in (b) in FIG. 1 includes a time control 102. The user clicks the time control 102, and in response to the user's click, the mobile phone displays the interface of the "time" function, such as Figure 1 As shown in Figure (c) in the figure. This interface can include image collections (also called albums) with multiple themes such as "Weekend Time", "Selected Portraits", and "Happy New Year". These image collections are generated by the mobile phone based on the images in the user's gallery and are recommended to the user in sequence at different times.

[0101] Then, how to recommend image collections that meet the user's expectations to the user, increase the user's interest in the recommended image collection, and thus improve the user experience is the problem to be solved by the image collection recommendation method provided in the embodiment of the present application. The method determines the recommendation score of the multi-way recalled image collection according to the type of image collection, recommendation frequency, etc., and recommends the image collection to the user based on the recommendation score, so that the recommended collection is more in line with the user's expectations and improves the user experience.

[0102] The image collection recommendation method provided in the embodiment of the present application can be applied to mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other electronic devices that can install application programs (APP). The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0103] For example, Figure 21 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0104] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0105] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0106] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0107] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0108] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0109] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Figure 2 The structures of antenna 1 and antenna 2 in the electronic device 100 are only examples. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0110] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0111] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0112] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0113] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0114] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0115] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0116] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0117] Figure 3 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0118] The application layer can include a series of application packages. Figure 3 As shown in the figure, the application package may include a gallery application and an artificial intelligence (AI) plug-in engine application. In addition, the application layer may also include camera, call, map, Bluetooth, short message and other applications, which are not shown in the figure.

[0119] It is understood that the gallery application may have its own database, hereinafter referred to as database 1 (not shown in the figure). Database 1 is used to store images and image information taken by the user through the camera, or images and image information downloaded through the network.

[0120] The AI ​​plug-in engine is also called collection recommendation, story recommendation, wonderful moments or memories, etc. The AI ​​plug-in engine can include an interface layer, a solution layer, and an algorithm plug-in layer. The interface layer can be open to the outside world to achieve docking and data interaction between the AI ​​plug-in engine and other applications. The solution layer is used to implement collection generation and recommendation functions. As a support for the solution layer, the algorithm plug-in layer provides relevant algorithms for the solution layer.

[0121] Optionally, the interface layer may include a collection recommendation toolkit. The solution layer includes a collection generation and recommendation module. The algorithm plug-in layer includes a social analysis module, an image scoring module, and a duplicate demove module. The collection recommendation toolkit is also called a story recommendation toolkit, which is used to interact with other applications (such as a camera application). Specifically, the collection recommendation toolkit can synchronize data to other applications, transmit instructions from other applications to the collection generation and recommendation module of the solution layer, and transmit the results output by the collection generation and recommendation module to other applications. The collection generation and recommendation module is used to realize the generation and recommendation of image collections. The social analysis module is used to accept requests from the collection generation and recommendation module, analyze the social relationships of the characters, and obtain social analysis results. The image scoring module is used to accept requests from the collection generation and recommendation module and score images. The duplicate removal module is used to accept requests from the collection generation and recommendation module and perform duplicate removal on images in the candidate collection.

[0122] It can be understood that the collection recommendation toolkit may have its own database, hereinafter referred to as database 2. Database 2 is used to store images, image information, generated image collections, and the like.

[0123] It should be noted that the structure of the above-mentioned AI plug-in engine, as well as the layer where the AI ​​plug-in engine is located, are all examples and are not intended to be limiting. In actual applications, the AI ​​plug-in engine may include more or fewer modules than those described above, and each module in the AI ​​plug-in engine may also be located in other layers of the software architecture. In addition, when implementing their functions, the gallery application and the AI ​​plug-in engine may also call related modules of other layers of the software architecture, and the embodiments of the present application do not impose any limitations on this.

[0124] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0125] The application framework layer may also include a window manager, content provider, view system, telephony manager, resource manager, notification manager, etc.

[0126] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0127] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0128] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0129] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0130] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0131] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0132] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0133] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0134] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0135] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0136] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0137] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0138] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0139] A 2D graphics engine is a drawing engine for 2D drawings.

[0140] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0141] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 2 and Figure 3 Taking the electronic device of the structure shown as an example, the image collection recommendation method provided by the embodiment of the present application is specifically described in combination with the accompanying drawings and application scenarios. For the convenience of explanation, the method is described below according to the data synchronization stage, the social circle analysis stage, the collection generation stage and the collection recommendation stage.

[0142] Figure 4 : is a flowchart of an image collection recommendation method provided in an embodiment of the present application, the method comprising:

[0143] 1. Data synchronization stage

[0144] S101. In response to a user adding an image containing a person to database 1 of a gallery application, or deleting an image containing a person from database 1 of the gallery application, the gallery application obtains information of an incremental image, wherein the incremental image refers to an image containing a person added to database 1, or an image containing a person deleted from database 1.

[0145] Optionally, in an embodiment of the present application, the information of the image may include, but is not limited to, one or more of the following: identification information of the image, characters contained in the image, the age of the characters, the gender of the characters, label information corresponding to the image, the aesthetic score of the image (also called aesthetic score, aesthetic rating, etc.), the shooting time information of the image, the shooting location information of the image, and the camera information used to shoot the image.

[0146] Optionally, the identification information of the image can be represented by a hashed identity document (ID). The person contained in the image can be represented by a person ID (also called a face ID). The aesthetic score refers to the score obtained by scoring the image from an aesthetic point of view. The aesthetic score can reflect the beauty of the image, the image quality, etc., and also reflect the user's liking for the image. Generally speaking, the higher the aesthetic score, the more beautiful the image, the higher the image quality, and the higher the user's liking for the image.

[0147] Specifically, for images added by users to database 1, the gallery application can perform computer vision (CV) analysis on the newly added images to obtain information such as the person, the person's age, and the person's gender contained in the newly added images. Optionally, CV analysis includes but is not limited to face clustering, aesthetic scoring, gender recognition, age recognition, and image aesthetic score calculation. At the same time, the gallery application analyzes the newly added images to obtain information such as shooting time information and shooting location information. The gallery application can save the information of the newly added images to database 1.

[0148] For the image deleted by the user from the database 1, since the image is added to the database 1, the gallery application saves the information of the image to the database 1 according to the above method. Therefore, in this step, the gallery application can obtain the information of the deleted image from the database 1.

[0149] S102. The gallery application synchronizes data with the collection recommendation toolkit in the AI ​​plug-in engine.

[0150] It can be understood that for deleted images, the gallery application can only synchronize the identification information of the incremental images to the collection recommendation toolkit; for newly added images, the gallery application can synchronize the newly added images and their information to the collection recommendation toolkit.

[0151] As a possible implementation, when synchronizing data, the gallery application may synchronize data only for images containing people that are added or deleted in a preset path.

[0152] S103. The collection recommendation toolkit updates the images and image information in database 2 of the AI ​​plug-in engine according to the information of the incremental images.

[0153] Specifically, for deleted images, the collection recommendation toolkit can delete the corresponding images in database 2 according to the information of the incremental images; for newly added images, the collection recommendation toolkit can save the newly added images and the information of the newly added images in database 2.

[0154] It can be understood that when there are incremental images containing people in database 1, the gallery application triggers the synchronization of data to the collection recommendation toolkit. Therefore, the images in database 2 can all be images containing people.

[0155] S104. The collection recommendation toolkit feeds back the data synchronization result to the gallery application.

[0156] Optionally, the data synchronization result may include, for example, data synchronization success, data synchronization failure, data error, etc.

[0157] 2. Social circle analysis stage

[0158] S105: When data synchronization is successful, the charger of the electronic device is plugged in, the screen of the electronic device is off, and the current time is in a preset time period, the gallery application sends a start analysis instruction to the collection recommendation toolkit. The start analysis instruction is used to instruct to start image collection generation and image collection recommendation.

[0159] Optionally, the gallery application can subscribe to the charger plug-in status from the relevant module in the electronic device, and when the charger plug-in status changes, the module sends a notification to the gallery application. Similarly, the gallery application can also subscribe to the screen status from the relevant module in the electronic device, and when the screen status changes, the module sends a notification to the gallery application. In this way, the gallery application can know in real time whether the charger is plugged in and whether the screen is off.

[0160] Optionally, the preset time period may be, for example, 00:00 to 07:00 every day.

[0161] As another possible implementation, the gallery application may also directly send a start analysis instruction to the collection recommendation toolkit after determining that the data synchronization is successful. That is, if the data synchronization result is successful in step S104, the gallery application is directly triggered to send a start analysis instruction to the collection recommendation toolkit.

[0162] S106. In response to the start analysis instruction, the collection recommendation tool updates the character-image heterogeneity graph according to the information of the incremental image, and determines the total number of characters included in the updated character-image heterogeneity (ie, the total number of characters after update).

[0163] The person-image heterogeneity graph is used to represent the correspondence between a person and an image containing the person. The total number of persons included in the person-image heterogeneity graph is also called the face number.

[0164] It can be understood that when the image in database 2 changes, the corresponding person-image heterogeneous graph also changes. After receiving the start analysis instruction, the collection recommendation tool adjusts the person-image heterogeneous graph according to the information of the incremental image and the information of the historical image in database 2. Specifically, each person currently contained in database 2 (i.e., after update) is regarded as a person node, each image containing a person is regarded as an image node, and the image node containing a certain person is connected to the person node corresponding to the person through an edge to obtain an updated person-image heterogeneous graph. For example, if image A includes person a, the node of image A is connected to the node of person a through an edge; the same is true for other images and other persons in database 2. In this way, a person-image heterogeneous graph can be obtained. In other words, the person-image heterogeneous graph includes person nodes and image nodes, and the edge between a person node and an image node indicates that the image contains the person, or the edge between a person node and an image node indicates that the person belongs to the image.

[0165] Optionally, the person-image heterogeneous graph may also include node information of person nodes and image nodes. Specifically, the information of the person node is also the relevant information of the person corresponding to the person node, including but not limited to the person ID, the person gender and the person age. The information of the image node is also the information related to the image corresponding to the image node, including but not limited to the image hash ID, the image shooting time information and the image shooting location, etc.

[0166] For example, Figure 5 An example of a person-image isomorphic graph provided in an embodiment of the present application. Figure 5 In the figure, the icons with round borders represent person nodes, and the icons with square borders represent image nodes. It can be understood that Figure 5 It is only used as an example to illustrate the composition structure of the person-image heterogeneous graph, etc. It is not intended to be limiting and does not represent actual data.

[0167] It should be noted that Figure 5 It is a concrete representation of the character-image heterogeneous graph. In practical applications, the character-image heterogeneous graph can be expressed in various forms. The embodiment of the present application does not impose any limitation on the specific expression form of the character-image heterogeneous graph, as long as it can reflect its content.

[0168] The number of character nodes included in the updated character-image heterogeneous graph is also the updated total number of characters.

[0169] S107. The collection recommendation tool determines whether the total number of characters after the update has changed compared to the total number of characters before the update.

[0170] The collection recommendation toolkit can save and manage the historical character connection diagram and the total number of historical characters. After each update of the total number of characters, the updated total number of characters is compared with the total number of characters before the update to determine whether the total number of characters before and after the update has changed.

[0171] S108. If the total number of characters after the update changes compared to the total number of characters before the update, the collection recommendation toolkit sends a collection recommendation analysis instruction and a social circle analysis instruction to the collection generation and recommendation module in the AI ​​plug-in engine.

[0172] The collection recommendation analysis instruction is used to instruct the generation of an image collection and recommend the collection. The social circle analysis instruction is used to instruct the social circle analysis of the images in the database 2. The social circle is used to characterize the social relationship between the central character and other characters in a number of images. Optionally, the social relationship can be characterized by parameters such as the division of the image, the type of relationship between the characters, and the degree of intimacy.

[0173] The central person refers to a person who is at the social center among the persons included in the multiple images. Optionally, the number of central persons can be one or more, which can be set according to actual needs. The central person can be the owner of the electronic device or a non-owner, such as a family member of the owner.

[0174] Intimacy is used to characterize the intimacy of the social relationship between two characters, that is, the degree of association between the two characters. The intimacy between two characters is a value greater than or equal to 0. The greater the intimacy, the closer the two characters are. If the intimacy is 0, it means that there is no social relationship between the two characters. The analysis process of the social circle, the determination of the central character, and the calculation of intimacy will be further explained in the subsequent embodiments.

[0175] It is understandable that when the total number of characters changes, the results of the social relationship between the central character or the central character and other characters may change. For example, the intimacy between characters may change, and the analysis results of the social circle may change. Therefore, in addition to sending the collection recommendation analysis instruction to the collection generation and recommendation module, the collection recommendation toolkit also sends a social circle analysis instruction to instruct the re-analysis of the social relationship between characters to improve the accuracy of the subsequently generated image collection.

[0176] Optionally, the collection recommendation analysis instruction and the social circle analysis instruction may be sent to the collection generation and recommendation module respectively through two independent instructions, or may be sent to the collection generation and recommendation module through one instruction. The embodiment of the present application does not impose any limitation on this.

[0177] S109. The collection generation and recommendation module responds to the collection recommendation analysis instruction and the social circle analysis instruction by sending a data query request to the collection recommendation toolkit, where the data query request is used to request to query data such as the updated person-image heterogeneous graph and user blocking information.

[0178] User shielding information refers to information related to the image in the database 2 that is shielded by the user. The user shielding information may include images, information about the images, or characters, etc. Optionally, the user shielding information may be represented by a user shielding field. The user shielding information may be pre-marked by the user, or may be obtained by the electronic device based on the user's usage behavior analysis, and the embodiments of the present application do not impose any limitation on this.

[0179] S110. The collection recommendation toolkit returns the queried data to the collection generation and recommendation module.

[0180] Specifically, the data obtained by the query may include data such as an updated person-image heterogeneous graph and user blocking information.

[0181] S111: After receiving the data, the collection generation and recommendation module sends a social circle analysis request to the social analysis module of the algorithm plug-in layer. The social circle analysis request is used to request analysis of the social circle.

[0182] Optionally, the social circle analysis request may carry the queried data (such as the updated person-image heterogeneous graph and user blocking information).

[0183] S112. The social circle analysis module responds to the social circle analysis request, analyzes the social circle according to the person-image heterogeneous graph and user shielding information, and obtains the social circle analysis result (also called social information).

[0184] Specifically, the social circle analysis module analyzes the social circle according to the person-image heterogeneous graph, user blocking information, etc., to obtain the social circle analysis result. The process of social circle analysis will be further described in detail in the subsequent embodiments.

[0185] Optionally, the social circle analysis result may include one or more of those in Table 1.

[0186] Table 1

[0187]

[0188] Wherein, the divergence sum refers to the sum of the time KL divergence and the location KL divergence of a certain person. The divergence sum ranking refers to the result obtained by the divergence sum ranking of multiple persons. In the embodiment of the present application, the image of a certain person, or the image corresponding to a certain person, refers to the image containing the person. For example, the image of person a, or the image corresponding to person a, refers to the image containing person a.

[0189] The co-occurrence image of two characters refers to an image in which the two characters appear together, that is, an image that contains the two characters at the same time. For example, if an image A contains a and a character b, then image A is a co-occurrence image of characters a and b. The time distribution of co-occurrence images refers to the distribution of the shooting time of the co-occurrence images. The location distribution of co-occurrence images refers to the distribution of the shooting locations of the co-occurrence images.

[0190] The type of social circle represents the type of relationship between the characters in the social circle and the central character. Optionally, the type of social circle may include family, colleagues, classmates, friends, etc.

[0191] S113. The social circle analysis module returns the social circle analysis results to the collection generation and recommendation module.

[0192] 3. Collection generation stage

[0193] After completing the social circle analysis according to the above process, the following process is further performed to generate a collection. In the collection generation process, multiple image collections of different types can be generated according to different generation strategies, different generation conditions, etc.

[0194] For ease of explanation, the types of image collections in the embodiments of the present application are first described. Referring to Table 2, the image collections are divided into several types shown in Table 2 according to several attributes such as the characters involved in the images in the collection (hereinafter referred to as the subject of the collection), the generation time of the collection, and the time span of the images in the collection (hereinafter referred to as the time span of the collection). Among them, the generation time of the collection refers to the time when the generation of the collection is triggered. The time span of the collection refers to the time difference between the earliest shooting time and the latest shooting time of the images in the collection.

[0195] Table 2

[0196]

[0197] Among them, the close people of the central character (also referred to as close people) refer to people whose intimacy with the central character meets the preset conditions, for example, people whose intimacy with the central character is greater than the preset intimacy threshold. The newly added image of person f is the image of person f included in the newly added image. The historical image refers to the image in database 2 before the newly added image. The historical image of person f refers to the image of person f in database 2 before the newly added image.

[0198] Among them, the above attributes of the image collection are different, and the final type of the collection is different. Different collection types have different corresponding generation conditions. For example, the generation conditions corresponding to the image collection with the central character as the subject, the generation time as a holiday, and the time span as 1 year include: a. The shooting time is within a preset time period, which is a time period including holidays; b. The number of images of close people in database 2 meets the quantity requirements.

[0199] It should be noted that the data in Table 2 are for example only and do not constitute any limitation to the present application.

[0200] As shown in Table 2, the types of the subject of the collection can include the central person, the central person and close people, the central person's close people and the central person's social circle, etc. The types of the generation timing of the collection can include regular generation, incremental type and holidays, etc. The types of the time span of the collection can include 1 month, 1 quarter, 1 year, etc.

[0201] For each image collection, the attribute values ​​corresponding to the collection's main characters, generation time, and time span determine the final type of the image collection. Optionally, the type identifier of the collection can be generated through a preset type naming rule to facilitate the subsequent management of the collection. For example, a collection type identifier field including three marker bits can be defined, wherein the first marker bit indicates the generation time, the second marker bit indicates the main characters of the collection, and the third marker bit indicates the time span. At the same time, the value of each marker bit is defined. For example, the value of the first marker bit can be fixed, holiday, or incremental, wherein fixed indicates regular generation, holiday indicates holidays, and incremental indicates incremental classes. Similarly, the second marker bit and the third marker bit can also represent different attributes through different values. For example, if the type identifier of a collection is fixed_01_01, it means that the type of the collection is a regularly generated image collection with a central character and a time span of 1 month.

[0202] Next, continue to combine Figure 4, the collection generation stage in this method is described. The collection generation stage may include:

[0203] S114. The collection generation and recommendation module obtains configuration parameters corresponding to various types of collection generation conditions from the cloud service.

[0204] The configuration parameters corresponding to the collection generation conditions are the parameters required in the collection generation conditions. For example, for a collection whose main character is the central character and whose generation timing is periodic generation, the configuration parameters corresponding to its generation conditions may include: the value of the time period requirement, the value of the quantity requirement, etc. For another example, for a collection whose main characters are the central character and close people and whose generation timing is incremental, the configuration parameters corresponding to its generation conditions may include the value of the preset intimacy x, the value of the preset threshold y, and the value of the preset duration z, etc.

[0205] Optionally, the collection generation and recommendation module can save the configuration parameters obtained from the cloud server. The obtained configuration parameters can have a preset time limit. When the preset time limit is exceeded, the collection generation and recommendation module obtains the configuration parameters from the cloud server again.

[0206] In another possible implementation, the electronic device may also directly obtain the configuration parameters from the memory, and the embodiments of the present application do not impose any limitation on this.

[0207] S115. The collection generation and recommendation module determines whether the generation conditions corresponding to various types of collections are met according to the configuration parameters and the social circle analysis results.

[0208] According to the configuration parameters, the specific generation conditions of various types of collections can be determined. According to various information in the social circle analysis results, it is judged whether these specific generation conditions are met.

[0209] S116. If the generation conditions corresponding to a certain type or several types of image collections are met, the collection generation and recommendation module obtains the images and image information required for generating the image collection from the collection recommendation toolkit.

[0210] Different types of image collections require different images. For example, if the generation conditions corresponding to the collection are that the subject is the central character, the generation timing is periodic generation, and the time span is 1 month, then the collection generation and recommendation module obtains the images and image information for generating the collection (type identification is fixed_01_01) from the collection recommendation toolkit, that is, obtains images that include the central character and are taken within 1 month from the current time, as well as information about these images. Optionally, all information about these images can be obtained in this step, or partial information about these images can be obtained as needed, for example, the shooting time, shooting location, aesthetic score, etc. of these images can be obtained.

[0211] S117, the collection generation and recommendation module generates an image collection based on the acquired images.

[0212] Continuing with the above example, the collection generation and recommendation module generates an image collection based on the acquired images that include the central person and are taken within 1 month from the current time.

[0213] It can be understood that according to the generation conditions shown in Table 2, one image collection may be generated, multiple collections may be generated, and of course no collection may be generated. This embodiment is described by taking the generation of one or more image collections as an example.

[0214] Optionally, the collection generation and recommendation module may generate a type identifier of the image collection while generating the image collection, and establish a corresponding relationship between the two.

[0215] In addition, for the same type of image collection, one or more may be generated. Therefore, a collection number may be further set for the image collection to distinguish it. In summary, after the image collection is generated, the identification information corresponding to the image collection may be set accordingly, and the identification information may include a collection type identification and a number identification, etc., and the embodiments of the present application do not impose any limitation on this.

[0216] Moreover, after generating the image collection, the collection generation and recommendation module may further record the number of images in the image collection and establish a corresponding relationship between the number and the image collection identification information.

[0217] 4. Collection recommendation stage

[0218] Continue to see Figure 4 , the collection recommendation stage can include:

[0219] S118, the collection generation and recommendation module obtains multi-channel recall results, which include the collection generated this time (ie, the target collection) and the historical non-recommended collection (ie, the generated collection).

[0220] The historical unrecommended collection refers to the image collection stored in the database 2, which was generated before the current image collection was generated and was not recommended. It can be understood that each time the collection generation and recommendation module responds to the collection recommendation analysis instruction, it can only recommend a preset number of (for example, 1) image collections generated, and the remaining unrecommended image collections can be sent to the collection recommendation toolkit, and the collection recommendation toolkit saves the unrecommended image collection (i.e., the historical unrecommended collection) to the database 2. After receiving the collection recommendation analysis instruction next time, the historical unrecommended collection saved in the database 2 can be used as a recall result and included in the recommendation analysis.

[0221] Optionally, the collection recommendation toolkit may set an expiration date for the historical non-recommended collection in database 2. If the historical non-recommended collection has been stored in database 2 for longer than the expiration date and has not been recommended, the image collection will be deleted.

[0222] In addition, the recommended image collections can also be stored in the database 2 and managed uniformly by the collection recommendation toolkit. The collection recommendation toolkit can also set a recommendation flag bit for the image collections in the database 2 to mark whether the image collection is recommended. Optionally, the value of the recommendation flag bit is 1, indicating that the image collection has been recommended, and the value of the recommendation flag bit is 0, indicating that the image collection has not been recommended.

[0223] Based on this, during multi-channel recall, the collection generation and recommendation module can send a recall request to the collection recommendation toolkit, and the recall request is used to request the recall of historically unrecommended collections. In response to the recall request, the collection recommendation toolkit searches for an image collection whose recommendation mark bit value is 0 from database 2, and sends the image collection whose mark bit value is 0 to the collection generation and recommendation module. At the same time, in the above step S117, the collection generation and recommendation module has generated one or more image collections, that is, one or more image collections are obtained according to the multi-channel recall conditions. At this point, the collection generation and recommendation module obtains the multi-channel recall result.

[0224] S119, the collection generation and recommendation module judges, scores and filters multiple image collections in the multi-channel recall results to obtain a preset number of collections to be recommended.

[0225] Specifically, the collection generation and recommendation module can select an image collection that meets the requirements from multiple image collections according to a preset judgment logic, a preset scoring rule, and a preset screening rule to obtain a collection to be recommended.

[0226] S120, the collection generation and recommendation module sends the identification information of the collection to be recommended, the image collection generated this time, and the social circle analysis results to the collection recommendation toolkit.

[0227] S121. The collection recommendation toolkit saves the image collection and social circle analysis results generated this time.

[0228] S122. The collection recommendation toolkit marks the image collection generated this time as not recommended and marks the collection to be recommended as recommended according to the identification information of the collection to be recommended.

[0229] Specifically, the collection recommendation toolkit modifies the value of the recommendation mark bit of the collection to be recommended to 1, and sets the value of the mark bit of the image collection that is not selected as the collection to be recommended in the image collection generated this time to 0.

[0230] Optionally, the collection recommendation toolkit can also record and manage the number of times each type of image collection is recommended (ie, the recommendation frequency). After marking the collection to be recommended as recommended, the number of times the collection of the same type as the collection to be recommended is recommended is increased by one.

[0231] S123. The collection recommendation toolkit returns the collection to be recommended to the gallery application.

[0232] Optionally, the collection recommendation toolkit may return the collection to be recommended to the gallery application via an asynchronous interface callback.

[0233] S124. The gallery application recommends and displays the collection to be recommended.

[0234] Optionally, the gallery application can display the collection to be recommended in the form of albums or short films in the "Moments", "Highlights" or "Memories" interface. Optionally, the gallery application can display the collection to be recommended in a preset "recommendation position" in the interface. For example, Figure 6 As shown, the collection to be recommended for a certain recommendation can be, for example, Figure 6 A collection of images with the theme name "Old Friends Reunited".

[0235] In another embodiment, based on Figure 4 In step S107, the collection recommendation tool determines whether the total number of characters after the update has changed compared to the total number of characters before the update. Figure 4 The steps S108 to S113 in the above method are replaced by the following steps: if the total number of characters after the update does not change compared to the total number of characters before the update, the collection recommendation toolkit sends a collection recommendation analysis instruction to the collection generation and recommendation module in the AI ​​plug-in engine. After that, the collection generation and recommendation module responds to the collection recommendation analysis instruction and executes step S114. After step S114, it also includes: the collection generation and recommendation module obtains the social circle analysis result from the collection recommendation toolkit.

[0236] That is to say, if the collection recommendation tool determines that the total number of characters after the update has not changed compared to the total number of characters before the update, there is no need to perform social circle analysis, and the collection generation stage and the collection recommendation stage are directly entered.

[0237] The image collection recommendation method provided in the embodiment of the present application can generate an image collection according to the preset generation conditions based on the social circle analysis results, which can reflect the social relationship between characters. Therefore, the generated image collection is more in line with the user's actual social situation, and then in the subsequent collection recommendation, it can recommend to the user an image collection that is more in line with the user's expectations, increase the user's interest in the image collection, and thus improve the user experience. In addition, in this embodiment, according to the multi-channel recall results, a preset number of collections to be recommended are screened and recommended to the user, avoiding excessive disturbance to the user, and can increase the user's interest in the recommended collection, further improving the user experience.

[0238] Above Figure 4 The illustrated embodiment introduces the overall process of collection generation and recommendation. The following describes the process of screening the collection to be recommended during the collection recommendation process (hereinafter referred to as the collection screening process).

[0239] For example, Figure 7 A flowchart of a collection recommendation method provided in another embodiment of the present application is shown as follows: Figure 7 As shown, the above step "S119, the collection generation and recommendation module judges, scores and screens multiple image collections in the multi-channel recall results to obtain a preset number of collections to be recommended" includes the following steps, which are executed by the collection generation and recommendation module and will not be repeated. In this embodiment, the preset number is 1 for example.

[0240] S201. Determine the number of high-priority image collections in the multi-channel recall results; if the number of high-priority image collections is 1, execute step S202; if the number of high-priority image collections is n (n is an integer), and n is greater than or equal to 2, execute step S205; if the number of high-priority image collections is 0, execute step S210.

[0241] Optionally, priorities may be set for the types of image collections, and some types may be set as high priority (also referred to as first priority). In a specific embodiment, a high priority image collection may include: 1) an image collection generated during holidays; 2) an image collection generated regularly with a time span of 1 year. Of course, the specific types of high priority image collections may also be set according to actual needs.

[0242] S202. If the number of high-priority image collections is 1, perform a repetition rate detection on the 1 high-priority image collection to determine whether it passes the repetition rate detection; if it fails, execute step S203; if it passes, execute step S204.

[0243] In this embodiment, performing repetition rate detection on a certain image collection A1 means: determining whether there is an image collection with a repetition rate greater than a preset repetition rate threshold among the image collections of the same type as the image collection A1 that have been recommended within the most recent preset time period; if so, determining that the image collection A1 has not passed the repetition rate detection; if not, determining that the image collection A1 has passed the repetition rate detection. Passing the repetition rate detection is also referred to as satisfying the preset repetition rate condition. The preset repetition rate condition of the image collection A1 is, that is, "there is no image collection with a repetition rate greater than the preset repetition rate threshold among the image collections of the same type as the image collection A1 that have been recommended within the most recent preset time period".

[0244] Optionally, the preset duration may be, for example, 3 months. The preset repetition rate threshold may be, for example, 70%.

[0245] The repetition rate between image collection A1 and any image collection A2 refers to the percentage of the number of identical images included in image collection A1 and image collection A2 to the total number of images in image collection A1. For example, image collection A1 includes 15 images, image collection A2 includes 18 images, and 9 images in image collection A1 and image collection A2 are identical, then the repetition rate between image collection A1 and image collection A2 is determined to be (9 / 15)*100%=60%.

[0246] Optionally, the number of identical images in two image collections may be determined by comparing identification information of the images (eg, hash IDs of the images).

[0247] In this step, taking the high-priority image collection as image collection B1 and the type identification of image collection B1 as holiday_01_03 as an example, a repetition rate detection is performed on the image collection, that is, determining whether there is an image collection with a repetition rate greater than 70% with image collection B1 among the image collections recommended in the last three months and whose type identification is also holiday_01_03. If so, the repetition rate detection fails; if not, the repetition rate detection passes.

[0248] S203, delete the high priority image collection; then execute step S210.

[0249] If the repetition rate test fails, it means that the image collection has a high repetition rate with the same type of image collections recommended in the past three months, that is, there are many identical images, and the user is unlikely to be interested in the image collection. Therefore, the image collection is not recommended, and it will not be recommended again in the future, so the image collection is deleted. In this way, it can not only prevent image collections that users are not interested in from being recommended to users, but also prevent the image collection from being included in the screening range when the next collection is recommended, reducing the amount of screening operations and alleviating the algorithm operation load.

[0250] S204: Determine the one high-priority image collection as the collection to be recommended, and then terminate the collection screening process.

[0251] S205, score the n high-priority image collections to obtain the recommendation score of each collection, and rank the n high-priority image collections according to the recommendation score. After step S205, let i=1 and execute step S206.

[0252] Optionally, the image collection can be scored according to a preset scoring rule. The specific process of scoring can be found in the following embodiment. After obtaining the recommendation score, the n high-priority image collections are ranked according to the recommendation score. In this embodiment, the recommendation score can characterize the degree to which the image collection meets the user's expectations, the user's possible interest, and the possibility of the image collection being recommended. Therefore, the higher the ranking result of the image collection, the more the image collection meets the user's expectations, the more interested the user may be in the image collection, and the greater the possibility of the image collection being recommended.

[0253] S206, performing a repetition rate detection on the TOPi high priority image collection in the ranking to determine whether it passes the repetition rate detection; if not, executing step S207; if passing, executing step S209.

[0254] S207, deleting the TOPi high priority image collection, and then ending the collection screening process.

[0255] S208, determine whether i is equal to n; if so, execute step S210; if not, set i=i+1 and return to execute step S206.

[0256] S209: Determine the TOPi high-priority image collection as a collection to be recommended, and end the process.

[0257] That is to say, if the TOP1 high-priority image collection passes the repetition rate detection, the TOP1 high-priority image collection is recommended as the collection to be recommended; if the TOP1 high-priority image collection fails to pass the repetition rate detection, the repetition rate detection is continued for the TOP2 high-priority image collection in the ranking; if the TOP2 high-priority image collection passes the repetition rate detection, the TOP2 high-priority image collection is recommended as the collection to be recommended; if the TOP2 high-priority image collection fails to pass the repetition rate detection, the repetition rate detection is continued for the TOP3 high-priority image collection... and so on, until a certain high-priority image collection is determined as the collection to be recommended, then the collection screening process is terminated; or, all n high-priority image collections are traversed, indicating that there is no collection that can be recommended among the n high-priority image collections, and step S210 is executed.

[0258] S210, determine whether the time difference between the current moment and the moment of the most recent recommendation of a non-high priority (also called second priority) image collection is greater than or equal to a preset time interval; if so, execute step S211; if not, end the collection screening process.

[0259] A non-high priority image collection refers to an image collection whose collection type is not a high priority type. Optionally, the preset time interval may be, for example, 7 days.

[0260] That is to say, for non-high priority image collections, the time interval between two adjacent recommendations must be greater than or equal to the preset time interval, which can prevent frequent recommendations and excessive disturbance to users, thereby further improving user experience.

[0261] S211. Determine the number of non-high priority image collections in the multi-channel recall results; if the number of non-high priority image collections is 1, execute step S212; if the number of non-high priority image collections is m (m is an integer), and m is greater than or equal to 2, execute step S215; if the number of non-high priority image collections is 0, end the collection screening process.

[0262] S212, performing a repetition rate detection on the one non-high priority image collection to determine whether it passes the repetition rate detection; if it fails the detection, executing step S213; if it passes the detection, executing step S214.

[0263] S213, deleting the one non-high priority image collection, and ending the collection screening process.

[0264] S214: Determine the one non-high priority image collection as the collection to be recommended, and end the collection screening process.

[0265] S215. Score the m non-high priority image collections to obtain a recommendation score for each non-high priority image collection, and rank the m non-high priority image collections according to the recommendation score; then, set j=1 and execute step S216.

[0266] S216, performing a repetition rate detection on the TOPj non-high priority image collection in the ranking to determine whether it passes the repetition rate detection; if not, executing step S217; if passing, executing step S219.

[0267] S217, delete the TOPj non-high priority image collection, and then execute step S218.

[0268] S218, determine whether j is equal to m; if so, end the screening process; if not, set j=j+1 and return to step S216.

[0269] S219: Determine the TOPj non-high priority image collection as the collection to be recommended, and end the process.

[0270] In this embodiment, when screening the collection to be recommended, the high priority image collection is preferentially screened. If there is no high priority image collection or there is no collection that meets the repetition rate requirement in the high priority image collection, the collection to be recommended is screened from the non-high priority image collection. Relatively speaking, the high priority image collection is more likely to increase user interest. For the high priority image collection generated on holidays, the earlier and more timely the push timing is, the higher the matching degree with the current moment and scene is, and the better the user experience is. Therefore, the priority push of the high priority image collection can increase user interest and thus improve user experience. In addition, in this embodiment, whether it is a high priority image collection or a non-high priority image collection, when there are multiple collections, each image collection is first scored to obtain the recommendation score of each collection, the collection is ranked according to the recommendation score, and the repetition rate is detected in turn according to the ranking, so as to screen out the collection to be recommended. In this way, a collection that better meets user expectations can be selected from multiple collections, which can increase user interest and thus improve user experience. Moreover, in this embodiment, when screening collections, a repetition rate test is required. Only collections that pass the repetition rate test can be determined as collections to be recommended. This can prevent the recommended collection from having too many repeated images with recently recommended collections, further increasing user interest and improving user experience.

[0271] It is understandable that in some cases, when the collection to be recommended cannot be determined according to the above process, the collection generation and recommendation module can generate a substitute collection according to the preset rules, and perform duplication detection on the substitute collection. If it passes the detection, the substitute collection is recommended, and if it fails the detection, the substitute collection is deleted. In this way, as a supplementary solution for collection recommendation, the effect of collection recommendation is further improved, and the user experience is improved.

[0272] The following is an explanation of the scoring and ranking process for the image collection.

[0273] In the embodiment of the present application, the recommendation score is determined based on the content score and the suppression rate score. The content score reflects the degree to which the image collection meets the user's expectations, the user's possible interest, and the possibility of the image collection being recommended from the aspects of the generation time, the subject of the characters, and the time span. The suppression rate score reflects the possibility of the image collection being recommended from the push frequency of the same type of image collection.

[0274] Taking the scoring and ranking of m non-high priority image collections as an example, the above step "S215, scoring the m non-high priority image collections to obtain the recommendation score of each collection, and ranking the m non-high priority image collections according to the recommendation score" includes:

[0275] 1) Calculate the content score of each non-high priority image collection separately.

[0276] In a specific embodiment, the content score of a collection can be determined by the following formula (1):

[0277] y content =w1*score class +w2*score body +w3*score time-range (1)

[0278] Among them, y content Indicates the content score of the collection, w1 indicates the weight corresponding to the generation timing, w2 indicates the weight corresponding to the character subject, and w3 indicates the weight corresponding to the time span. score class Indicates the score corresponding to the type of generation timing of this collection, score body Indicates the score corresponding to the type of the subject in the collection, score time-range Indicates the score corresponding to the type of the time span of the collection.

[0279] The values ​​of each weight in formula (1) can be preset values. It has been found that the generation time has a greater impact on the attractiveness of the collection than the time span, and the time span has a greater impact on the attractiveness of the collection than the main body of the characters. Therefore, optionally, when setting the weights, it can be followed that the weight corresponding to the generation time is greater than the weight corresponding to the time span, and the weight corresponding to the time span is greater than the weight corresponding to the main body of the characters, that is, w1>w3>w2. For example, set w1 to 3, w2 to 1, and w3 to 2.

[0280] Each score in formula (1) can be obtained by querying the corresponding relationship between the pre-set type scores. Specifically, it has been found that image collections generated for holidays are more likely to touch users psychologically and attract users than image collections generated regularly; and image collections generated regularly are more likely to attract users than image collections generated incrementally. Therefore, when setting the corresponding relationship between the type of generation time and the score, it can be followed that: the score of "holidays" is greater than the score of "regular generation", and the score of "regular generation" is greater than the score of "incremental", that is: score class-holiday >score class-fixed >score class-incremental For example, score class-holiday 5 points, score class-fixed 4 points, score class-incremental It is 3 points.

[0281] According to research, the degree of psychological impact of image collections of different types of characters on users is from strong to weak: the social circle of the central character, the central character and close people, the close people of the central character, and the central character. Therefore, when setting the corresponding relationship between the type of character and the score, the corresponding scores decrease in the above order. In other words: score body-04 >score body-03 >score body-02 >score body-01 Among them, score body-04 Indicates the score corresponding to the social circle of the central character; score body-03 Indicates the score corresponding to when the character subject is the central character and the close person; score body-02 Indicates the score corresponding to when the character subject is a close person of the central character; score body-01 Indicates the score corresponding to when the character subject is the central character.

[0282] According to research, the psychological impact of image collections of different types of time spans on users is from strong to weak: many years (greater than 1 year), 1 year, 1 quarter, 1 month. Therefore, when setting the correspondence between the type of time span and the score, the corresponding scores decrease in the above order. In other words: score time-range-x >score time-range-03 >score time-range-02 >score time-range-01 , where score time-range-03 Indicates the score corresponding to the time span of many years; score time-range-03 Indicates the score corresponding to a time span of 1 year; score time-range-02 Indicates the score corresponding to the quarterly time span; score time-range-01 Indicates the score corresponding to the time span of 1 month.

[0283] In this embodiment, for each non-high priority image collection, the value of each weight in formula (1) is obtained, and according to the type identifier of the non-high priority image collection, the above correspondence is searched to determine the value of each score in formula (1), and then based on formula (1), the content score of the image collection is obtained.

[0284] In this embodiment, weights are set based on the research results, and a correspondence between image collection types and scores is set so that the calculated content score can more effectively reflect the extent to which the collection meets user expectations and the degree of interest in the collection to the user, thereby making the image collection ultimately recommended to the user more attractive and improving the user experience.

[0285] 2) Rank the m non-high priority image collections according to their content scores, and determine the content ranking score of each non-high priority image collection based on the ranking results.

[0286] Optionally, the m non-high priority image collections can be ranked in descending order according to the content scores, and each non-high priority image collection corresponds to a ranking value, and the ranking value is used as the content ranking score of the non-high priority image collection. For example, if the content score of a non-high priority image collection is ranked 4th, the content ranking score of the image collection is 4. Of course, it is also possible to assign content ranking scores to each image collection according to certain assignment rules based on the ranking results, and the embodiments of the present application do not impose any restrictions on this.

[0287] 3) Calculate the frequency suppression scores of each non-high priority image collection respectively.

[0288] In a specific embodiment, the frequency suppression score of the collection can be determined by the following formula (2-1):

[0289] y frequency =f(n class ) (2-1)

[0290] Among them, y frequency represents the frequency suppression fraction of the collection, n class Indicates the number of times the same type of image collection is recommended. The same type of collection refers to the same type of collection as the collection being calculated. The frequency suppression score of the collection is negatively correlated with the number of times the same type of image collection is recommended.

[0291] Take any image collection m1 (collection m1 for short) among the m non-high priority image collections as an example. When calculating the frequency suppression score of collection m1, n class It is the number of times that image collections of the same type as collection m1 are recommended (also called the number of similar recommendations), that is, the number of image collections of the same type as collection m1 among the recommended image collections.

[0292] Optionally, the function f(n class ) can be a family of convex functions that are continuously decreasing and locally Lipschitz continuous, for example: f(n class )=1 / log(n class +1). In other words, the frequency suppression score can be calculated using formula (2-2):

[0293] y frequency =1 / log(n class +1) (2-2)

[0294] Optionally, the curve graph corresponding to formula (2-2) can be as follows Figure 8shown.

[0295] Depend on Figure 7 It can be seen that when the frequency suppression score is calculated using a continuously decreasing local Lipschitz continuous convex function family, the frequency suppression score corresponding to a smaller number of recommendations is relatively large, and the frequency suppression score corresponding to a larger number of recommendations is small. In this way, it is reflected in the final collection recommendation as follows: the image collection types with a large number of recommendations are suppressed to a certain extent. In other words, the image collection types with low frequency recommendations are recommended as much as possible, and the image collection types with high frequency recommendations are suppressed to a certain extent. In this way, it can prevent the same type of image collection from being disturbed by too high a recommendation frequency, thereby improving the user experience; and the image collections with a lower recommendation frequency are more likely to attract users, further improving the user experience.

[0296] Moreover, by Figure 7 It can be seen from the trend of the middle curve that the frequency suppression score is not linearly related to the number of recommendations. The difference in the degree of suppression of the function for two larger recommendation times is smaller than the difference in the degree of suppression of the function for two smaller recommendation times. For example, the difference in the suppression of the 1000th and 1001st recommendations by the function is significantly smaller than the difference in the suppression of the 10th and 11th recommendations. In other words, the function tends to suppress image collections with a large number of recommendations more slowly. Because, to a certain extent, the number of recommendations can reflect the actual usage scenarios of users. When the number of recommendations is large, it means that this type of image collection meets the actual usage scenarios of users. Therefore, in this embodiment, a continuously decreasing local Lipschitz continuous convex function family is used to calculate the frequency suppression score, and the degree of suppression of image collections with a large number of recommendations tends to be slow, which can prevent excessive suppression of collection types with high frequency recommendations, so that image collections that are more in line with the actual usage scenarios of users can be effectively recommended.

[0297] 4) Rank the m non-high priority image collections according to the frequency suppression scores, and determine the frequency suppression ranking score of each non-high priority image collection according to the ranking results.

[0298] Optionally, the m non-high priority image collections can be ranked in descending order according to the frequency suppression scores, and each non-high priority image collection corresponds to a ranking value, and the ranking value is used as the frequency suppression ranking score of the non-high priority image collection. For example, if the frequency suppression score of a non-high priority image collection is ranked 7th, then the frequency suppression ranking score of the image collection is 7. Of course, it is also possible to assign frequency suppression ranking scores to each image collection according to certain assignment rules based on the ranking results of the frequency suppression scores, and the embodiments of the present application do not impose any restrictions on this.

[0299] 5) Determine the recommendation score of each non-high priority image collection according to the score ranking score and frequency suppression ranking score of each non-high priority image collection.

[0300] In a specific embodiment, the recommendation score of the image collection can be calculated according to the following formula (3):

[0301] socre final =rank(y content )+rank(y frequency ) (3)

[0302] Among them, socre final Represents the recommendation score of the image collection, rank(y content ) represents the content ranking score of the image collection, rank(y frequency ) represents the frequency suppression ranking score of the image collection.

[0303] In this embodiment, the image collection is scored according to the type of the image collection to obtain a content score; the frequency suppression score of the image collection is calculated according to the number of times the same type of collection is recommended, and then the recommendation score of each image collection is determined based on the ranking of the content scores and the ranking of the frequency suppression scores of multiple image collections recalled by multiple channels. The recommendation score can quantitatively and appropriately reflect the relative attractiveness of each image collection, the relative preference and interest of the user in each image collection. In the later stage, based on the recommendation score, an image collection that better meets the user's expectations and is more attractive to the user can be recommended to the user, thereby improving the user experience.

[0304] The following describes the generation process of an image collection in which the type of the subject is "central person and close people" and the type of the generation time is "incremental type". In the following embodiments, this type of image collection is called an old friend reunion collection.

[0305] For example, Fig. 9 The following is a flowchart of another method for recommending image collections provided in an embodiment of the present application. Figure 8 As shown, the above steps S115 to S117 may include the following steps:

[0306] S301, the collection generation and recommendation module determines, based on the configuration parameters and the social circle analysis results, that the image of person f meets the generation conditions corresponding to the old friend regrouping collection.

[0307] The generation conditions corresponding to the old friend reunion set can include the following three sub-conditions:

[0308] ① The newly added images include images of person f whose intimacy with the central person is greater than the preset intimacy x;

[0309] ② The sum of the number of historical images of person f and the number of newly added images of person f is greater than or equal to the preset threshold y;

[0310] ③ The time difference between the earliest shooting time in the newly added image of person f and the latest shooting time in the historical images of person f is greater than the preset time length z.

[0311] For sub-condition ①, the image of person f is the image that contains person f. The intimacy between each person included in the newly added image and the central person can be obtained from the “person relationship information” in the social circle analysis results (see Table 1).

[0312] For sub-condition ②, that is, the total number of images of person f in the current database 2 is greater than or equal to the preset threshold y. The images of person f in database 2 can be obtained from the "person co-occurrence information" in the social circle analysis results (see Table 1).

[0313] For sub-condition ③, for example, there are n newly added images of person f, and the n images correspond to n shooting moments, of which the shooting moment of one image is t1, and t1 is the earliest moment among the n shooting moments; there are m historical images of person f, and the m images correspond to m shooting moments respectively, of which the shooting moment of one image is t2, and t2 is the latest moment among the m shooting moments. If t1-t2 is greater than the preset duration z, it is determined that sub-condition ③ is satisfied. Satisfying sub-condition ③ indicates that the current meeting between the central person and person f exceeds the preset duration z since the last meeting. The preset duration z can be set according to actual needs, for example, it can be 90 days.

[0314] If the person f and the image of the person f meet the above three sub-conditions, the person f is determined to be an "old friend" of the central person, and an old friend reunion cluster can be generated based on the image of the person f.

[0315] It can be understood that by executing step S301, the number of characters that meet the above conditions may be 1 or more. In the embodiment of the present application, any character f that meets the above conditions is taken as an example for explanation.

[0316] S302 , the collection generation and recommendation module obtains images and image information of person f from a collection recommendation toolkit, and obtains a candidate image collection 1 of person f (hereinafter referred to as candidate collection 1 ).

[0317] S303. The collection generation and recommendation module sends an image scoring request and candidate collection 1 to the image scoring module.

[0318] The image scoring request is used to request a comprehensive score for each image in the candidate collection 1.

[0319] S304 , the image scoring module performs comprehensive scoring on each image in the candidate collection 1 in response to the image scoring request, and obtains a comprehensive score of each image in the candidate collection 1 .

[0320] The comprehensive score of an image can reflect the quality of the image and the user's preference for the image. Optionally, the comprehensive score can be a relative score, which is used to reflect the quality of an image compared to other images within a certain range and the user's preference for the image. For example, the comprehensive score of an image in candidate collection 1 is used to reflect the quality of the image compared to other images in candidate collection 1 and the user's preference for the image.

[0321] Optionally, the comprehensive score of the image may be obtained by summing the interaction score, semantic score, and aesthetic score of the image. The specific implementation process will be further described in subsequent embodiments.

[0322] S305. The image scoring module returns the comprehensive score of each image in the candidate collection 1 to the collection generation and recommendation module.

[0323] S306. The collection generation and recommendation module sends a deduplication request, candidate collection 1, and the comprehensive score of each image in candidate collection 1 to the deduplication module.

[0324] The deduplication request is used to request deduplication of images in image collection 1.

[0325] S307 . The deduplication module deduplicates candidate collection 1 in response to the deduplication request to obtain candidate collection 2 .

[0326] Specifically, the deduplication module can calculate the similarity between two images in candidate set 1. For two images whose similarity is greater than a preset similarity threshold, the image with a lower comprehensive score is removed according to the comprehensive scores of the two images, and the image with a higher comprehensive score is retained. The final image collection is called candidate set 2.

[0327] When users take pictures, they often take a large number of pictures with high similarity in a certain place according to different expressions or slight angles. Placing such images in the same image collection will reduce the user experience. To this end, in this embodiment, by deduplicating images with high similarity according to the comprehensive score, images with high comprehensive scores are retained. On the one hand, this prevents image duplication in the image collection, improves image diversity, and thus improves user experience. On the other hand, deduplication is performed based on the comprehensive score, so that the images retained in the image collection are images with high comprehensive scores, thereby improving the quality of the image collection and further improving the user experience.

[0328] As a possible implementation, when deduplication is performed on candidate collection 1, the preset image blacklist, user shielding information, etc. can be further combined to remove images in candidate collection 1 that belong to the preset image blacklist or images corresponding to the user shielding information, thereby generating candidate collection 2. In this way, images that users do not want to see can be shielded to prevent a bad experience for users.

[0329] S308. The deduplication module returns the candidate collection 2 to the collection generation and recommendation module.

[0330] S309, the collection generation and recommendation module samples the images in candidate collection 2 according to a preset uniform distribution sampling mechanism based on the comprehensive score and aesthetic score of each image in candidate collection 2 to obtain an old friend reunion collection.

[0331] The sampling process of the preset uniform distribution sampling mechanism is: bucketing (also called grouping) the images in the candidate collection according to the time dimension and the location dimension, and then sampling a certain number of images from each bucket based on the number of images in each bucket, based on the comprehensive score and aesthetic score of the image, to obtain the old friend reunion collection. Among them, when sampling images from each bucket according to the number of images in each bucket, the principle of sampling images in all buckets as much as possible is followed, so that the obtained old friend reunion collection is as close to uniform distribution as possible in the time dimension and the location dimension. The specific implementation process of sampling according to the uniform distribution sampling mechanism can be found in the subsequent embodiments.

[0332] Since the images taken by users are random, the quality of images in a certain shooting location or a certain shooting time period in database 2 may be overall high, or the user's preference may be overall high, while the quality of images in another shooting location or another shooting time period may be overall low, or the user's preference may be overall low. If sampling is directly based on the comprehensive score or aesthetic score, images in certain shooting time periods or certain shooting locations cannot be sampled, resulting in the lack of images of certain shooting time periods or certain shooting locations in the final generated image collection, making it difficult to effectively cover the user's "memories", affecting the user experience. The method provided in the embodiment of the present application is based on the comprehensive score of the image and samples according to a preset uniform distribution sampling mechanism. It not only takes into account factors such as image quality and user preference, but also takes into account the shooting time and shooting location of the image, so that the time distribution and location distribution of the images in the obtained image collection are as close to the uniform distribution as possible, and the user's "memories" are covered in the time dimension and location dimension to the greatest extent, thereby improving the user experience.

[0333] For example, Fig.10 This is a comparison diagram of the time distribution of an example candidate set 2 provided in the embodiment of the present application and the images in the final generated old friends reunion. Fig.10It can be seen that the time distribution of the images in the final reunion of old friends can cover four years, which is close to a uniform distribution.

[0334] For example, Fig.11 This is a comparison diagram of the location distribution of an example candidate set 2 provided in the embodiment of the present application and the images in the final generated old friends reunion. Fig.11 It can be seen that the location distribution of the images in the final reunion of old friends can cover four locations, approaching a uniform distribution.

[0335] In this embodiment, by mining the association between the newly added images and the images in database 2, the social relationships of the characters, etc., an old friend reunion collection is generated, and image collections with strong storylines, high excitement, and the ability to attract users are recommended to users, thereby increasing the richness of the recommended image collections and improving the user experience.

[0336] The above steps S301 to S309 illustrate the generation process of the image collection by taking the old friends reunion collection as an example. It can be understood that the generation process of other types of image collections is similar to the old friends reunion collection. The main difference is that the corresponding generation conditions are different, and the images selected when generating the image collection are different. For example, when generating a collection of related images of the central character, the collection generation and recommendation module obtains the image of the central character from the collection recommendation toolkit. The remaining steps may be the same or similar to the above steps and will not be repeated. In addition, as a possible implementation method, for image collections with a smaller time span, such as 1 month or 1 quarter, sampling may not be performed according to step S309, and sampling may be performed directly based on the aesthetic score and / or the comprehensive score.

[0337] The following is an explanation of the sampling process according to the preset uniform distribution sampling mechanism. Fig.12 A schematic diagram of an image sampling process provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the above step "S309, the collection generation and recommendation module samples the images in the candidate collection 2 according to the preset uniform distribution sampling mechanism based on the comprehensive score and aesthetic score of each image in the candidate collection 2 to obtain the old friend reunion collection" includes the following steps. The execution body of the following steps is the collection generation and recommendation module, which will not be repeated.

[0338] S401 . Bucket the images in candidate collection 2 according to the shooting location to obtain at least one location bucket (also called location grouping).

[0339] Optionally, when bucketing according to shooting locations, the city can be used as the bucket granularity. For example, the images in candidate collection 2 can be divided into Xi'an bucket, Shanghai bucket, Hangzhou bucket, etc. Each location bucket includes at least one image in candidate collection 2.

[0340] S402: Calculate the frequency attenuation weight of the location corresponding to each location bucket according to the number of images in each location bucket.

[0341] Optionally, the frequency attenuation weight of each location can be calculated according to formula (4):

[0342]

[0343] Among them, w frequency Represents the frequency attenuation weight of a certain location. Num(L) represents the number of images in the location bucket corresponding to the location. That is, the frequency attenuation weight of a certain location is negatively correlated with the number of images in the location bucket corresponding to the location.

[0344] For example, the location corresponding to the Xi'an bucket is Xi'an, and the frequency attenuation weight of Xi'an is: Among them, Num(Xi'an) represents the number of images in the Xi'an bucket.

[0345] Formula (4) is similar to the above formula (2-2). Therefore, the frequency attenuation weight of the location calculated according to formula (4) can suppress the appearance frequency of images of high-frequency locations. High-frequency locations refer to shooting locations with a large number of images.

[0346] S403: bucket the images in each location bucket again according to the shooting time to obtain at least one comprehensive bucket (also called comprehensive grouping).

[0347] Optionally, when the images in the location buckets are re-bucketed according to the shooting time, the selected time granularity can be set according to actual needs. For example, the images in the location buckets can be re-bucketed with the granularity of month, or with the granularity of quarter or year. The embodiments of the present application do not impose any limitations on this.

[0348] Based on location bucketing, images are bucketed again according to the shooting time, resulting in a smaller bucket granularity, and involving both the time dimension and the location dimension. In this way, subsequent sampling can cover images in both the time dimension and the location dimension, making the image collection more comprehensive and improving the user experience.

[0349] For example, the comprehensive buckets obtained by further bucketing the location buckets according to the shooting time may include: 2020 Xi'an bucket, 2021 Xi'an bucket, 2022 Xi'an bucket, 2020 Shanghai bucket, 2022 Hangzhou bucket... Each comprehensive bucket includes at least one image in candidate collection 2.

[0350] S404. According to a preset aesthetic score threshold, the images in each comprehensive bucket are divided into a preliminary screening group A and a candidate group B, wherein the aesthetic scores of the images in the preliminary screening group A are all greater than the preset score threshold, and the aesthetic scores of the images in the candidate group B are all less than or equal to the preset aesthetic score threshold.

[0351] In the above step S116, the collection generation and recommendation module obtains the images and image information required for generating the image collection from the collection recommendation toolkit, and the image information may include aesthetic scores. Therefore, in this step, the aesthetic score of each image in the candidate collection 2 is known.

[0352] Specifically, for any image in the comprehensive bucket, determine whether the aesthetic score of the image is greater than the preset aesthetic score threshold; if so, place the image in the preliminary screening group A, if not, place the image in the candidate group B. That is, in this step, the images in the comprehensive bucket are divided into high aesthetic score groups and low aesthetic score groups. The preliminary screening group A is a high aesthetic score group, in which the images have high quality and high beauty. The candidate group B is a low aesthetic score group, in which the images have low quality and low beauty.

[0353] In specific implementation, taking the total number of comprehensive buckets as g as an example, we can set t=1 and execute grouping step a: for comprehensive bucket t, according to the aesthetic score threshold, divide the images of comprehensive bucket t into initial group At and candidate group Bt; then, determine whether t is equal to g. If so, complete the grouping of all comprehensive buckets; if not, set t=t+1 and return to execute grouping step a.

[0354] S405 , determining the expected number of samples for each comprehensive bucket according to the number of images in each comprehensive bucket and the frequency attenuation weight of the location corresponding to the comprehensive bucket.

[0355] The expected number of samples for a comprehensive bucket refers to the number of images that are expected to be sampled from the comprehensive bucket if the uniform distribution requirement is met. Meeting the uniform distribution requirement means that the sampled collection is close to uniform distribution in terms of time and location. In other words, if each comprehensive bucket can be sampled according to the expected number of samples, the image collection can be close to uniform distribution in terms of time and location.

[0356] Optionally, for any comprehensive bucket t, it can be determined whether the number of images in the comprehensive bucket t is less than a preset number threshold; if so, the number of images in the comprehensive bucket t is used as the expected sampling number; if not, the expected sampling number of the comprehensive bucket is calculated according to the following formula: expected sampling number = number of images in the comprehensive bucket * frequency attenuation weight of the corresponding location * preset ratio 1. The preset ratio 1 can be set as needed, for example, it can be 30%.

[0357] That is to say, for comprehensive buckets with a small number of images (less than the preset number threshold), the comprehensive bucket is fully sampled to cover all time dimensions and location dimensions as much as possible; for comprehensive buckets with a large number of images (greater than or equal to the preset number threshold), the comprehensive bucket is partially sampled and suppressed according to the frequency attenuation weight of the location to prevent too many images of the same location from being included in the image collection, affecting the user experience.

[0358] S406. For the preliminary screening group At obtained by the comprehensive bucket t grouping, determine whether the number of images contained in the preliminary screening group At is greater than 0; if so, do not sample the comprehensive bucket t and execute step S412; if not, execute step S407.

[0359] The comprehensive bucket t can be any comprehensive bucket.

[0360] It is understandable that if the images taken by the user in a certain time period or at a certain location may have poor overall quality, then in the above grouping step, no images are allocated to the initial screening group At (high aesthetic score group) obtained by a certain comprehensive bucket grouping. In this case, the comprehensive bucket is not sampled to prevent too many low-quality images from being added to the image collection and affecting the user experience.

[0361] S407, determine whether the image in the comprehensive bucket t is greater than or equal to the expected sampling number Ct of the comprehensive bucket t; if so, execute step S408; if not, execute step S409.

[0362] S408, sampling an expected number of images Ct from the initial screening group At according to the comprehensive score. Then, executing step S413.

[0363] That is to say, the images in the initial screening group At (high aesthetic score group) obtained by the comprehensive bucket t grouping are sufficient for sampling according to the expected sampling number Ct. Therefore, according to the comprehensive score calculated in step S304 in the above embodiment, Ct images with higher comprehensive scores are selected to be added to the old friend regrouping set.

[0364] S409. Calculate the number of supplementary images Rt based on the number of images in the candidate group Bt obtained by the comprehensive bucket t grouping, the frequency attenuation weight of the location corresponding to the comprehensive bucket t, and the preset ratio 2, select the number of supplementary images Rt from the candidate group Bt and add them to the initial screening group At to obtain the supplementary group Dt.

[0365] Optionally, taking the location Lt corresponding to the comprehensive bucket t as an example, for the comprehensive bucket t, the number of supplements Rt=the number of images in the candidate group Bt*the frequency attenuation weight of the location Lt*the preset ratio 2. The preset ratio 2 may be, for example, 20%.

[0366] That is to say, in this embodiment, when adding images to the initial screening group At (high aesthetic score group), the images of the candidate group Bt are not added as a whole, or added according to the number of missing images, but the number of additions Rt is calculated in combination with the number of images in the candidate group Bt, the frequency attenuation weight of the location and the preset ratio, so that both image quality and distribution uniformity are taken into account when adding images, thereby preventing too many low-quality images from flowing into the final image collection, ensuring the distribution uniformity of the final image collection, and improving the user experience.

[0367] S410, determine whether the number of images in the supplementary group Dt is greater than or equal to the expected sampling number Ct; if so, execute step S411; if not, do not sample the comprehensive bucket t, and execute step S412.

[0368] S411, sampling an expected number Ct of images from the supplementary group Dt according to the comprehensive score, and then executing step S412.

[0369] S412, determine whether to traverse all comprehensive buckets; if so, execute step S414; if not, take the next comprehensive bucket as comprehensive bucket t, and return to execute step S406.

[0370] S413, reunite with old friends.

[0371] That is to say, in this embodiment, when sampling each comprehensive bucket, priority is given to sampling from the initial screening group At (high aesthetic score group) obtained by comprehensive bucket grouping. If the sampling is insufficient, the images in the candidate group Bt (low aesthetic score group) are added to the high aesthetic score group for sampling. In this way, while ensuring the quality of the images in the sampled image collection as much as possible, the number of samples of the images in the bucket is guaranteed, so that the location distribution and time distribution of the images in the obtained image collection are as close to uniform distribution as possible, covering the user's "memory" and improving the user experience.

[0372] In addition, after the images of the candidate group Bt (low aesthetic score group) are added to the initial screening group At (high aesthetic score group), it is further determined whether the expected sampling number is reached. In essence, the image quality requirements are slightly lowered to try to meet the uniform distribution requirements. If the number of images in the supplementary group Dt is greater than or equal to the expected sampling number Ct, it means that the uniform distribution requirements can be met after slightly lowering the image quality requirements. Therefore, sampling is directly performed from the supplementary group Dt. Although the image quality is slightly sacrificed, the user experience is improved in terms of the uniformity of distribution in the time dimension and the location dimension. If the number of images in the supplementary group Dt is still less than the expected sampling number Ct, it means that the uniform distribution requirements cannot be met after lowering the image quality requirements. Therefore, the sampling of the comprehensive bucket is directly abandoned to ensure the quality of the sampled image and improve the user experience from the quality aspect. The sampling method provided in this embodiment can flexibly take into account both image quality and distribution uniformity, effectively improve the attractiveness of the sampled image collection to users, and improve the user experience.

[0373] The following is an explanation of the image scoring process. Fig.13 A schematic diagram of an image scoring process provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the above step "S304, the image scoring module responds to the image scoring request, and performs comprehensive scoring on each image in the candidate collection 1 to obtain a comprehensive score of each image in the candidate collection 1" may include the following steps. The execution subject of the following steps is the image scoring module, which will not be repeated.

[0374] S501. Rank the images in candidate collection 1 according to the interaction amounts corresponding to the various interaction behaviors of the users on the images, and determine the interaction score of each image according to the ranking results.

[0375] The user's interactive behavior on the image is also the user's operation on the image. The user's interactive behavior on the image may include, for example, clicking, browsing, editing, sharing, etc. The amount of interaction refers to the number of interactive behaviors. The amount of interaction can be characterized by one or more of the number of interactive behaviors, the duration of interactive behaviors, etc.

[0376] In this embodiment, the interaction score of the image is used to reflect the relative degree of interaction between the user and the image in the candidate collection 1. The degree of interaction reflects the relative degree of liking of the image by the user. In other words, the interaction score is a relative quantity. Optionally, the higher the interaction score of the image, the more interaction the user has with the image, and the higher the degree of liking of the image by the user.

[0377] Table 3 is a statistical table of the interaction amount corresponding to an example of user interaction behavior on an image provided by an embodiment of the present application. In Table 3, the interaction amount corresponding to click is represented by the number of clicks, the interaction amount corresponding to browse is represented by the browsing time, the interaction amount corresponding to edit is represented by the number of edits, and the interaction amount corresponding to share is represented by the number of shares.

[0378] Table 3

[0379] Interaction volume Number of clicks Edit times … Shares Viewing time Image 1 12 23 … 3 108 Image 2 1 0 … 0 34 … … … … … … Image 2 1 … 1 47

[0380] Through Table 3, the interaction amounts corresponding to various interaction behaviors of each image in the candidate set 1 can be determined.

[0381] Optionally, the interaction amount corresponding to various interaction behaviors of each image in candidate collection 1 can be normalized, or weighted summed, etc., to obtain an initial interaction score for each image. According to the initial interaction score, the images in candidate collection 1 are ranked, and the interaction score of each image is determined according to the ranking result. In one possible implementation, the ranking value of each image can be used as the interaction score of the image. In another possible implementation, the interaction score of each image can be obtained by assigning a value according to a preset rule based on the ranking value of each image.

[0382] Optionally, the interaction amounts corresponding to various interaction behaviors of each image in the candidate collection 1 may be directly ranked to obtain a ranking matrix, a ranking matrix. For any image in the candidate collection 1, the rankings corresponding to multiple interaction behaviors are added together to obtain the interaction score of the image.

[0383] The embodiment of the present application does not impose any limitation on the specific method for calculating the interaction score of an image, and can be designed according to actual needs.

[0384] S502. Rank the semantic categories corresponding to the images in candidate collection 1 according to the interaction amounts corresponding to the various interaction behaviors of the user images, and determine the semantic score of each image according to the ranking result and the semantic category corresponding to each image in candidate collection 1.

[0385] Optionally, the image may contain semantics, and the semantics may have different categories. Specifically, the semantic categories may include major categories such as object, scene, time, etc., and each major category may include multiple subcategories. For example, the object major category may include subcategories such as person, animal, and landscape. Therefore, the semantic category corresponding to each image may be determined according to the semantics it contains.

[0386] In this embodiment, semantic recognition is performed on each image in the candidate collection 1, and the semantics contained in each image can be obtained, and the category to which the semantics belongs can be determined, and the semantic category corresponding to each image can be obtained. Afterwards, for any semantic category, it can be determined which images in the candidate collection 1 correspond to the semantic category, and then the interaction amount corresponding to the user's interaction behavior with these images can be counted, and the interaction amount corresponding to the user's various interaction behaviors with the images of the semantic category can be obtained. The interaction amount corresponding to the user's interaction behavior with the image corresponding to a certain semantic category can reflect the user's liking for the image of the semantic category. Generally, for images of a certain semantic category, the greater the interaction amount, the higher the user's liking for the image of the semantic category.

[0387] Exemplarily, Table 4 is a statistical table of the interaction amounts corresponding to the user's interaction behaviors on images of various semantic categories.

[0388] Table 4

[0389] Interaction volume Number of clicks Edit times … Shares Viewing time Images of semantic category 1 33 53 … 34 1234 Images of semantic category 2 222 32 … 23 334 … … … … … … Images of semantic category n 1 3 … 2 489

[0390] Based on Table 4, the total amount of interactions of images of various semantic categories is determined. For example, for images of any semantic category, the weighted sum of the interaction amounts corresponding to various interaction behaviors is taken to obtain the total amount of interactions of the images of the semantic category. All semantic categories are ranked according to the total amount of interactions of images of various semantic categories. Then, based on the ranking results of the semantic categories and the semantic categories corresponding to each image in candidate collection 1, the semantic scores of each image in candidate collection 1 are determined.

[0391] Of course, the semantic score of each image may also be determined by other methods, and the embodiments of the present application do not impose any limitation on this.

[0392] S503: Obtain the aesthetic score of each image in candidate collection 1.

[0393] In the above step S116, the collection generation and recommendation module obtains the images and image information required for generating the image collection from the collection recommendation toolkit, and the image information may include aesthetic scores. Therefore, in this step, the aesthetic score of each image in the candidate collection 1 is known.

[0394] S504 , determining the comprehensive score of each image in the candidate collection 1 according to the interaction score, semantic score and aesthetic score of each image.

[0395] For any image, the interaction score, semantic score and aesthetic score of the image can be weighted and accumulated, and the result can be used as the comprehensive score of the image.

[0396] The comprehensive score of the image is calculated through steps S501 to S504. The comprehensive score can quantitatively reflect the relative image quality, beauty, user preference, etc. of each image in the candidate collection 1, which is convenient for subsequent image screening and sampling, thereby improving the quality of the final image collection and its attractiveness to users.

[0397] The specific implementation process of analyzing the social circle, that is, the above-mentioned step S112, is described below.

[0398] The analysis of social circles mainly includes co-occurrence statistics, intimacy calculation, central person identification, social circle identification, etc. The following is a brief description of each process.

[0399] (1) Co-occurrence statistics

[0400] Through co-occurrence statistics, the various contents in the co-occurrence information of the characters in Table 1 above can be obtained.

[0401] In one embodiment, the co-occurrence statistics include the following process:

[0402] ①. Determine the co-occurring image set based on the person-image heterogeneous graph.

[0403] Specifically, according to the person-image heterogeneous graph, the number of person nodes that are connected to the image node can be determined. If a certain image node is connected to two or more person nodes, the image corresponding to the image node is a co-occurrence image. The determined multiple co-occurrence images are used to generate a co-occurrence image set. It can be understood that in other embodiments, the determined multiple co-occurrence images may not be expressed in a set, but in other forms (such as a list). The embodiments of the present application do not impose any restrictions on this.

[0404] According to the person-image heterogeneity map, the co-occurrence image of each two persons can also be determined. For example, according to the person-image heterogeneity map, the co-occurrence image of person a and person b is determined, that is, the image containing person a and person b is determined. The co-occurrence image of person a and person b is also referred to as the person ab co-occurrence image hereinafter. Similarly, the co-occurrence image of any other two persons can be determined according to the person-image heterogeneity map.

[0405] ②. Based on the co-occurrence image set, count the time distribution of co-occurrence images of two characters.

[0406] Take the co-occurrence image of person ab as an example for explanation. The time distribution of the co-occurrence image of person ab is hereinafter referred to as the co-occurrence time distribution of person ab. The co-occurrence time distribution of person ab is used to characterize the distribution of the shooting time of the co-occurrence image of person ab in multiple preset time intervals. The interval length and number of intervals of the preset time interval can be determined according to actual conditions.

[0407] As a possible implementation, a day (00:00 to 24:00) can be divided into multiple preset time intervals based on the timing information, wherein the duration of each preset time interval is less than 24 hours. In a specific embodiment, a day can be divided into 24 preset time intervals, and the duration of each preset time interval is 1 hour. For example, the 24 preset time intervals may include: [00:00, 01:00), [01:00, 02:00), [02:00, 03:00) ... [23:00, 24:00).

[0408] It should be noted that the preset time interval can also be divided in other ways. For example, a month (00:00 on the 1st to 24:00 on the 31st) can be divided into multiple preset time intervals based on day information and timing information. This application does not impose any restrictions on the division method of the preset time interval. The following is an example of dividing a day into multiple preset times based on timing information. The execution process of the method under other division methods is similar to this and will not be repeated.

[0409] In a specific embodiment, the co-occurrence time distribution of characters ab may be counted according to the following process:

[0410] a. Count the number of matching co-occurrence images in each preset time interval among the co-occurrence images of person a and b.

[0411] Among them, the matching co-occurrence image within a preset time interval refers to: in the co-occurrence image of the characters ab, the timing information belongs to the preset time interval (that is, the shooting time matches the preset time interval). For example, the matching image within the preset time interval [08:00, 09:00) refers to the co-occurrence image of the characters ab, the timing information belongs to [08:00, 09:00). For example, the shooting time of a certain character ab co-occurrence image x (referred to as image x) is 08:18:12 on April 12, 2023, and the timing information of image x is 08:18:12, which belongs to [08:00, 09:00), so image x is the matching co-occurrence image of the preset time interval [08:00, 09:00). According to this method, the matching co-occurrence images within each preset time interval are determined, and the number of matching co-occurrence images within each preset time interval is obtained by counting.

[0412] Optionally, during the statistical process, for multiple matching co-occurrence images with the same date information within the same preset time interval, the number can be counted only once. That is: the co-occurrence images of characters ab with the same date information and timing information belonging to the same preset time interval are counted as one image. For example, the shooting time of image x is 08:18:12 on April 12, 2023, and the shooting time of character ab co-occurrence image y (referred to as image y) is 08:25:10 on April 12, 2023. The date information of image x and image y is the same, both April 12, 2023. The timing information of image x and image y both belong to the preset time interval [08:00, 09:00), which are matching co-occurrence images within the preset time interval. Therefore, when counting, image x and image y are regarded as one image, and the number is only counted once. This is to take into account that the co-occurrence time distribution of characters ab will be used to determine the divergence (i.e., uniformity) of the time distribution of characters ab in the future, and the intimacy between characters a and b is reflected by the time distribution divergence. The more evenly the time distribution is, the closer the characters A and B are. In actual use, even if two very close characters are taking photos with a camera, they may take photos together in a preset time interval on a certain day, resulting in multiple images. However, adding concentrated photos taken in a preset time interval to the time distribution statistics will result in an uneven distribution of the co-occurrence time of the two characters, which may cause the intimacy of two people with a closer relationship to be lower, while the intimacy of two people with a less intimate relationship who did not take concentrated photos will increase. Therefore, the method provided by this implementation method counts the number of images taken in concentrated photos only once, and eliminates images produced by concentrated photos, so as to prevent affecting the accuracy of the co-occurrence time distribution statistics of characters A and B, thereby improving the accuracy of intimacy calculation.

[0413] In a specific embodiment, taking multiple preset time intervals including: [00:00, 01:00), [01:00, 02:00), [02:00, 03:00) ... [23:00, 24:00) as an example, the process of counting the number of matching co-occurring images in each preset time interval can be expressed by the following formula:

[0414] By vector Indicates the number of matching co-occurrence images in each preset time interval. ab [i] represents vector t ab The element i in represents the serial number of the preset time interval, i = 1, 2, 3 ... 24. When i = 1, the preset time interval 1 can be [00:00, 01:00), when i = 2, the preset time interval 2 can be [01:00, 02:00), when i = 3, the preset time interval 3 can be [02:00, 03:00) ... when i = 24, the preset time interval 24 can be [23:00, 24:00). t ab[i] represents the number of matching images within the preset time interval i.

[0415] First, the vector t ab Each element in is initialized to 0. Then, the number of matching co-occurrence images in each preset time interval is counted based on formula (5):

[0416]

[0417] Wherein, m represents the co-occurrence image m of person ab (abbreviated as image m). Function f(m) represents the preset time interval to which the timing information of image m belongs. ab [f(m)] represents the number of matching images within the preset time interval f(m). g(m) = 1 means that the co-occurrence images of person ab with the same date information as image m and the same preset time interval have been counted once. In this case, they are no longer counted. Therefore, t ab The value of [f(m)] remains unchanged, that is, t ab [f(m)]=t ab [f(m)] Correspondingly, g(m) = 0 means that the co-occurrence image with the same date information as image m and the same preset time interval has not been counted. In this case, the image needs to be included in the statistics. Therefore, t ab The value of [f(m)] is increased by 1, that is, t ab [f(m)]=t ab [f(m)]+1.

[0418] M represents the set of co-occurring images of characters ab, M1, M2, ..., Mx all represent co-occurrence images of characters ab, and x represents the total number of co-occurrence images of characters ab.

[0419] It can be understood that, when multiple preset time intervals include: [00:00, 01:00), [01:00, 02:00), [02:00, 03:00) ... [23:00, 24:00), when judging which preset time interval the timing information of a certain image belongs to, it can be judged only by the hour information of the image, without obtaining minute information and second information, etc. For example, if the hour information of image m is 10 hours, then image m belongs to the preset time interval [10:00, 11:00). Therefore, in this embodiment, the multiple preset time intervals are divided into 24 preset time intervals according to the above method, and the lower limit of each preset time interval is an integer number of hours, which can reduce the amount of calculation when calculating time distribution statistics and improve the efficiency of algorithm operation.

[0420] b. Determine the co-occurrence time distribution of characters ab according to the number of matching co-occurrence images in each preset time interval.

[0421] Based on the number of matching co-occurrence images in each preset time interval determined above, the proportion of matching co-occurrence images in each preset time interval in all co-occurrence images of characters ab is determined to obtain the co-occurrence time distribution of characters ab.

[0422] For the sake of ease of description, the proportion of the number of matching co-occurrence images within a preset time interval in all co-occurrence images of characters ab is referred to as the co-occurrence image proportion corresponding to the preset time interval.

[0423] Optionally, the process of determining the co-occurrence time distribution of characters ab can be expressed by formula (6):

[0424]

[0425]

[0426] Among them, the vector represents the co-occurrence time distribution of characters a and b, d ab，t [i] is any element in the vector, d ab,t [i] represents the proportion of co-occurring images corresponding to the preset time interval i.

[0427] ③. Based on the co-occurrence image set, count the location distribution of co-occurrence images of two characters.

[0428] Let's continue to use the co-occurrence image of person ab as an example for explanation. The location distribution of the co-occurrence image of person ab is hereinafter referred to as the co-occurrence location distribution of person ab. The co-occurrence location distribution of person ab is used to characterize the distribution of the shooting locations of the co-occurrence image of person ab in multiple known locations. Among them, the known locations can be obtained through the shooting locations of the images in the person-image heterogeneous map.

[0429] In a specific embodiment, the co-occurrence location distribution of characters ab can be counted according to the following process:

[0430] a. Determine a set of shooting locations, which includes the shooting locations of all images in the person-image heterogeneous graph.

[0431] Optionally, the set of shooting locations can be represented as N1, N2, ..., Ny are elements of the set N, representing the shooting locations. For example, the shooting location set y represents the total number of shooting locations.

[0432] Optionally, at any time before executing step a, a set of shooting locations may be generated by searching for shooting locations of all images in the person-image heterogeneous graph.

[0433] For the convenience of description, the elements in the shooting location set are referred to as location elements in the following. That is, the shooting location set includes multiple location elements, each of which is a shooting location of an image in the person-image heterogeneous graph.

[0434] b. Count the number of co-occurring images of characters ab corresponding to each location element.

[0435] For any location element Ni, the number of co-occurrence images of people ab corresponding to the location element refers to the number of co-occurrence images of people ab taken at location Ni. For example, the number of co-occurrence images of people ab corresponding to the location element "Beijing" refers to the number of co-occurrence images of people ab taken at location "Beijing".

[0436] c. Determine the co-occurrence location distribution of characters ab based on the number of co-occurrence images of characters ab corresponding to each location element.

[0437] Based on the number of co-occurrence images of the characters ab corresponding to each location element determined above, the proportion of the number of co-occurrence images of the characters ab corresponding to each location element in all co-occurrence images of the characters ab is determined to obtain the co-occurrence location distribution of the characters ab.

[0438] For the convenience of description, the proportion of the number of co-occurrence images of characters ab corresponding to a certain place element in all co-occurrence images of characters ab is referred to as the proportion of co-occurrence images corresponding to the place element.

[0439] Optionally, when determining the proportion of co-occurrence images corresponding to each location element, if the number of co-occurrence images of characters ab corresponding to the location element is 0, the proportion is determined to be 0; if the number of co-occurrence images of characters ab corresponding to the location element is not 0 (i.e., greater than 0), the proportion of co-occurrence images corresponding to the location element is determined to be 1 / the number of non-zero location elements. A non-zero location element refers to a location element whose number of corresponding co-occurrence images of characters ab is not 0. For example, the shooting location set includes 10 location elements. According to statistics, the number of co-occurrence images of characters ab corresponding to the five location elements N1, N3, N5, N7 and N10 in the set is not 0, so the proportion of co-occurrence images corresponding to these five location elements is 1 / 5=0.2, and the proportion of co-occurrence images corresponding to other location elements is 0. Then, the co-occurrence location distribution of characters ab can be expressed as:

[0440] As another optional implementation, in the process of counting the location distribution of co-occurrence images of characters ab, the above steps b and c can also be replaced by: counting whether co-occurrence images of characters ab corresponding to each location element exist; and determining the co-occurrence location distribution of characters ab based on the statistical results.

[0441] Optionally, the tag set includes a tag value corresponding to each place element, and the tag value is used to mark whether there is an image with the shooting location of the place element in the set of co-occurring images of the characters ab. Specifically, for any place element Ni, determine whether there is an image with the shooting location Ni in the set of co-occurring images of the characters ab; if so, determine the tag value corresponding to the place element Ni as 1; if not, determine the tag value corresponding to the place element Ni as 0, and then count the total number of place elements with a tag value of 1 to obtain the number of non-zero place elements. Then set the proportion of co-occurring images corresponding to the place elements with a tag value of 1 to 1 / the number of non-zero place elements.

[0442] Specifically, the above process can be expressed as follows through the formula:

[0443] By vector Represents the set of tag values ​​corresponding to each location element. ab [j] represents vector l ab The element in the , j is the number of the element in position, j = 1, 2, 3...y. ab [j] represents the tag value corresponding to location element j.

[0444] First, the vector l ab Each element in is initialized to 0. Then, based on formula (7), vector l ab Assign values ​​to each element:

[0445]

[0446] Where m represents the co-occurrence image m of person ab (abbreviated as image m). Function h(m) represents the shooting location of image m. ab [h(m)] represents the tag value corresponding to the shooting location h(m). q(m) = 0 means that the shooting location h(m) has not been counted, and the tag value of the location element with the same shooting location h(m) is currently 0. In this case, since the shooting location of image m is h(m), the tag value corresponding to the location element with the same shooting location h(m) is assigned to 1, that is, l ab [h(m)] = 1. Correspondingly, q(m) = 1 means that the shooting location h(m) has been counted once, that is, it has been determined that there is a co-occurrence image of person ab with the shooting location h(m), and the tag value of the location element with the same shooting location h(m) has been assigned 1. In this case, l ab The value of [h(m)] remains unchanged, that is, l ab [h(m)]=l ab [h(m)] M represents the set of co-occurring images of person ab, which will not be described in detail.

[0447] Then, the co-occurrence location distribution of characters ab is determined by formula (8):

[0448]

[0449]

[0450] Among them, the vector represents the co-occurrence location distribution of characters a and b, d ab, [j] is any element in the vector, d ab, [j] represents the proportion of co-occurrence images corresponding to location element j.

[0451] In this implementation, the co-occurrence location distribution of characters ab can be used to determine the divergence (i.e., uniformity) of the location distribution of characters ab, and the intimacy between characters a and b is reflected by the location distribution divergence. The more uniform the location distribution, the closer the characters a and b are. In actual use, even if two very close characters use a camera to take pictures, they may take pictures together at a certain location, resulting in multiple images. However, adding concentrated photos at a certain location to the location distribution statistics will result in an uneven distribution of the co-occurrence locations of the two characters, which may make the intimacy of two people with a closer relationship lower, while the intimacy of two people with a less intimate relationship who did not take concentrated photos will increase. Therefore, in this implementation, by counting whether there are co-occurrence images of characters ab corresponding to each location element, the location distribution of characters ab is analyzed, without considering the number of images taken at each location element, to prevent the images generated by concentrated photos from affecting the accuracy of the co-occurrence location distribution statistics of characters ab, thereby improving the accuracy of intimacy calculation.

[0452] (2) Intimacy calculation

[0453] By calculating the intimacy, the character relationship information in the above Table 1 can be obtained, that is, the intimacy between the characters can be obtained.

[0454] In a specific embodiment, taking any two characters, character a and character b, as an example, the intimacy calculation process of character a and character b is described.

[0455] ① Calculate the mutual information between person a and person b.

[0456] Optionally, the mutual information PMI(a,b) between person a and person b can be calculated by formula (9):

[0457]

[0458] Among them, pic a Indicates the number of co-occurrence images containing person a in the co-occurrence image set. b Indicates the number of co-occurrence images containing person b in the co-occurrence image set.a,b Indicates the number of co-occurrence images of characters a and b in the co-occurrence image set. total_pics indicates the total number of all co-occurrence images in the co-occurrence image set.

[0459] ② Determine the KL divergence of the co-occurrence time distribution and the uniform time distribution of characters ab, and obtain the temporal KL divergence of characters ab.

[0460] The statistical process of the co-occurrence time distribution of characters ab is as described in the above embodiment.

[0461] Uniform time distribution means that all co-occurrence images of characters ab are evenly distributed in multiple preset time intervals, that is, the proportion of co-occurrence images corresponding to each preset time interval is the same, which is 1 / the number of preset time intervals. For example, when the number of preset time intervals is 24, uniform time distribution means that the proportion of co-occurrence images corresponding to each preset time interval is 1 / 24.

[0462] The temporal KL divergence of character ab is used to characterize the difference between the co-occurrence time distribution of character ab and the uniform time distribution. In other words, the temporal KL divergence of character ab is used to characterize the uniformity of the distribution of the co-occurrence images of character ab in multiple preset time intervals, thereby reflecting the uniformity of the co-occurrence of character a and character b in time.

[0463] The time distribution can be expressed as a vector The elements in this set are all 1 / 24. Then the temporal KL divergence of character ab can be calculated by formula (10):

[0464]

[0465] Among them, D ab,t represents the temporal KL divergence of character ab, d u,t [i] represents vector d u,t Any element in d u,t [i]=1 / 24.

[0466] ③ Determine the KL divergence of the co-occurrence location distribution of character ab and the uniform location distribution, and obtain the location KL divergence of character ab.

[0467] The statistical process of the co-occurrence location distribution of characters ab is as described in the above embodiment.

[0468] Uniform location distribution means that all co-occurrence images of all characters ab are evenly distributed among all location elements in the shooting location set, that is, the proportion of co-occurrence images corresponding to each location element is the same, which is 1 / the total number of location elements. For example, when the total number of location elements is 10, uniform location distribution means that the proportion of co-occurrence images corresponding to each location element is 1 / 10=0.1.

[0469] The location KL divergence of person ab is used to characterize the difference between the co-occurrence location distribution of person ab and the uniform location distribution. In other words, the location KL divergence of person ab is used to characterize the uniformity of the distribution of the co-occurrence images of person ab in multiple preset time intervals, thereby reflecting the uniformity of the co-occurrence of person a and person b in location.

[0470] The locations are distributed and can be represented as vectors All elements in this set are 1 / y. Then the KL divergence of the location of character ab can be calculated by formula (11):

[0471]

[0472] Among them, D ab,l represents the KL divergence of the location of character ab, d u,l [j] represents vector d u,l Any element in d u,l [j] = 1 / y.

[0473] ④ Determine the intimacy between person a and person b based on the time KL divergence of person ab, the location KL divergence of person ab, and PMI(a,b).

[0474] In a specific embodiment, the intimacy between character a and character b can be determined according to formula (12-1):

[0475] closure(a,b)=PMI(a,b)×(1-D ab,t )×(1-D ab,l ) (12-1)

[0476] Among them, closure(a,b) represents the intimacy between character a and character b, D ab,t represents the time KL divergence of character ab, D ab,l Indicates the KL divergence of the location of person ab.

[0477] In another specific embodiment, the intimacy between character a and character b may also be determined according to formula (12-2):

[0478]

[0479] Among them, T and L are constant parameters, and the definitions of other parameters are the same as formula (12-1).

[0480] It can be seen from formula (12-1) and formula (12-2) that D ab,t The smaller it is, the larger the closure(a,b); D ab,l The smaller it is, the larger the closure(a,b). ab,tThe smaller it is, the more uniform the co-occurrence time distribution of characters a and b is; D ab,l The smaller it is, the more evenly the co-occurrence locations of characters ab are distributed. That is, according to formula (12-1) or formula (12-2), the more evenly the co-occurrence time distribution of characters ab is, the greater the intimacy between characters a and b, and the closer characters a and b are; moreover, the more evenly the co-occurrence time and location of characters ab are, the greater the intimacy between characters a and b, and the closer characters a and b are.

[0481] In this embodiment, when determining the intimacy between person a and person b, on the basis of PMI(a,b), the co-occurrence time KL divergence and co-occurrence location KL divergence of person ab are further added. The co-occurrence time KL divergence of person ab can characterize the uniformity of the temporal distribution of the co-occurrence images of person a and person b. The co-occurrence location KL divergence of person ab can characterize the uniformity of the location distribution of the co-occurrence images of person a and person b. The closer the social relationship between person a and person b, the more uniform the distribution of their co-occurrence images in time and location. Therefore, in this embodiment, the co-occurrence time KL divergence and co-occurrence location KL divergence of person ab are integrated into the calculation of intimacy, which improves the accuracy of intimacy calculation, thereby improving the accuracy of social circle recognition.

[0482] (3) Identification of central person

[0483] By identifying the central person, the central person information in the above Table 1 can be obtained, that is, the central person's ID, the central person's divergence and ranking, etc. can be obtained.

[0484] In one embodiment, the central person identification includes the following process:

[0485] ① According to the person-image heterogeneous graph, determine the image set of each person.

[0486] As described in the above embodiment, an image containing a certain person may be referred to as an image of the person.

[0487] Specifically, according to the person-image heterogeneous graph, all image nodes connected to a certain person node can be determined, and a set of images corresponding to these image nodes can be generated, that is, an image set of the person of the person node can be obtained.

[0488] ② According to the image information of each image in the image set of person a, determine the time KL divergence of the image of person a (hereinafter referred to as the time KL divergence of person a) and the location KL divergence of the image of person a (hereinafter referred to as the location KL divergence of person a).

[0489] Wherein, person a is any person in the person-image heterogeneous graph. That is to say, for each person in the person-image heterogeneous graph, its time KL divergence and location KL divergence are determined according to the method in this step, thereby obtaining the time KL divergence and location KL divergence of each person.

[0490] In a specific embodiment, the time KL divergence and location KL divergence of person a may be determined according to the following process:

[0491] a. According to the shooting time of each image in the image set of person a, the time distribution of the images of person a (hereinafter referred to as the time distribution of person a) is counted.

[0492] The time distribution of person a is used to characterize the distribution of the shooting moments of the image of person a in a plurality of preset time intervals. The preset time intervals are similar to those in the above embodiment and will not be described in detail.

[0493] Counting the time distribution of person a is similar to counting the co-occurrence time distribution of person ab in the above embodiment, except that: in this embodiment, the scope of statistics is the image set of person a, while in the above embodiment, the scope of statistics is all co-occurrence images of person a, and the object of statistics in this embodiment is the image of person a, while in the above embodiment, the object of statistics is the co-occurrence image of person ab. In this embodiment, in the image set of person a, the image whose shooting time matches a preset time interval can be called the matching image of the preset time interval. In the above embodiment, the image whose shooting time matches a preset time interval can be called the matching co-occurrence image of the preset time interval. The specific implementation process and beneficial effects of this embodiment refer to the above embodiment and will not be repeated.

[0494] b. Determine the KL divergence between the time distribution of character a and the uniform time distribution, and obtain the time KL divergence of character a.

[0495] The uniform time distribution is the same as in the above embodiment. The temporal KL divergence of character a is used to characterize the difference between the temporal distribution of character a and the uniform time distribution. In other words, the temporal KL divergence of character a is used to characterize the uniformity of the distribution of the image of character a in multiple preset time intervals, that is, to reflect the uniformity of the appearance of character a in time.

[0496] The calculation method of the temporal KL divergence of character a is similar to the calculation method of the temporal KL divergence of character ab, and will not be repeated here.

[0497] c. According to the shooting location of each image in the image set of person a, the location distribution of the images of person a (hereinafter referred to as the location distribution of person a) is counted.

[0498] The location distribution of person a is used to represent the distribution of shooting locations of images of person a.

[0499] In a specific embodiment, the co-occurrence location distribution of characters ab can be counted according to the following process:

[0500] The location distribution of person a is similar to the location distribution of person ab in the above embodiment, except that in this embodiment, the scope of statistics is the image set of person a, not the image set of person ab co-occurrence, and the object of statistics is the image of person a, not the image of person ab co-occurrence. The specific implementation process and beneficial effects of this process are shown in the above embodiment, and will not be repeated here.

[0501] d. Determine the KL divergence between the location distribution of person a and the uniform location distribution, and obtain the location KL divergence of person a.

[0502] The uniform location distribution is the same as in the above embodiment. The location KL divergence of person a is used to characterize the difference between the location distribution of person a and the uniform location distribution. In other words, the location KL divergence of person a is used to characterize the uniformity of the distribution of the image of person a in multiple preset time intervals, thereby reflecting the uniformity of the appearance of person a in the location.

[0503] The calculation method of the location KL divergence of character a is similar to the calculation method of the location KL divergence of character ab, and will not be repeated here.

[0504] ③ Determine the central character based on the time KL divergence and location KL divergence of all characters.

[0505] In one embodiment, the time KL divergence and the location KL divergence of each character can be summed to obtain the divergence sum of each character. Afterwards, the divergence and ranking of all characters (i.e., the first ranking) are taken, and the Q1 characters with the smallest divergence and are taken as the central characters, where Q1 is an integer greater than or equal to 1. If the ranking is in the order of the divergence and ranking from small to large, the top Q1 characters (TOP Q1) in the divergence and ranking are taken as the central characters. If the ranking is in the order of the divergence and ranking from large to small, the last Q1 characters (END Q1) in the divergence and ranking are taken as the central characters. It can be understood that when summing the KL divergence, it can be a direct summation or a weighted summation, and the embodiments of the present application do not impose any restrictions on this.

[0506] In another embodiment, the time KL divergence of all characters can be ranked in order from small to large, and the location KL divergence can be ranked in order from small to large. Then, the time KL divergence ranking and location KL divergence ranking of each character are summed to obtain the divergence ranking sum corresponding to each character. The Q2 characters with the smallest sum of all divergence rankings are taken as the central characters, and Q2 is an integer greater than or equal to 1. For example, the time KL divergence ranking of characters A, B, and C is {1,2,3}, and the location KL divergence ranking is {2,3,1}. Then, the two rankings are summed to obtain the divergence ranking sum of characters A, B, and C: {3,5,4}. If Q2 is 1, the character A with the smallest divergence ranking sum is taken as the central character. It should be noted that when determining the central character, if the Q2 central characters cannot be directly obtained because the divergence ranking sums of multiple characters are the same, the character with the smaller time KL divergence and / or location KL divergence is selected from the multiple characters as the central character. In addition, when the rankings are summed, it can be a direct summation or a weighted summation, and the embodiments of the present application do not impose any limitation on this.

[0507] That is to say, the person whose time KL divergence and location KL divergence meet the first preset condition is determined as the central person, wherein the first preset condition is any one of the following conditions: ① The divergence and ranking are in the top Q1 of the first ranking, and the first ranking is obtained by ranking the characters in order of divergence and ranking from small to large; ② The divergence and ranking are in the top Q2 of the second ranking, and the second ranking is obtained by ranking the characters in order of divergence and ranking from small to large; ③ The divergence and ranking are in the last Q1 of the fourth ranking, and the fourth ranking is obtained by ranking the characters in order of divergence and ranking from large to small; ④ The divergence and ranking are in the last Q2 of the fifth ranking, and the fifth ranking is obtained by ranking the characters in order of divergence and ranking from large to small.

[0508] (4) Social circle identification

[0509] Through social circle identification, the social circle information in the above Table 1 can be obtained, that is, the social circle type, the IDs of the people included in the social circle, etc.

[0510] In one embodiment, the process of social circle identification may include:

[0511] ① According to the intimacy between two characters and the user’s shielding information, the character-image heterogeneous graph is simplified into a character connection graph.

[0512] The character connection graph is used to represent the social relationship between characters and the corresponding intimacy. The character connection graph is a homogeneous graph.

[0513] Specifically, based on the person-graph heterogeneous graph, the image nodes are removed, and the person-related information blocked by the user is removed. Then, two person nodes whose intimacy is not 0 are connected by an edge, and the intimacy is used as the weight of the edge. In this way, a person connection graph can be obtained.

[0514] In one embodiment, Figure 5 The character connection graph obtained by simplifying the character-image heterogeneous graph shown in the figure can be shown as Fig.14 shown.

[0515] It is also understandable that Fig.14 It is a concrete representation of a character connection diagram. In practical applications, the character connection diagram can be expressed in a variety of forms. For example, the character connection diagram can be expressed in a list. The embodiment of the present application does not impose any restrictions on the specific expression of the character connection diagram, as long as it can reflect its content.

[0516] ② Remove the central character nodes in the character connection graph to obtain a decentralized character connection graph.

[0517] Specifically, based on the character connection graph, the node corresponding to the identified central character is removed, and the edges connecting the central character node and other character nodes are removed to obtain a decentralized character connection graph.

[0518] ③ Based on the decentralized character connection graph, conduct community detection and obtain at least one social circle.

[0519] The result of community discovery is multiple communities, each of which corresponds to a social circle. Each social circle includes multiple (at least two) people.

[0520] Optionally, community discovery can be performed based on the Louvain algorithm to obtain at least one social circle. When community discovery is performed based on the Louvain algorithm, the time complexity of the calculation is low, and the community division results are stable, so that the generated social circle results are stable and accurate. Of course, in some embodiments, other algorithms can also be used to perform community discovery and obtain social circles. For example, community discovery can be performed based on an information resource map (infomap) algorithm. The embodiments of the present application do not impose any restrictions on the specific method of community discovery.

[0521] ④ Based on at least one community, determine at least one social circle.

[0522] In one embodiment, community discovery is performed based on the Louvain algorithm. After obtaining at least one community, the set of characters corresponding to all nodes in each community can be directly determined as a social circle, so that the social circles corresponding to each community can be obtained.

[0523] In another embodiment, after the community is discovered based on the Louvain algorithm, the obtained community may be filtered to obtain the final social circle. Specifically, the following may be included:

[0524] For any community, the sum of the weights of the edges connecting each character node in the community with other character nodes in the community is calculated respectively; the N character nodes with the smallest sum of weights are removed from the community to obtain a de-marginalized community; the set of characters corresponding to all nodes in the de-marginalized community is determined as a social circle.

[0525] Among them, the N person nodes with the smallest sum of weights can be called edge person nodes. That is, the edge person nodes in the community are eliminated to obtain a marginal community.

[0526] Each community is processed according to the above process to obtain the social circle corresponding to each community.

[0527] As mentioned above, the weight of an edge is the intimacy between two characters. Therefore, the sum of the weights of the edges connecting a character node to other character nodes characterizes the number of connections between the character node and other character nodes in the community and the size of the weights of each edge. A smaller sum of weights indicates that the character node is less connected to other characters in the community, and / or that the weights of the edges between the node and other nodes are smaller, which further indicates that the character corresponding to the character node is less associated with other characters in the social circle, and / or has a lower degree of intimacy with other characters, that is, the character is a character with a lower degree of connectivity, and is considered to be a marginal character for the social circle. In this embodiment, marginal characters are filtered, and only characters with higher degrees of connectivity are included in the social circle, so that the identified social circle is more in line with the user's actual social situation, further improving the user experience.

[0528] Optionally, the social circle results obtained by community discovery can be represented in the form of a connection graph, that is, a social circle graph is generated. The social circle graph is used to represent the characters included in each social circle, as well as the social relationships and corresponding intimacy between the characters. In one embodiment, the generated social circle graph can be as follows: Fig.15 shown. Fig.15 In the figure, black dots represent character nodes, and the lines between character nodes are edges. The thickness of the edge represents the weight of the edge. The thicker the edge, the greater the weight of the edge, and vice versa. It can be seen that the central character node is connected to multiple character nodes, and some of the edges have larger weights. Fig.12 The social circles shown include three: social circle 1, social circle 2 and social circle 3.

[0529] It can be understood that after the social circle is discovered in the community, the central character node can be added to the social circle graph according to the character connection graph to obtain a character-social circle graph.

[0530] In this embodiment, community discovery is performed based on a decentralized character connection graph to obtain a social circle. This is because: Generally speaking, in a character connection graph, there are many edges connecting the central character node to other nodes, and the central character has a high degree of intimacy with other characters, so the weight of the edge connected to the central character node will be very high. If the central character node is included in community discovery, it is easy to divide the central character node into a certain community in a certain round of iteration. In this way, in the next round of iteration, when the community where the central character node is located is used as a node, there are many edges with other nodes, and / or the sum of the weights of the edges pointing to the community where the central character node is located will be very high, so other points that do not belong to the community will be divided into the community, resulting in inaccurate social circles generated in the end. Therefore, in this embodiment, community discovery is performed based on a decentralized character connection graph to obtain a social circle, which can improve the accuracy of community discovery, thereby improving the accuracy of the obtained social circle and improving user experience.

[0531] ④ According to the central figure and the figure-image heterogeneous graph, determine the type of each social circle. The type of social circle represents the type of relationship between the characters in the social circle and the central figure.

[0532] As a possible implementation method, the type of each social circle can be determined based on a graph neural network model and a multilayer perceptron (MLP) model. Specifically, the following process can be included:

[0533] a. Input the person-image heterogeneous graph into the pre-trained graph neural network model to obtain the feature vectors of each person node and image node.

[0534] The graph neural network model is used to extract the feature vector of the node. Optionally, an initial graph neural network model can be established first. Afterwards, the initial graph neural network model is trained with the person-image heterogeneous graph samples of the labeled data to obtain the graph neural network model. When determining the type of each social circle, the person-image heterogeneous graph is input into the graph neural network model to obtain the feature vector of each person node and image node.

[0535] Among them, the character-image heterogeneous graph includes information of each character node and information of image nodes. As mentioned above, the information of the character node may include the character's gender, character's age, etc. The feature vector of the character node extracted by the graph neural network model may include the character's gender feature vector, the character's age feature vector, etc. Optionally, the value of the character's gender feature vector is 0 or 1, and optionally, a value of 0 indicates that the gender is male, and a value of 1 indicates that the gender is female. The value range of the character's age feature vector can be [0,100], and the value of the character's age can be an integer.

[0536] As described above, the information of the image node may include the shooting time information of the image, the shooting location of the image, whether it is shot by the front camera, etc. The feature vector of the image node extracted by the graph neural network model may include: shooting time feature vector, shooting location feature vector and whether it is a front camera feature vector, etc. Among them, the shooting time feature vector may include shooting time feature vector, time information feature vector, month difference feature vector and week feature vector, etc. Specifically, the time information feature vector refers to the feature vector of the time information at the shooting time. The value range of the time information feature vector may be [0,23], and the value of the time information feature vector is an integer. The month difference feature vector refers to the feature vector of the difference between the month information at the shooting time and the month information at the current time. The value range of the month difference feature vector may be [0,100], and the value of the month difference feature vector is an integer. The week feature vector refers to the feature vector of the week information at the shooting time. The value range of the week feature vector may be [1,7], and the value of the week feature vector is an integer. The shooting location feature vector may include a one-hot encoding vector defined by home, company, scenic spot, etc. The value of the feature vector of whether the front camera is used can be 0 or 1, which is optional. A value of 0 indicates that the image is not taken with the front camera, and a value of 1 indicates that the image is taken with the front camera.

[0537] b. Input the feature vectors of each character node and image node into the pre-trained multi-layer perceptron model to obtain the relationship type between each character node and the central character node (referred to as the relationship type of the character node).

[0538] The multi-layer perceptron model is used to predict the type of relationship between each character node and the central character node. The type of relationship between the character node and the central character node may include but is not limited to family members, colleagues, and classmates.

[0539] Optionally, an initial multi-layer perceptron model may be established first. Then, the initial multi-layer perceptron model is trained using node feature vector samples of labeled data to obtain a multi-layer perceptron model. When determining the type of a social circle, the feature vectors of the person node and the image node obtained in step a are input into the multi-layer perceptron model to obtain the relationship type of each person node and the central person node, that is, to obtain the relationship type of each person node.

[0540] c. Count the number of people corresponding to each relationship type in each social circle, and determine the relationship type with the largest number of people as the type of the social circle.

[0541] For example, for any social circle C, the relationship types of the characters in the social circle C include "colleague" and "classmate", then the number of characters with the relationship type of "colleague" in the social circle C is counted (for example, x1), and the number of characters with the relationship type of "classmate" in the social circle C is counted (for example, x2). If x1 is greater than x2, "colleague" is determined as the type of the social circle C.

[0542] d. Modify the relationship type of characters in each social circle that are different from the type of the social circle (i.e., the nodes corresponding to the characters to be modified) to the type of the social circle.

[0543] For example, in the above example, if the determined type of social circle C is "colleague", the relationship type of each character in social circle C whose relationship type is not "colleague" is modified to "colleague". In this way, the relationship type of the characters in the social circle C that are judged to be biased can be corrected, so that the relationship between the characters in the determined social circle C and the central character is consistent and accurate, which is convenient for generating image collections based on social circles in the future.

[0544] The above describes in detail an example of an image collection recommendation method provided by an embodiment of the present application. It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0545] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0546] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0547] The electronic device provided in this embodiment is used to execute the above-mentioned image collection recommendation method, and thus can achieve the same effect as the above-mentioned implementation method.

[0548] In the case of an integrated unit, the electronic device may further include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the electronic device to communicate with other devices.

[0549] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0550] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 2 Device of the structure shown.

[0551] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the image collection recommendation method of any of the above embodiments.

[0552] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the image collection recommendation method in the above-mentioned embodiment.

[0553] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the image collection recommendation method in the above-mentioned method embodiments.

[0554] The electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0555] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0556] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0557] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0558] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0559] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0560] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for recommending an image collection, the method being executed by an electronic device, characterized in that: The method comprises: Obtain at least one image collection; If there are n first collections in the at least one image collection, the type of each first collection is obtained respectively; n is an integer greater than 1, and the first collection refers to a collection whose priority type is a first priority; Determine the number of times the same type of image collection corresponding to the second collection has been recommended, where the second collection is any one of the n first collections, and the number of times the same type of image collection corresponding to the second collection has been recommended refers to the number of times the image collection of the same type as the second collection has been recommended; sorting the n first collections according to the type of each first collection and the corresponding number of times the same type has been recommended, to obtain a first sorting result; Determining a collection of images to be recommended according to the first sorting result; The image collection to be recommended is recommended.

2. The method according to claim 1, characterized in that: The step of sorting the n first collections according to the type of each first collection and the corresponding number of times the same type of collection has been recommended to obtain a first sorting result includes: Determining content scores of the first collections respectively according to the types of the first collections; Determine the frequency suppression score of each of the first collections according to the number of times the same type of collection has been recommended corresponding to each of the first collections, wherein the frequency suppression score of the first collection is negatively correlated with the corresponding number of times the same type of collection has been recommended; The n first collections are sorted according to the content score and the frequency suppression score of each of the first collections to obtain the first sorting result.

3. The method according to claim 2, characterized in that The step of sorting the n first collections according to the content score and the frequency suppression score of each first collection to obtain the first sorting result includes: sorting the n first collections according to the content scores to obtain a content sorting result; Sort the n first collections according to the frequency suppression scores to obtain a frequency sorting result; Determine a recommendation score for each of the first collections according to the sum of the ranking of each of the first collections in the content ranking result and the ranking of each of the first collections in the frequency ranking result; The n first collections are sorted according to the recommendation scores to obtain the first sorting result.

4. The method according to claim 2 or 3, characterized in that: The type of the image collection includes a plurality of preset attributes, and the type of each of the first collections includes attribute values ​​corresponding to each of the preset attributes; and determining the content scores of each of the first collections according to the type of each of the first collections includes: For the second collection, according to a preset correspondence between attribute values ​​and scores, determining the scores corresponding to the attribute values ​​in the types of the second collection; The scores corresponding to the attribute values ​​are weighted and summed to obtain the content score of the second collection.

5. The method according to claim 4, characterized in that The multiple preset attributes include a character subject, a generation time and a time span. The character subject represents the character included in the images in the collection. The generation time represents the time characteristics of the collection generation. The time span represents the time difference between the latest shooting time and the earliest shooting time of the images in the collection.

6. The method according to any one of claims 2 to 5, characterized in that The determining, according to the number of times the same type of recommendation corresponding to each of the first collections has been made, the frequency suppression score of each of the first collections comprises: The frequency suppression scores of each of the first collections are determined respectively according to the following formula: y frequency =1 / log(n class +1); Among them, y frequency The frequency suppression score of the first collection indicates the number of times the same category corresponding to the first collection has been recommended.

7. The method according to any one of claims 1 to 6, characterized in that The determining the image collection to be recommended according to the first sorting result includes: Taking the collection ranked first in the first sorting result as the third collection, and determining whether the third collection meets a preset repetition rate condition; If the third collection meets the preset repetition rate condition, determining the third collection as the image collection to be recommended; If the third collection does not meet the preset repetition rate condition, the fourth collection is taken as the third collection, and the process returns to the step of determining whether the third collection meets the preset repetition rate condition until the image collection to be recommended is determined or the n first collections are traversed; the fourth collection refers to a collection whose ranking in the first sorting result is adjacent to the third collection and whose ranking is after the third collection.

8. The method according to claim 7, characterized in that The preset repetition rate condition is: there is no fifth collection, the fifth collection is a collection of the same type as the third collection and has been recommended, and the number of the same images in the fifth collection and the third collection is less than a preset threshold.

9. The method according to claim 8, characterized in that The time difference between the current time and the recommended time of the fifth collection is less than the first preset duration.

10. The method according to any one of claims 7 to 9, characterized in that The method comprises: If the third collection does not meet the preset repetition rate condition, the third collection is deleted.

11. The method according to any one of claims 7 to 10, characterized in that The method further comprises: If the image collection to be recommended is not determined by traversing the n first collections, determine whether the time difference between the current moment and the moment of the most recent recommendation of the sixth collection is greater than a preset time interval; if so, select the image collection to be recommended from the sixth collection of the at least one image collection, the sixth collection being a collection of the second priority type, and the level of the second priority is lower than that of the first priority.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: If there is one first collection in the at least one image collection, determining whether the one first collection meets a preset repetition rate condition; If the one of the first collections meets the preset repetition rate condition, determining the one of the first collections as the image collection to be recommended; If the first collection does not meet the preset repetition rate condition, deleting the first collection, and determining whether the time difference between the current moment and the moment of the most recent recommendation of the sixth collection is greater than a preset time interval; If so, the image collection to be recommended is selected from a sixth collection of the at least one image collection, the sixth collection being a collection of a second priority type, the second priority being lower than the first priority.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: If the first collection does not exist in the at least one image collection, determining whether the time difference between the current moment and the moment when the sixth collection was most recently recommended is greater than a preset time interval; If so, the image collection to be recommended is selected from a sixth collection of the at least one image collection, the sixth collection being a collection of a second priority type, the second priority being lower than the first priority.

14. The method according to any one of claims 1 to 13, characterized in that The at least one image collection includes a target collection and a generated collection, wherein the generated collection is an image collection that has been generated and has not been recommended; Before acquiring at least one image collection, the method further includes: Acquire social information, where the social information represents social relationships between people included in the images in the gallery; If it is determined based on the social information that the images containing the person in the gallery meet the first collection generation condition, then images meeting the second condition are obtained from the gallery to generate a first candidate collection; Performing deduplication processing on the images in the first candidate collection to obtain a second candidate collection; The second candidate collection is sampled according to the shooting time and shooting location of the images in the second candidate collection to obtain the target collection.

15. The method according to claim 14, characterized in that The social information includes the intimacy between each character and the first character, and the first collection generation condition includes: The image most recently added to the gallery includes at least one first image; the first image is an image containing a second person, and the intimacy between the second person and the first person is greater than a preset intimacy threshold; The sum of the number of historical images and the first image in the gallery is greater than or equal to a preset number, and the historical image refers to an image of the second person included in the gallery before the most recent image addition; The time difference between the first shooting moment and the second shooting moment is greater than a second preset duration, the first shooting moment refers to the latest shooting moment of the historical images; the second shooting moment refers to the earliest shooting moment of the at least one first image.

16. The method according to claim 15, characterized in that The image satisfying the second condition refers to an image containing the second person.

17. The method according to any one of claims 14 to 16, characterized in that The step of sampling the second candidate collection according to the shooting time and shooting location of the images in the second candidate collection to obtain the target collection includes: Grouping the images in the second candidate collection according to shooting locations and shooting times to obtain at least one comprehensive group; The images in each of the comprehensive groups are grouped according to the aesthetic scores of the images in each of the comprehensive groups to obtain a preliminary screening group and a candidate group corresponding to each of the comprehensive groups, wherein the aesthetic scores of the images in the preliminary screening group are greater than a preset score threshold, and the aesthetic scores of the images in the candidate group are less than or equal to the preset score threshold; the aesthetic scores represent the aesthetic quality of the images; Determining the expected number of samples corresponding to each of the comprehensive groups; For the first comprehensive group, if the number of images in the first preliminary screening group is 0, the first comprehensive group is not sampled; the first comprehensive group is any one of the at least one comprehensive group, and the first preliminary screening group is the preliminary screening group corresponding to the first comprehensive group; If the number of images in the first preliminary screening group is greater than or equal to the first expected sampling number, sampling the first expected sampling number of images from the first preliminary screening group, where the first expected sampling number is the expected sampling number corresponding to the first comprehensive group; If the number of images in the first preliminary screening group is greater than 0 and less than the first expected sampling number, a first number of images in the first candidate group are selected and added to the first preliminary screening group to obtain a supplementary group; if the number of images in the supplementary group is greater than or equal to the first expected sampling number, the first expected sampling number of images are sampled from the supplementary group; if the number of images in the supplementary group is less than the first expected sampling number, the first comprehensive group is not sampled; the first candidate group is a candidate group corresponding to the first comprehensive group, and the first number is determined according to the number of images in the first candidate group; The images sampled from the at least one comprehensive group are aggregated to obtain the target collection.

18. The method according to claim 17, characterized in that The step of grouping the images in the second candidate collection according to the shooting location and the shooting time to obtain at least one comprehensive group includes: Grouping the images in the second candidate collection according to shooting locations to obtain at least one location group; Grouping the images in each of the location groups according to shooting time to obtain the at least one comprehensive group; For the first comprehensive grouping, determining the first expected sampling quantity comprises: determining, according to the number of images in a first location group, a first attenuation weight of a shooting location corresponding to the first location group, wherein the first attenuation weight is negatively correlated with the number of images in the first location group; The first expected sampling quantity is determined according to the product of the quantity of images in the first comprehensive group, the first attenuation weight and a first preset ratio.

19. The method according to claim 18, characterized in that The first number is determined according to the product of the number of images in the first candidate group, the first attenuation weight, and a second preset ratio.

20. The method according to any one of claims 17 to 19, characterized in that The step of sampling the first expected sampling number of images from the first preliminary screening group includes: The first expected sampling number of images are sampled from the first preliminary screening group according to the comprehensive score of each image in the first preliminary screening group; the comprehensive score is determined according to at least two of the aesthetic score, interaction score and semantic score of the image, the interaction score is determined according to the amount of interaction between the user and the image, the semantic score is determined according to the amount of interaction between the user and images with the same type of semantics, and the aesthetic score represents the aesthetic quality of the image.

21. The method according to any one of claims 14 to 20, characterized in that Before obtaining the social information, the method further includes: After adding an image containing a person to the gallery, obtaining a person image heterogeneous graph and user shielding information; the person image heterogeneous graph represents the corresponding relationship between the images in the gallery and the persons contained in the images, and the user shielding information refers to information related to the images in the gallery that is shielded by the user; The images in the gallery are analyzed according to the character image heterogeneous graph and the user shielding information to obtain the social information.

22. The method according to any one of claims 14 to 21, characterized in that After sampling the second candidate collection according to the shooting time and shooting location of the images in the second candidate collection to obtain the target collection, the method further includes: The type of the target collection and the number of images in the target collection are recorded.

23. The method according to any one of claims 1 to 22, characterized in that After determining the image collection to be recommended according to the first sorting result, the method further includes: Record at least one of the type and the total number of recommendations corresponding to the image collection to be recommended, wherein the total number of recommendations is: a value obtained by adding one to the number of times the same type of recommendation corresponding to the image collection to be recommended has been recommended; the number of times the same type of recommendation corresponding to the image collection to be recommended refers to the number of times the image collection of the same type as the image collection to be recommended has been recommended.

24. An electronic device, characterized in that: include: Processors, memory, and interfaces; The processor, the memory and the interface cooperate with each other so that the electronic device executes the method as claimed in any one of claims 1 to 23.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 23.

Images