Cover Generation Method, System, Device and Storage Medium Based on Travel Photography Albums

Abstract

The present invention provides a method, system, device and storage medium for generating a cover based on a travel photography album. The method includes: establishing a travel photography album based on travel photos; performing image recognition on all travel photos in the travel photography album to obtain face regions, and performing face recognition to obtain face information in each travel photo, sorting the occurrence frequencies of each face information in the travel photography album from largest to smallest, determining whether the travel photography album is a couple travel photography or a group travel photography based on the differences between the face information with the largest frequency, the face information with the second largest frequency, and the face information with the third largest frequency, and selecting a cover according to the ratio of the pixels occupied by the total pattern of the people to the total pixels of the photo. The present invention can automatically classify travel photography albums and provide targeted cover selection, reducing manpower consumption and optimizing the user experience.

Description

Background Art

[0002] Travel photography, also known as travel shooting, is the abbreviation of traveling and shooting. During the journey, all the scenery of the unknown world is recorded with the aperture and shutter. Travel photography is widely used in wedding photography. With the upgrading of people's consumption, traditional tourism and extreme sports tourism have also started to incorporate shooting service elements. For example, photo studios or large photography agencies will set up points in various places, covering aspects such as local dining, accommodation, and photography staff. At the destination, there are full-time local photographers providing local shooting services and also taking on part of the work of a tour guide to provide local guidance for the shooters.

[0003] Currently, travel photography mainly includes two forms: group travel photography and couple travel photography (including wedding travel photography). In order to attract more customers, the travel photography list page usually manually selects high-quality pictures as the first pictures for the product details page or the user album module. Facing a large number of travel photography images, a large amount of manpower is required for selection. How to intelligently select high-quality travel photography images and reduce manpower consumption has become an urgent problem to be solved.

[0004] Therefore, the present invention provides a method, system, device, and storage medium for generating a cover based on a travel photography album. Summary of the Invention

[0005] Aiming at the problems in the prior art, the purpose of the present invention is to provide a method, system, device, and storage medium for generating a cover based on a travel photography album, which overcomes the difficulties of the prior art, can automatically classify travel photography albums and provide targeted cover selection, reduces manpower consumption, and optimizes the user experience.

[0006] An embodiment of the present invention provides a method for generating a cover based on a travel photography album, including the following steps:

[0007] S110. Establish a travel photography album based on travel photos;

[0008] S120. Perform image recognition on all travel photos in the travel photography album to obtain face regions;

[0009] S130. Perform face recognition on the face regions to obtain face information in each travel photo;

[0010] S140. Sort the appearance frequencies of each face information in the travel photography album from largest to smallest, and judge whether the travel photography album is couple travel photography or group travel photography based on the differences between the face information with the largest frequency, the face information with the second largest frequency, and the face information with the third largest frequency; and

[0011] S150. Select the photos that only include the couple from the travel photos of the couple's travel photography, and select a photo as the cover of the travel photo album at least based on the ratio of the pixels occupied by the two people in the photo to the total pixels of the photo; or, obtain the photos that only include the top N faces with the highest appearance frequency from the travel photos of the group travel photography, and select a photo as the cover of the group travel photography at least based on the ratio of the pixels occupied by the top N people with the highest appearance frequency in the photo to the total pixels of the photo, where N > 2.

[0012] Preferably, the step S120 includes:

[0013] S121. Perform image recognition on all the travel photos of the travel photo album;

[0014] S122. Establish the face region, body region, and background region of each travel photo, and establish the corresponding relationship between the body region and the face region pair.

[0015] Preferably, the step S130 includes:

[0016] S131. Perform face recognition on each face region of the travel photo to obtain the face information in each travel photo;

[0017] S132. Perform similarity comparison on the face information in the travel photo album to establish the attribution relationship between the face information and the user.

[0018] Preferably, the step S140 includes:

[0019] S141. Sort the appearance frequencies of each face information in the travel photo album from largest to smallest, and determine whether the difference between the face information with the second highest frequency and the face information with the third highest frequency is greater than or equal to a second preset threshold. If so, execute step S142; if not, execute step S143;

[0020] S142. Mark the travel photo album as a couple's travel photography;

[0021] S143. Mark the travel photo album as a group travel photography.

[0022] Preferably, the step S141 is replaced by:

[0023] Sort the appearance frequencies of each face information in the travel photo album from largest to smallest, and determine whether the difference between the face information with the highest frequency and the face information with the second highest frequency is less than or equal to a first preset threshold, and whether the difference between the face information with the second highest frequency and the face information with the third highest frequency is greater than or equal to a second preset threshold; if so, execute step S142; if not, execute step S143.

[0024] Preferably, the step S150 includes:

[0025] S151. Select the photos that only include the couple from the travel photos of the couple travel shooting as candidate covers to establish a first candidate cover set;

[0026] S152. Conduct weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters of each candidate cover in the first candidate cover set to obtain candidate parameters, where the visual balance parameter is the ratio of the total pixel of the local images of the two people to the total pixel of the photo; and

[0027] S153. Select the candidate cover with the highest candidate parameter as the cover of the couple travel shooting.

[0028] Preferably, the step S150 includes:

[0029] S156. Select the photos that only include the people with the top N appearance frequencies from the travel photos of the group travel shooting as candidate covers to establish a second candidate cover set;

[0030] S157. Conduct weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters of each candidate cover in the second candidate cover set to obtain candidate parameters, where the visual balance parameter is the ratio of the total pixel of the local images of the people with the top N appearance frequencies to the total pixel of the photo; and

[0031] S158. Select the candidate cover with the highest candidate parameter as the cover of the group travel shooting.

[0032] An embodiment of the present invention further provides a cover generation system based on a travel shooting album for implementing the above-mentioned cover generation method based on a travel shooting album. The cover generation system based on a travel shooting album includes:

[0033] An album establishment module for establishing a travel shooting album based on travel photos;

[0034] An image segmentation module for performing image recognition on all travel photos of the travel shooting album to obtain face regions;

[0035] A face recognition module for performing face recognition on the face regions to obtain face information in each travel photo;

[0036] An album classification module for sorting the appearance frequencies of each face information in the travel shooting album from large to small, and judging whether the travel shooting album is a couple travel shooting or a group travel shooting based on the difference between the face information with the largest frequency, the face information with the second largest frequency, and the face information with the third largest frequency; and

[0037] A cover selection module that selects, from the travel photoshoot photos of the couple, only the photos that include only the couple, and selects photos as the cover of the travel photo album based at least on the ratio of the pixels occupied by the two people in the photo to the total pixels of the photo; or, obtains, from the travel photoshoot photos of the group, only the photos that include the top N faces with the highest appearance frequencies, and selects photos as the cover of the group travel photoshoot based at least on the ratio of the pixels occupied by the top N people with the highest appearance frequencies in the photo to the total pixels of the photo, where N > 2.

[0038] An embodiment of the present invention further provides a cover generation device for a travel photo album, including:

[0039] A processor;

[0040] A memory that stores executable instructions of the processor;

[0041] Wherein, the processor is configured to execute the steps of the above-mentioned cover generation method for a travel photo album by executing the executable instructions.

[0042] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, and when the program is executed, it implements the steps of the above-mentioned cover generation method for a travel photo album.

[0043] The purpose of the present invention is to provide a cover generation method, system, device, and storage medium for a travel photo album, which can automatically classify the travel photo album and provide targeted cover selection, reduce labor consumption, and optimize the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objectives, and advantages of the present invention will become more apparent.

[0045] Figure 1 is a flowchart of the cover generation method for a travel photo album of the present invention.

[0046] Figure 2 is a schematic diagram of a scoring process in the cover generation method for a travel photo album of the present invention.

[0047] Figure 3 is a schematic diagram of feature extraction in the cover generation method for a travel photo album of the present invention.

[0048] Figure 4 is a schematic diagram of the modules of the cover generation system for a travel photo album of the present invention.

[0049] Figure 5 is a schematic diagram of the structure of the cover generation device for a travel photo album of the present invention.

[0050] Figure 6 It is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. Detailed implementation manners

[0051] The following uses specific specific examples to illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present application. The present application can also be implemented or applied through other different specific implementation manners. Various details in the present application can also be modified or changed according to different viewpoints and application systems without departing from the spirit of the present application. It should be noted that, without conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0052] The following takes the drawings as a reference and details the embodiments of the present application so that those skilled in the technical field to which the present application belongs can easily implement it. The present application can be embodied in many different forms and is not limited to the embodiments described herein.

[0053] In the description of the present application, the reference terms such as "an embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials or characteristics represented in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics represented can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples represented in the present application and the features of different embodiments or examples.

[0054] In addition, the terms "first" and "second" are only used for the purpose of indication and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality" is two or more unless otherwise specifically defined.

[0055] In order to clearly illustrate the present application, devices irrelevant to the description are omitted, and the same or similar constituent elements throughout the specification are given the same reference numerals.

[0056] Throughout the specification, when it is said that a device is "connected" to another device, this includes not only the case of "direct connection", but also the case of "indirect connection" with other elements placed therebetween. In addition, when it is said that a certain device "includes" a certain constituent element, unless there is a particularly contrary record, it does not exclude other constituent elements, but means that other constituent elements can also be included.

[0057] When it is said that a device is "above" another device, this can be directly above the other device, but there can also be other devices in between. When it is said, in contrast, that a device is "directly" "above" another device, there are no other devices in between.

[0058] Although in some instances the terms first, second, etc. are used herein to denote various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first interface and a second interface, etc. are indicated. Furthermore, as used herein, the singular forms "a", "an", and "the" are intended to also include the plural forms unless the context clearly dictates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of the features, steps, operations, elements, components, items, kinds, and / or groups, but do not preclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". An exception to this definition only occurs when the combination of elements, functions, steps, or operations are mutually exclusive in some manner.

[0059] The technical terms used herein are only for referring to specific embodiments and are not intended to limit this application. The singular forms used herein also include the plural forms as long as the statements do not clearly indicate the contrary meaning. The meaning of "including" used in the specification is to embody specific characteristics, regions, integers, steps, operations, elements, and / or components, and does not exclude the existence or addition of other characteristics, regions, integers, steps, operations, elements, and / or components.

[0060] Although not defined differently, including the technical terms and scientific terms used herein, all terms have the same meaning as generally understood by those skilled in the technical field to which this application pertains. Terms defined in commonly used dictionaries are additionally interpreted as having a meaning consistent with the relevant technical literature and the content currently presented. As long as they are not defined, they should not be over-interpreted as ideal or overly formulaic meanings.

[0061] Figure 1 is a flowchart of the method for generating a cover of a travel photo album according to the present invention. As Figure 1 shown, the method for generating a cover of a travel photo album according to the present invention includes:

[0062] S110. Establish a travel photo album based on travel photos.

[0063] S120. Perform image recognition on all the travel photography pictures in the travel photography album to obtain the face regions.

[0064] S130. Perform face recognition on the face regions to obtain the face information in each travel photography picture.

[0065] S140. Sort the occurrence frequencies of each piece of face information in the travel photography album from largest to smallest, and judge whether the travel photography album is a couple travel photography or a group travel photography based on the differences among the face information with the largest frequency, the second largest frequency, and the third largest frequency. And

[0066] S150. Select only the pictures that include only the couple in the travel photography pictures of the couple travel photography, and select pictures as the cover of the travel photography album based at least on the ratio of the pixels occupied by the two people in the picture to the total pixels of the picture. Or, obtain the pictures that include only the faces of the top N people with the highest occurrence frequencies in the travel photography pictures of the group travel photography, and select pictures as the cover of the group travel photography based at least on the ratio of the pixels occupied by the top N people with the highest occurrence frequencies in the picture to the total pixels of the picture, where N > 2. Obviously, in the couple travel photography, the occurrence frequency of the obvious couple is far higher than that of others, and there will be an obvious gap between the second and the third; differently, the occurrence frequencies of the top N people in the group are relatively close. By statistically analyzing the occurrence frequency of each face in the album through face recognition, the differences between the two can be identified, which also provides an accurate classification effect for the subsequent targeted selection of the cover.

[0067] In a preferred embodiment, step S120 includes:

[0068] S121. Perform image recognition on all the travel photography pictures in the travel photography album. In this embodiment, an existing trained first neural network model for image segmentation is used, but it is not limited thereto.

[0069] S122. Establish the face region, body region, and background region of each travel photography picture, and establish the corresponding relationship between the body region and the face region pair and the pixels occupied by each region for facilitating the subsequent calculation of the proportion of the picture, but it is not limited thereto.

[0070] In a preferred embodiment, step S130 includes:

[0071] S131. Perform face recognition on each face region of the travel photography picture to obtain the face information in each travel photography picture. In this embodiment, an existing trained second neural network model for face recognition is used to classify the faces that appear in different travel photography pictures, so as to identify which user each face that appears in the picture belongs to, but it is not limited thereto.

[0072] S132. Compare the facial information in the travel photography album for similarity, and establish the attribution relationship between the facial information and the user, but not limited thereto.

[0073] In a preferred embodiment, step S140 includes:

[0074] S141. Sort the appearance frequencies of each facial information in the travel photography album from largest to smallest, and determine whether the difference between the facial information with the second highest frequency and the facial information with the third highest frequency is greater than or equal to a second preset threshold. If so, execute step S142; if not, execute step S143.

[0075] S142. Mark the travel photography album as a couple travel photography.

[0076] S143. Mark the travel photography album as a group travel photography, but not limited thereto.

[0077] In a preferred embodiment, step S141 is replaced by:

[0078] Sort the appearance frequencies of each facial information in the travel photography album from largest to smallest, and determine whether the difference between the facial information with the largest frequency and the facial information with the second largest frequency is less than or equal to a first preset threshold, and whether the difference between the facial information with the second largest frequency and the facial information with the third largest frequency is greater than or equal to a second preset threshold. If so, execute step S142; if not, execute step S143, but not limited thereto.

[0079] In a preferred embodiment, step S150 includes:

[0080] S151. Select only the photos that include only the couple from the travel photos of the couple travel photography as candidate covers to establish a first candidate cover set.

[0081] S152. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters for each candidate cover in the first candidate cover set to obtain candidate parameters, where the higher the clarity, the higher the clarity score; the greater the foreground-background contrast, the higher the contrast score; the greater the color saturation, the higher the saturation score, and the visual balance parameter is the ratio of the total pixel of the local images of the two people to the total pixel of the same photo.

[0082] S153. Select the candidate cover with the highest candidate parameter as the cover of the couple travel photography, but not limited thereto.

[0083] In a preferred embodiment, step S150 includes:

[0084] S156. Select only the photos that include only the people with the top N appearance frequencies from the travel photos of the group travel photography as candidate covers to establish a second candidate cover set.

[0085] S157. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters for each candidate cover in the second candidate cover set to obtain candidate parameters. Among them, the higher the clarity, the higher the clarity score; the greater the foreground-background contrast, the higher the contrast score; the greater the color saturation, the higher the saturation score. The visual balance parameter is the ratio of the total pixels of the local images of the top N people with the highest appearance frequency to the total pixels of the same photo. And

[0086] S158. Use the candidate cover with the highest candidate parameters as the cover for the group travel photography, but this is not the only limit.

[0087] Through the statistics of the appearance frequency of human faces in the photo album, the present invention shows the classification of the couple travel photography album and the group travel photography album. According to the classification results, the most representative group photo of the couple is selected as the cover of the couple travel photography album in the couple travel photography album, and the group photo representing the top N people with the highest appearance frequency is selected as the cover of the group travel photography album in the group travel photography album. The present invention can automatically classify the travel photography album and provide targeted cover selection, reducing labor consumption and optimizing the user experience.

[0088] The specific implementation process of the present invention is as follows:

[0089] First, establish a travel photography album based on the travel photos. Perform image recognition on all the travel photos in the travel photography album. Establish the face area, body area, and background area of each travel photo, and establish the corresponding relationship between the body area and the face area pair. Perform face recognition on each face area of the travel photo to obtain the face information in each travel photo. Compare the similarity of the face information in the travel photography album to establish the attribution relationship between the face information and the user. Sort the appearance frequency of each face information in the travel photography album from large to small, and judge whether the difference between the face information with the highest frequency and the face information with the second highest frequency is less than or equal to the first preset threshold, and whether the difference between the face information with the second highest frequency and the face information with the third highest frequency is greater than or equal to the second preset threshold. If so, the travel photography album is marked as a couple travel photography album; if not, the travel photography album is marked as a group travel photography album.

[0090] Select the photos that only include the couple in the travel photos of couple travel shooting as candidate covers to establish the first candidate cover set. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameter of each candidate cover in the first candidate cover set to obtain candidate parameters. The visual balance parameter is the ratio of the total pixel of the local images of the two people to the total pixel of the photo. And select the candidate cover with the highest candidate parameter as the cover of the couple travel shooting. Or, select the photos that only include the people with the top N appearance frequencies in the travel photos of group travel shooting as candidate covers to establish the second candidate cover set. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameter of each candidate cover in the second candidate cover set to obtain candidate parameters. The visual balance parameter is the ratio of the total pixel of the local images of the people with the top N appearance frequencies to the total pixel of the photo. Select the candidate cover with the highest candidate parameter as the cover of the couple travel shooting, but not limited to this.

[0091] The present invention can automatically classify travel shooting albums and provide targeted cover selection, reducing manpower consumption and optimizing user experience.

[0092] In addition, the present invention can first perform low-resolution screening on travel shooting pictures, that is, set a picture pixel threshold and screen out the pictures below the threshold; secondly, extract features for the clarity, foreground-background (portrait and background) contrast, color saturation, depth of field, visual balance, and composition rules of travel shooting images, and perform binary classification on the screened pictures according to the extracted features, that is, high-quality pictures and low-quality pictures; finally, extract features and score the high-quality pictures in the above steps. In order to make the scoring more accurate, the scoring rule scores the images based on the probability distribution of the comprehensive score.

[0093] The present invention also relates to image aesthetic degree scoring, which includes two parts: feature extraction and decision-making. Among them, the feature extraction part is the reference features for aesthetic degree scoring, mainly including features such as clarity, contrast, color saturation, visual balance, depth of field, and the rule of thirds; the decision-making part scores more accurately based on the probability distribution of the image scores after passing through the model on the basis of classification.

[0094] The present invention proposes a system for intelligently scoring travel shooting images. The invention is mainly divided into a picture pre-screening module, a scoring system classification module, and a scoring system scoring module. In the classification module and scoring module of the scoring system, according to the characteristics of travel shooting images, traditional features such as image clarity, exposure, and color saturation, the feature of the proportion of the human body in the image, and face features are added on the basis of extracting general features. The technical solution is implemented in the following way:

[0095] (1) First, perform low-pixel screening on the image, that is, set a pixel threshold and screen out the images below the threshold from entering the scoring system;

[0096] (2) Extract image technical features such as image sharpness, foreground-background contrast, and color saturation through traditional image processing;

[0097] (3) Extract image perception features such as visual balance and depth of field of the image through traditional image processing;

[0098] (4) Segment the travel photography pictures through the image segmentation algorithm FCN (Fully Convolution Networks), and calculate the distances between the human body center and each boundary of the image to calculate whether the proportion of the human body in the image conforms to the rule of thirds;

[0099] (5) Extract face features through the face recognition algorithm, mainly to judge whether the face is blocked;

[0100] (6) Scoring system classification module: Use the MobileNet V1 network to extract general features of the pictures, and combine the features extracted in (2), (3), (4), and (5) to classify the pictures screened in (1) into high-quality pictures and low-quality pictures, so as to reduce the impact of low-quality pictures on the subsequent scoring module;

[0101] (7) Scoring system scoring module: Use the MobileNet V1 network to extract general features and combine the features extracted in (2), (3), (4), and (5) to score the pictures classified as high-quality in (6), and score the pictures from 0 to 5 points;

[0102] (8) In order to make the scoring more accurate, the scoring rule scores the image based on the probability distribution of the comprehensive score.

[0103] Figure 2 is a schematic diagram of a scoring process in the cover generation method based on travel photography albums of the present invention. Figure 3 is a schematic diagram of feature extraction in the cover generation method based on travel photography albums of the present invention. As Figure 2 and 3 shown, the present invention uses MobileNet V1 to extract general features in both the classification module of the scoring system and the scoring module of the scoring system. At the same time, on the basis of extracting general features, image technical features such as image sharpness, foreground-background contrast, and color saturation, composition, image perception features such as visual balance and depth of field, and face features are added.

[0104] The present invention can fully automatically achieve the aesthetic score of travel photography images and select high-quality pictures as the first pictures in the product details module and the user album module; on the basis of extracting the general features of pictures, combined with the characteristics of travel photography pictures, traditional features such as the clarity, exposure and color saturation of the images, the proportion of the human body in the image, and the face features are extracted to classify and score the pictures; in order to make the scoring more accurate, the scoring rule of the scoring module comprehensively considers the probability distribution of the scores to score the pictures. The present invention can provide a method for fully automatic aesthetic scoring. This method can achieve full-automatic scoring for travel photography pictures and disclose the high-scoring high-quality pictures in the header module of the product details page and the user album module.

[0105] As Figure 2 、 3 shown, the main process of the score calculation mechanism in the cover generation method based on travel photography albums of the present invention is as follows:

[0106] (1) First, set a pixel threshold to screen out the images with pixels lower than the threshold, that is, the low-pixel images.

[0107] (2) Input the non-low-pixel images into the classification module using the classification network, that is, classify the non-low-pixel images into high-quality and low-quality pictures. Since the purpose of this invention is to make the pictures with high scores evaluated by the scoring system as much as possible conform to the aesthetics of human scoring, in order to reduce the influence of low-score pictures on the next scoring module, binary classification is first performed to screen out the low-score pictures as low-quality pictures as much as possible, and the low-quality pictures do not enter the scoring module;

[0108] (3) Input the high-quality pictures extracted in (2) into the scoring module and score the pictures from 0 to 5 points;

[0109] (4) The features extracted in the classification module and the scoring module in (2) and (3) are all based on using MobileNet V1 to extract general features, and image technical features such as the clarity, foreground-background contrast and color saturation of the image, image perception features such as composition, visual balance and depth of field, and face features are added;

[0110] (5) For the clarity, foreground-background contrast, and color saturation features in (4), use traditional image processing methods to extract them:

[0111] (a) Clarity mainly affects the aesthetic quality of the image from the aspect of details. Clarity feature extraction method: Perform FFT transformation on the grayscale image I and calculate the number N of frequencies with power greater than a pre-set threshold θ. The ratio of N to I is used as the clarity feature:

[0112] F = FFT(I) (1)

[0113] N = {(u, v)||F(u, v)| > 0} (2)

[0114]

[0115] (b) Contrast mainly affects the aesthetic quality of the image from the aspect of brightness. This paper first uses the Canny edge detection algorithm to detect the foreground and frame it with a rectangle. After framing the foreground, first extract its 4-dimensional hue histogram as features f1_f4, and calculate the average brightness feature f5 of the foreground, and at the same time calculate the ratio of the foreground to the entire image as feature f6. At the same time, calculate the difference in hue and brightness between the foreground and the background as features f7, f8. Among them, the hue and brightness values ​​are obtained by calculating the average values ​​of all pixels in the foreground and background.

[0116] (c) Saturation mainly affects the aesthetic quality of an image from the perspective of color. In this paper, a saturation evaluation algorithm based on logistic regression is used to extract saturation features. The algorithm uses regression analysis to predict the optimal saturation of an image based on the relationship between image saturation and brightness, and then compares it with the average saturation of the image.

[0117] (d) Visual balance refers to the visual center of the object being observed in accordance with certain principles, with the image having a balanced visual effect, a prominent theme, and a clear distinction between the primary and secondary. In travel photography images, this is manifested as a balance between the person and the background. This is mainly reflected in the ratio of the human body area to the area of ​​the entire image.

[0118] (e) Travel photography usually uses a shallow depth of field, that is, the visual subject (person) is highlighted by blurring the background, so that the person part can be clearly displayed. The invention determines the depth of field of the image by taking the difference between the clarity of the human body area and the overall image, and uses the Laplace operator to calculate the average gradient energy of the image as a measure of clarity.

[0119] (6) In order to make the scoring more accurate, the scoring rules of the scoring module use the probability distribution of the comprehensive score to score the image. The main method is as follows:

[0120] An example is as follows. For example, the classification probability of an image obtained through a classification model is as follows:

[0121] [0.1, 0.2, 0.25, 0.4, 0.05], and its final score is:

[0122] 3+(0.1*0+0.2*1+0.25*2+0.4*3+0.05*4) / 5=3.42

[0123] [0.05, 0.1, 0.1, 0.4, 0.35], and its final score is:

[0124] 3 + (0.05 * 0 + 0.1 * 1 + 0.1 * 2 + 0.4 * 3 + 0.35 * 4) / 5 = 3.58

[0125] [0.05, 0.05, 0.05, 0.8, 0.05], and its final score is:

[0126] 3 + (0.05 * 0 + 0.05 * 1 + 0.05 * 2 + 0.8 * 3 + 0.05 * 4) / 5 = 3.55.

[0127] The cover generation system based on travel photography albums of the present invention can automatically classify travel photography albums and provide targeted cover selections, reducing labor consumption and optimizing the user experience.

[0128] Figure 4 It is a schematic diagram of the modules of the cover generation system based on travel photography albums of the present invention. As Figure 4 shown, an embodiment of the present invention also provides a cover generation system based on travel photography albums for implementing the above-mentioned cover generation method based on travel photography albums. The cover generation system 5 based on travel photography albums includes:

[0129] An album creation module 51 that creates a travel photography album based on travel photos.

[0130] An image segmentation module 52 that performs image recognition on all travel photos of the travel photography album to obtain face regions.

[0131] A face recognition module 53 that performs face recognition on the face regions to obtain face information in each travel photo.

[0132] An album classification module 54 that sorts the appearance frequencies of each face information in the travel photography album from largest to smallest, and determines whether the travel photography album is a couple travel photography or a group travel photography based on the difference between the face information with the largest frequency, the face information with the second largest frequency, and the face information with the third largest frequency. And

[0133] A cover selection module 55 that selects only the photos including only the couple in the travel photos of the couple travel photography, and selects photos as the cover of the travel photography album based at least on the ratio of the pixels occupied by the two people in the photo to the total pixels of the photo. Alternatively, for the travel photos of the group travel photography, obtain only the photos including the faces of the top N people with the highest appearance frequencies, and select photos as the cover of the group travel photography based at least on the ratio of the pixels occupied by the top N people with the highest appearance frequencies in the photo to the total pixels of the photo, where N > 2.

[0134] In a preferred embodiment, the image segmentation module 52 is configured to perform image recognition on all travel photos of the travel photography album. Establish the face region, body region, and background region of each travel photo, and establish the corresponding relationship between the body region and the face region pair, but not limited thereto.

[0135] In a preferred embodiment, the face recognition module 53 is configured to perform face recognition on each face area of the travel photos to obtain the face information in each travel photo. The face information in the travel photo album is compared for similarity to establish the attribution relationship between the face information and the user, but not limited thereto.

[0136] In a preferred embodiment, the album classification module 54 is configured to sort the appearance frequencies of each face information in the travel photo album from largest to smallest, and determine whether the difference between the face information with the second highest frequency and the face information with the third highest frequency is greater than or equal to a second preset threshold. If so, the travel photo album is marked as a couple travel photo album; if not, the travel photo album is marked as a group travel photo album.

[0137] In a preferred embodiment, the album classification module 54 can also be configured to: sort the appearance frequencies of each face information in the travel photo album from largest to smallest, and determine whether the difference between the face information with the highest frequency and the face information with the second highest frequency is less than or equal to a first preset threshold, and whether the difference between the face information with the second highest frequency and the face information with the third highest frequency is greater than or equal to a second preset threshold. If so, step S142 is executed; if not, step S143 is executed, but not limited thereto.

[0138] In a preferred embodiment, the cover selection module 55 is configured to select only the photos including only the couple from the travel photos of the couple travel photo album as candidate covers to establish a first candidate cover set. Weighted statistics of clarity, foreground-background contrast, color saturation, and visual balance parameters are performed on each candidate cover in the first candidate cover set to obtain candidate parameters. The visual balance parameter is the ratio of the total pixel of the local images of the two people to the total pixel of the photo. The candidate cover with the highest candidate parameter is used as the cover of the couple travel photo album, but not limited thereto.

[0139] In a preferred embodiment, the cover selection module 55 is configured to select only the photos including only the top N people with the highest appearance frequencies from the travel photos of the group travel photo album as candidate covers to establish a second candidate cover set. Weighted statistics of clarity, foreground-background contrast, color saturation, and visual balance parameters are performed on each candidate cover in the second candidate cover set to obtain candidate parameters. The visual balance parameter is the ratio of the total pixel of the local images of the top N people with the highest appearance frequencies to the total pixel of the photo. The candidate cover with the highest candidate parameter is used as the cover of the couple travel photo album, but not limited thereto.

[0140] The embodiment of the present invention also provides a cover generation device based on a travel photo album, including a processor and a memory, in which executable instructions of the processor are stored. The processor is configured to execute the steps of the cover generation method based on the travel photo album by executing the executable instructions.

[0141] As shown above, the cover generation system of the present invention based on travel photo albums can automatically classify travel photo albums and provide targeted cover selections, reducing manual labor consumption and optimizing the user experience.

[0142] Those skilled in the art of the present technical field can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to herein as "circuitry", "module", or "platform".

[0143] Figure 5 is a schematic structural diagram of the cover generation device of the present invention based on travel photo albums. The following will refer to Figure 5 to describe the electronic device 600 according to this embodiment of the present invention. Figure 5 The electronic device 600 shown is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0144] As Figure 5 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0145] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription transfer processing method part of this specification. For example, the processing unit 610 can execute the steps as Figure 1 shown in

[0146] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0147] The storage unit 620 may also include a program / utilities 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0148] The bus 630 can represent one or more of several types of bus structures, including a memory unit bus or a memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of the various bus structures.

[0149] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0150] An embodiment of the present invention also provides a computer-readable storage medium for storing a program, and when the program is executed, it implements the steps of the cover generation method based on travel photo albums. In some possible implementation manners, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription transfer processing method part of this specification.

[0151] As shown above, the cover generation system based on travel photo albums of the present invention in this embodiment can automatically classify travel photo albums and provide targeted cover selections, reducing human consumption and optimizing the user experience.

[0152] Figure 6 is a schematic structural diagram of the computer-readable storage medium of the present invention. Refer to Figure 6 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described. It can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device.

[0153] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0154] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0155] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0156] In summary, the object of the present invention is to provide a method, system, device, and storage medium for generating a cover based on a travel photo album, which can automatically classify the travel photo album and provide targeted cover selection, reduce labor consumption, and optimize the user experience.

[0157] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for generating a cover based on a travel photo album, characterized in that, it includes the following steps: S110. Establish a travel photo album based on travel photos; S120. Perform image recognition on all travel photos in the travel photo album to obtain face regions; S130. Perform face recognition on the face regions to obtain face information in each travel photo; S140. Sort the occurrence frequencies of each face information in the travel photo album from largest to smallest, and determine whether the difference between the face information with the largest frequency and the face information with the second largest frequency is less than or equal to a first preset threshold, and whether the difference between the face information with the second largest frequency and the face information with the third largest frequency is greater than or equal to a second preset threshold; if so, mark the travel photo album as a couple travel photo album, if not, mark the travel photo album as a group travel photo album; and S150. Select only the photos including only the couple from the travel photos of the couple travel photo album, and select photos as the cover of the travel photo album at least based on the ratio of the pixels occupied by the two people in the photo to the total pixels of the photo; or, obtain the photos including only the faces of the top N in terms of occurrence frequency from the travel photos of the group travel photo album, and select photos as the cover of the group travel photo album at least based on the ratio of the pixels occupied by the top N people in terms of occurrence frequency in the photo to the total pixels of the photo, N > 2.

2. The method for generating a cover based on a travel photo album according to claim 1, characterized in that, the step S120 includes: S121. Perform image recognition on all travel photos in the travel photo album; S122. Establish the face region, body region and background region of each travel photo, and establish the corresponding relationship between the body region and the face region pair.

3. The method for generating a cover based on a travel photo album according to claim 1, characterized in that, the step S130 includes: S131. Perform face recognition on each face region of the travel photo to obtain face information in each travel photo; S132. Perform approximation comparison on the face information in the travel photo album, and establish the attribution relationship between the face information and the user.

4. The method for generating a cover based on a travel photo album according to claim 1, characterized in that, the step S150 includes: S151. Select only the photos including only the couple from the travel photos of the couple travel photo album as candidate covers to establish a first candidate cover set; S152. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters of each candidate cover in the first candidate cover set to obtain candidate parameters, and the visual balance parameter is the ratio of the total pixels of the local images of the two people to the total pixels of the photo; and S153. Select the candidate cover with the highest candidate parameter as the cover of the couple travel photo album.

5. The method for generating a cover based on a travel photo album according to claim 1, characterized in that, the step S150 includes: S156. Select the photos that only include the people with the top N frequencies of appearance in the travel photoshoot photos of the group as candidate covers to establish a second candidate cover set; S157. Perform weighted statistics on the clarity, foreground-background contrast, color saturation, and visual balance parameters for each candidate cover in the second candidate cover set to obtain candidate parameters, where the visual balance parameter is the ratio of the total pixels of the local images of the people with the top N frequencies of appearance to the total pixels of the photo; and S158. Select the candidate cover with the highest candidate parameter as the cover of the group travel photoshoot.

6. A cover generation system based on a travel photo album, used to implement the cover generation method based on a travel photo album described in claim 1, Characterized in that it includes: An album establishment module that establishes a travel photo album based on travel photos; An image segmentation module that performs image recognition on all travel photos in the travel photo album to obtain face regions; A face recognition module that performs face recognition on the face regions to obtain face information in each travel photo; An album classification module that sorts the frequencies of appearance of each piece of face information in the travel photo album from largest to smallest, and determines whether the difference between the face information with the largest frequency and the face information with the second largest frequency is less than or equal to a first preset threshold, and whether the difference between the face information with the second largest frequency and the face information with the third largest frequency is greater than or equal to a second preset threshold; if so, mark the travel photo album as a couple travel photoshoot, if not, mark the travel photo album as a group travel photoshoot; and A cover selection module that selects the photos that only include the two people in the couple from the travel photoshoot photos of the couple travel photoshoot, and selects photos as the cover of the travel photo album at least based on the ratio of the pixels occupied by the two people in the photo to the total pixels of the photo; or, obtains the photos that only include the faces of the top N people with the highest frequencies of appearance from the travel photoshoot photos of the group travel photoshoot, and selects photos as the cover of the group travel photoshoot at least based on the ratio of the pixels occupied by the top N people with the highest frequencies of appearance in the photo to the total pixels of the photo, where N > 2.

7. A cover generation device based on a travel photo album, Characterized in that it includes: A processor; A memory that stores executable instructions of the processor; wherein, the processor is configured to execute the steps of the cover generation method based on a travel photo album described in any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium for storing a program, Characterized in that when the program is executed by a processor, it implements the steps of the cover generation method based on a travel photo album described in any one of claims 1 to 5.

Images