Method, device and equipment for generating electronic photo album

Abstract

The present application discloses a method, device, and apparatus for generating an electronic photo album, which is used to solve the problem of time-consuming generation of existing electronic photo albums. Since the present application can obtain a set number of target photos that match the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the requirements one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style. Any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of generating electronic photo albums to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method, device, equipment and medium for generating an electronic photo album. Background Art

[0002] With the development of the times, users' demands for electronic photo albums are becoming more and more diversified and personalized. Some users like to make personal electronic photo albums, requiring that each photo in the album has a different self; while some photography enthusiasts like to make electronic photo albums of beautiful scenery they have taken, with dynamic background music, so that they can indulge themselves in the beauty of nature.

[0003] At present, the method of generating an electronic photo album mainly includes: the user first selects each photo and background music according to his or her own needs; then the electronic photo album generation device performs content analysis on each photo, generates descriptive information of the photo, and then synthesizes the photo and the descriptive information into a video, and finally generates an electronic photo album based on the synthesized video and the background music.

[0004] For this method, when a user wants to generate an electronic photo album based on many photos, the user needs to spend a lot of time to select photos that meet the requirements and background music, making the entire electronic photo album generation process very dependent on manpower, reducing user experience. Summary of the Invention

[0005] The present application provides a method, apparatus, device and medium for generating an electronic photo album, so as to solve the problem that the generation process of the existing electronic photo album is time-consuming.

[0006] In a first aspect, the present application further provides a method for generating an electronic photo album, the method comprising:

[0007] According to at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos matching the target tags are obtained, wherein the tags are determined based on the content contained in the photos;

[0008] Determining the duration of the background music according to the set number and the preset display duration of a single photo;

[0009] Determine the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style, and determine any background music that meets the duration value in each target background music style as the target background music;

[0010] An electronic photo album is generated according to the target background music and each of the target photos.

[0011] In a second aspect, the present application further provides a device for generating an electronic photo album, the device comprising:

[0012] an acquisition unit, configured to acquire, based on at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos that match the target tag, wherein the tag is determined based on the content contained in the photo;

[0013] A first determining unit is configured to determine a duration of the background music according to the set number and a preset display duration of a single photo;

[0014] A second determining unit is configured to determine a target background music style corresponding to each target tag according to a preset correspondence between the tag and the music style, and determine any background music in each target background music style that meets the duration value as the target background music;

[0015] A processing unit is used to generate an electronic photo album according to the target background music and each target photo.

[0016] In a third aspect, the present application further provides an electronic device, which includes at least a processor and a memory, and the processor is configured to implement the steps of any of the above-described methods for generating an electronic photo album when executing a computer program stored in the memory.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for generating an electronic photo album.

[0018] Since the present application can obtain a set number of target photos matching the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the needs one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style, and any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of electronic album generation to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a process for generating an electronic photo album provided in some embodiments of the present application;

[0021] Figure 2 A schematic diagram of the types of objects contained in a photograph provided in some embodiments of the present application;

[0022] Figure 3 Schematic diagram of the process of determining the target style of a photo using the ResNet network model provided in some embodiments of the present application;

[0023] Figure 4 A schematic diagram of a label determination process provided in some embodiments of the present application;

[0024] Figure 5 A schematic diagram of a specific electronic photo album generation process provided in some embodiments of the present application;

[0025] Figure 6 A schematic diagram of a specific electronic photo album generation process provided in some embodiments of the present application;

[0026] Figure 7 A schematic diagram of the structure of a device for generating an electronic photo album provided in some embodiments of the present application;

[0027] Figure 8 A schematic diagram of the structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0028] In order to improve the efficiency of generating electronic photo albums and enhance user experience, the present application provides a method, apparatus, device and medium for generating electronic photo albums.

[0029] To make the purpose, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of this application without making any creative efforts are within the scope of protection of this application.

[0030] In actual application, when a user wants to make an electronic photo album from the acquired photos, the electronic device receives the user's demand for generating an electronic photo album and generates an album generation instruction carrying at least one target tag. The electronic device can be a smart phone, a smart housekeeper server, a computer and other devices. The electronic device determines each target photo that meets the user's needs based on at least one tag corresponding to each pre-saved photo, and determines the background music based on the correspondence between the preset tags and the music style, thereby generating an electronic photo album.

[0031] Figure 1 A schematic diagram of a process for generating an electronic photo album provided in some embodiments of the present application, the process including:

[0032] S101: According to at least one pre-saved tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos matching the target tag are obtained, wherein the tag is determined according to the content contained in the photo.

[0033] The method for generating an electronic photo album provided in this application is applied to an electronic device, which may be a smart device such as a smartphone, a computer, or a smart housekeeper server.

[0034] In the present application, after the electronic device receives an album generation instruction carrying at least one target tag, it performs corresponding processing on each pre-saved photo based on the at least one target tag carried in the album generation instruction, thereby generating an electronic album.

[0035] The album generation instruction received by the electronic device may be an instruction generated after receiving a trigger operation for generating its own album, or may be an album generation instruction received from another smart device.

[0036] In the present application, photos are pre-stored in the electronic device, and each photo corresponds to at least one tag. When the electronic device receives an album generation instruction, it parses the album generation instruction and obtains at least one target tag carried in the album generation instruction. Based on the at least one tag corresponding to each pre-stored photo and the at least one target tag carried in the album generation instruction, each candidate photo that matches each target tag is determined, and then a set number of candidate photos are randomly selected as target photos based on a pre-set number and each candidate photo. The tag corresponding to the photo is determined based on the content contained in the photo. For example, if a photo contains two animals, a cat and a dog, and each animal has a corresponding tag, then the photo corresponds to two tags, a cat and a dog.

[0037] Furthermore, when determining candidate photos, a photo must contain at least one tag and at least one target tag included in the album generation instruction. If a photo contains a tag that matches any of the target tags, the photo is considered a candidate photo. When selecting a set number of target photos from each candidate photo, the selection can be based on the number of tags matching the target tags within the candidate photo. Preferably, a set number of candidate photos containing tags matching each target tag are randomly selected as target photos.

[0038] For example, the electronic device pre-stores photo A with the tags cat, dog, person, and flower; photo B with the tags flower, person, and tree; photo C with the tags bicycle, house, and flower; photo D with the tags fish, lotus leaf, flower, and person; and photo E with the tags person, bicycle, and tree. After receiving the album generation instruction, the electronic device obtains the target tags person and flower carried in the album instruction. Then, each candidate photo matching any of the target tags "person" and "flower" is obtained as photo A, photo B, photo C, photo D, and photo E. Based on a pre-set number 2, a set number of candidate photos containing tags matching each target tag, such as photo A and photo B, are randomly selected from the five candidate photos as target photos.

[0039] Taking the above example, the set number is 4, and the determined number of candidate photos is 3, which is less than the set number 4. Then, candidate photos matching any target label "flower" or "person" are obtained, that is, the five candidate photos of photo A, photo B, photo C, photo D, and photo E. From these five candidate photos, photos containing a larger number of photos matching the target label, such as photo A, photo B, photo C, and photo D, are selected as target photos.

[0040] S102: Determine the duration of the background music according to the set number and the preset display duration of a single photo.

[0041] To facilitate the determination of background music, this application pre-sets the display duration of each photo, such as 3 seconds, 5 seconds, etc., as well as the correspondence between labels and music styles, such as the music style corresponding to food is cheerful, the music style corresponding to people is soothing, etc. The duration of the background music can be determined based on the set number and the preset display duration of each photo.

[0042] S103: Determine the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style, and determine any background music that meets the duration value in each target background music style as the target background music.

[0043] In order to accurately determine the background music, in this application, a correspondence between tags and music styles is pre-set. After the duration value of the background music is obtained based on the above embodiment, the target background music style corresponding to each target tag is determined based on the preset correspondence between tags and music styles.

[0044] In order to accurately determine the target background music, in this application, the playback duration value of each background music, as well as each background music and its corresponding music style, are pre-determined and saved, wherein each background music corresponds to at least one music style. After obtaining the duration value and each target background music style based on the above embodiment, each candidate background music that meets the duration value is determined from the pre-saved background music. Among each candidate background music, a candidate background music whose corresponding music style matches any target background music style is randomly selected as the target background music. Among them, the satisfied duration value here means that the difference between the playback duration value and the duration value is within a preset value range.

[0045] In addition, when selecting the target background music from each candidate background music, the target background music can be selected based on the number of music styles that match the target background music style in the candidate background music. Preferably, a candidate background music that has a music style that matches each target background music style is randomly selected as the target background music.

[0046] For example, the duration value is 1 minute and 50 seconds, and the target background music style is determined to be soothing and pastoral. Then, among the pre-saved background music, search for each candidate background music whose difference between any playback duration value and the duration value is within a certain value range. The music style corresponding to candidate background music p is pastoral and cheerful, the music style corresponding to candidate background music y is soothing, and the music style corresponding to candidate background music z is pastoral and soothing. Among each candidate background music, randomly select a candidate background music whose corresponding music style matches any target background music style, such as candidate background music p, candidate music y, and candidate music z, as the target background music. Preferably, candidate music z is used as the target background music.

[0047] Of course, in order to improve the user experience, each candidate background music that matches the determined corresponding music style with any target background music style may be recommended to the user, so that the user can select a target background music according to his or her needs.

[0048] S104: Generate an electronic photo album according to the target background music and each of the target photos.

[0049] According to each target photo and target background music determined in the above embodiment, an electronic photo album is generated. The specific process of generating an electronic photo album based on photos and music belongs to the existing technology and will not be described in detail here.

[0050] Since the present application can obtain a set number of target photos matching the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the needs one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style, and any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of electronic album generation to a certain extent.

[0051] In order to accurately determine the label corresponding to the photo, based on the above embodiment, in this application, the label is obtained in the following way:

[0052] Determine the type of object contained in the received photo through an object detection algorithm;

[0053] Determining first tags corresponding to the photos according to types of objects contained in the photos;

[0054] If the type of object included in the photo is a person, when the face detection algorithm is used to determine that the photo contains a face, the face feature vector in the photo is obtained through the face feature extraction algorithm, and when the face feature vector meets the preset conditions, the second label corresponding to the photo is determined.

[0055] In the present application, the electronic device can process the received photos accordingly, determine the tags corresponding to the photos, and save the photos and their corresponding tags accordingly.

[0056] Specifically, an object detection algorithm, such as the YoloV3 algorithm, is used to identify the types of objects contained in the received photo, and the types of all objects in the preset object types contained in the photo are determined, for example, the types of cats, dogs, horses, people, etc. Then, a first label corresponding to the photo is determined based on each type of object contained in the photo.

[0057] For example, the types of objects contained in a certain photo are person, tree, and flower, and the first tags corresponding to the photo are determined to be person, tree, and flower respectively.

[0058] Since users may wish to generate an electronic photo album based on photos of people with specific identity identifiers, in this application, the second tag corresponding to the photo can also be determined based on the identity identifier of the person contained in the photo, so that each photo of a person containing a certain identity identifier can be determined based on the second tag, thereby generating an electronic photo album. Specifically, when determining the identity identifier of the person contained in the photo, it is generally recognized by face. When, based on the above embodiment, the first tag corresponding to a photo is obtained, it is determined that the type of object contained in the photo includes a person, then face detection is performed on the photo using a face detection algorithm, such as the FaceBoxes algorithm. If it is determined that the photo contains a face, it means that the identity identifier of the person contained in the photo can be identified, and then a face feature extraction algorithm is used to obtain a face feature vector in the photo. The face feature vector is further processed to determine whether the photo has a corresponding second tag.

[0059] In actual application, photos may contain faces of strangers or passers-by, and users generally do not pay attention to the faces of strangers or passers-by. Therefore, in this application, for photos containing faces, after obtaining the facial feature vector in the photo, the facial feature vector must be further processed to determine whether the facial feature vector in the photo meets the preset conditions, so as to determine whether the face in the photo is the face of a person with an identity identifier that the user has preset to be more concerned about, and when it is determined that the facial feature vector meets the preset conditions, the second label corresponding to the photo is determined.

[0060] Specifically, when the facial feature vector satisfies a preset condition, determining the second label corresponding to the photo includes:

[0061] Obtaining the similarity between the facial feature vector and any pre-stored registered facial feature vector;

[0062] If the maximum similarity value is greater than a set first threshold, it is determined that the facial feature vector meets a preset condition, and the second label corresponding to the photo is determined based on the identity identifier corresponding to the registered facial feature vector to which the maximum similarity value belongs.

[0063] In order to accurately determine whether the face in the received photo is the face of the person with the identity identifier that the user is concerned about, in this application, the user pre-labels the photo containing the face of the person with the identity identifier that the user is concerned about, and labels the identity identifier corresponding to the face in the photo. The electronic device obtains the facial feature vector of the face of the person with the identity identifier that is being concerned about in the photo through a facial feature extraction algorithm, uses the facial feature vector as the registered facial feature vector, and saves the registered facial feature vector and the corresponding identity identifier accordingly.

[0064] After the electronic device subsequently obtains the facial feature vector of the face included in the photo, it obtains the similarity between the facial feature vector and any pre-saved registered facial feature vector. The maximum value of the similarity is compared with the set first threshold value to determine whether the facial feature vector meets the preset conditions. Specifically, if the maximum similarity value is greater than the set first threshold value, it is determined that the facial feature vector meets the preset conditions, and the second label corresponding to the photo is determined based on the identity identifier corresponding to the registered facial feature vector to which the maximum similarity value belongs. For example, if the identity identifier corresponding to the registered facial feature vector to which the maximum similarity value belongs is "Mom", the second label corresponding to the photo is determined to be "Mom". If the maximum similarity value is not greater than the set first threshold value, it means that the face corresponding to the facial feature vector may be the face of a person that the user does not pay attention to, such as a passerby, stranger, etc., then it is determined that the facial feature vector does not meet the preset conditions and the photo does not have a corresponding second label.

[0065] For example, the first threshold is set to 0.5, and the similarities between the facial feature vector of a face in a certain photo and the registered facial feature vectors a, b, and c are 0.3, 0.1, and 0.6 respectively. It is determined that the maximum similarity value of 0.6 is greater than the set first threshold value of 0.5, indicating that the facial feature vector meets the preset conditions. Then, according to the identity identifier "Li Er" corresponding to the registered facial feature vector c to which the maximum similarity value 0.6 belongs, the second label corresponding to the photo is determined to be "Li Er".

[0066] In order to further accurately determine the label corresponding to each photo, based on the above embodiments, in this application, the label is obtained in the following way:

[0067] Using the pre-trained classification model, we can obtain the probability that the received photo is of each style.

[0068] determining a target style of the photo according to each of the probabilities;

[0069] A third label of the photo is determined according to the target style.

[0070] Since a photo may contain objects such as people, cats, and houses at the same time, but the proportion of each type of object in the photo is different, for example, Figure 2 A schematic diagram of the type of objects contained in a photograph provided in some embodiments of the present application, such as Figure 2 As shown, the person is in the upper left corner of the picture, occupying 10% of the entire picture area, and the cat is in the center of the picture, occupying 80% of the entire picture area. If the style of the photo is determined directly based on the type of objects contained in the photo, the determined style of the photo may be very inaccurate.

[0071] Therefore, in this application, a classification model, such as a ResNet network model, is pre-trained. Through the trained classification model, the probability of the received photo being of each preset style is obtained, such as the probability of the photo being of a person, an animal, a beautiful view, a delicious meal, a building, text, or other preset styles. Based on each probability obtained, the target style to which the photo belongs is determined. For example, the style corresponding to the largest set of probability values is determined as the target style of the photo. Based on each target style, the third label of the photo is determined.

[0072] Specifically, taking the ResNet network model as an example of a classification model, the ResNet network model is a common basic backbone network model due to its advantages such as many layers, few parameters, and easy training. On the basis of ordinary convolutional neural networks, it directly connects the input and output of a convolutional layer, and constructs a network model by fitting the residuals of the input and output, thereby solving problems such as information loss and vanishing backpropagation gradients.

[0073] Figure 3 Schematic diagram of the process of determining the target style of a photo by the ResNet network model provided in some embodiments of the present application. Figure 3 First, the input photo is preprocessed and scaled to a set size, such as 224*224. The pixel values of each pixel in the photo are then normalized and input into the feature extraction layer of the ResNet network model for feature extraction. Logistic regression is performed on the final abstract feature matrix output by the feature extraction layer through the output layer of the ResNet network model to determine the probability of the photo belonging to each style, such as 61% for people, 23% for animals, 86% for scenery, 12% for food, 7% for architecture, 3% for text, and 31% for other. Based on each probability, corresponding processing is performed to determine the target style of the photo. Based on the target style of the photo, the third label of the photo is determined to be scenery and people, respectively.

[0074] Each element value in each column of the final abstract feature matrix represents the extracted feature value of the corresponding style of the photo. For example, in the 10*7 final abstract feature matrix, the 7 element values contained in the first column represent each feature value extracted for the food style of the photo, and the 7 element values contained in the second column represent each feature value extracted for the animal style of the photo. The output layer of the ResNet network model pre-determines a corresponding weight value for each feature value of each style. The output layer uses the weight matrix determined by the weight values corresponding to each feature value of each style and the final abstract feature matrix to determine the probability of the photo belonging to each style through a logistic regression formula.

[0075] Specifically, the probability that photo i is of each style is determined by the following formula:

[0076]

[0077] Among them, Pi represents the vector composed of the probability of photo i being of each style, W T represents the transpose of the weight matrix, and X represents the final abstract feature matrix.

[0078] Furthermore, when the probability of a photo belonging to each style is obtained based on the above embodiment, each probability is further processed to determine the target style to which the photo belongs. Specifically, determining the target photo type of the photo based on each probability includes:

[0079] If there is a probability greater than the set second threshold, the style corresponding to the probability greater than the set second threshold is determined as the target style of the photo; otherwise, the target style of the photo is determined to be the preset style.

[0080] To further accurately determine the target style of a photo, this application pre-sets a second threshold. After obtaining the probability of a photo being of each style based on the above embodiment, each probability is compared with the set second threshold to determine the target style of the photo. If a probability exists that is greater than the set second threshold, the style corresponding to each probability greater than the set second threshold is determined as the target style of the photo.

[0081] For example, the probability of a certain photo being of each style is 61% for people, 23% for animals, 86% for scenery, 12% for food, 7% for architecture, 3% for text, and 31% for others. The second threshold is set to 0.5, and the probabilities of 61% and 86% are greater than the set second threshold. The styles of people and scenery corresponding to the probabilities of 61% and 86% are determined as the target styles of the photo.

[0082] In addition, this application also pre-sets a preset style, such as "Other," for a photo when there is no probability greater than a set second threshold. When the pre-trained classification model is subsequently used to obtain the probability of a photo being in each style, and it is determined that there is no probability greater than the set second threshold, the target style corresponding to the photo is determined to be the preset style.

[0083] For example, the preset style is "Others", and the probability of a photo being of each style is 41% for people, 23% for animals, 36% for scenery, 12% for food, 7% for architecture, 3% for text, and 31% for others. If the second threshold is set to 0.5, it is determined that there is no probability greater than the set second threshold, and the target style of the photo is determined to be "Others".

[0084] In another possible implementation, in order to save time spent on determining tags, in the present application, the electronic device can simultaneously determine the first tag, the second tag, and the third tag when determining the tag corresponding to the received photo. Figure 4 A schematic diagram of a label determination process provided in some embodiments of the present application is shown in FIG. Figure 4 For the received photos, the electronic device uses object detection algorithms, face detection algorithms, and pre-trained classification models to obtain the type of objects contained in the photo, whether it contains faces, and the probability corresponding to each style.

[0085] A first label corresponding to the photo is determined according to the type of the object contained in the photo.

[0086] According to the detection results of the face detection algorithm, if the photo contains a face, the face feature vector in the photo is obtained through the face feature extraction algorithm, and the similarity between the face feature vector and any pre-saved registered face feature vector is determined. If the maximum similarity is greater than a set first threshold, it is determined that the face feature vector meets the preset conditions, and the second label corresponding to the photo is determined based on the identity identifier corresponding to the registered face feature vector to which the maximum similarity belongs.

[0087] Based on the probability of the photo belonging to each style output by the classification model, if any probability exceeds a set second threshold, the style corresponding to each probability exceeding the second threshold is determined as the target style of the photo. If no probability exceeds the set second threshold, the target style corresponding to the photo is determined to be the preset style. Based on the target style of the photo, a third label for the photo is determined.

[0088] In order to accurately obtain the probability of a photo belonging to each style, the classification model is trained as follows:

[0089] Get any image sample in the sample set and each corresponding style;

[0090] The original classification model is trained based on the image samples and each corresponding style.

[0091] To accurately determine the probability of a photo belonging to each style, the original classification model can be trained based on each image sample and its corresponding style in the pre-acquired sample set. For example, image sample 1 corresponds to the styles of people and animals, while image sample 2 corresponds to the styles of architecture and scenery. To increase the diversity of the image samples, the image samples include images from different angles and sizes.

[0092] It should be noted that the device for training the classification model may be the same as or different from the electronic device for subsequently generating the electronic photo album.

[0093] Through the original classification model, each recognition style corresponding to the above-mentioned picture sample can be identified. According to the recognition style and each style corresponding to the picture sample, the original classification model is trained to adjust the parameter values of each parameter of the original classification model.

[0094] For example, the style corresponding to the image sample is beautiful scenery and architecture. Through the original classification model, the recognition style corresponding to the image sample is identified as animals. The recognition style is inconsistent with the corresponding style, and it is determined that the style of the image sample is misidentified by the original classification model.

[0095] The sample set for training the original classification model contains a large number of image samples. The above operation is performed on each image sample. When the preset convergence condition is met, the training of the original classification model is completed.

[0096] The preset convergence condition can be satisfied when the number of image samples in the sample set that are correctly identified by the original classification model exceeds a set number, or when the number of iterations of training the original classification model reaches a set maximum number of iterations. This can be flexibly set in specific implementations and is not specifically limited here.

[0097] In one possible implementation, when training the original classification model, the image samples in the sample set can be divided into training image samples and test image samples. The original classification model is first trained based on the training image samples, and then the reliability of the trained classification model is verified based on the test image samples.

[0098] To further accurately determine the target photos, based on the above embodiments, in this application, if the album generation instruction also carries a target location, a target number, and a target time period, obtaining a set number of target photos that match the target tag based on at least one pre-stored tag corresponding to each photo and at least one target tag carried in the album generation instruction includes:

[0099] updating the set quantity according to the target quantity;

[0100] According to the collection location, collection time and at least one tag corresponding to each pre-saved photo, and the target location, the target time period and the target tag carried in the album generation instruction, the target number of target photos that match the target tag, the target location and the target time period are obtained.

[0101] In this application, the electronic device obtains the collection location and collection time carried in the received photo based on the photo, and saves each photo and its corresponding collection location and collection time, so that the subsequent electronic device can determine each target photo that meets the user's needs based on the target time period, target location and at least one target tag carried in the received album generation instruction.

[0102] For example, each photo received by an electronic device is collected and sent by a smart terminal. When the smart terminal collects a photo, it will record the GPS (Global Positioning System) location information of the photo and the collection time by default, and save it to the Exif file of the photo. After the electronic device receives the photo, it determines the collection location corresponding to the photo by reading the GPS location information in the Exif file of the photo, and reads the collection time saved in the Exif file. Based on the collection location and collection time of the photo obtained, the photo and its corresponding collection location and collection time are saved accordingly.

[0103] In some embodiments, when the album generation instruction carries a target location, a target time period, and at least one target tag, the electronic device obtains a set number of target photos that match the target tag, target location, and target time period based on the pre-saved collection location, collection time, and at least one tag corresponding to each photo, as well as the target location, target time period, and at least one target tag carried in the album generation instruction; determines the duration value of the background music based on the set number and the preset display duration of a single photo; determines each target background music style based on the correspondence between the preset tag and the music style, and each target tag, and determines any pre-saved background music that meets the duration value and each target background music style as the target background music; and generates an electronic album based on the target background music and each target photo.

[0104] Because users may wish to generate electronic photo albums with different numbers of target photos, the album generation instruction may include a target number to further enhance the user experience. Upon receiving the target number in the album generation instruction, the electronic device updates a pre-stored set number based on the target number, thereby determining the target number of target photos.

[0105] In order to further accurately determine the target background music style, the determining of the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style includes:

[0106] The target background music style corresponding to each target tag is determined according to the corresponding relationship between the preset time period and the music style, and the corresponding relationship between the preset tag and the music style.

[0107] Generally speaking, the user's emotions reflected by different target time periods will also be different. For example, if a user wants to generate an electronic photo album based on photos collected during college, the user's emotions reflected may be more inclined towards a nostalgic music style; if a user wants to generate an electronic photo album based on photos collected within a day, the user's emotions reflected may be more inclined towards a cheerful music style. Based on this, in order to further accurately determine the target background music style, in this application, a corresponding music style is pre-set for each time period. For example, the music style corresponding to a time period less than a day is an inspiring style, and the music style corresponding to a time period greater than a year is a nostalgic style, or the music style corresponding to a time period including Christmas is a Christmas style, and the music style corresponding to a time period including New Year's Day is a New Year style, etc. When the subsequent processing module 101 determines the target background music style, it can determine the target background music style corresponding to each target tag based on the preset correspondence between time periods and music styles, and the preset correspondence between tags and music styles, so as to more accurately meet the user's needs.

[0108] Figure 5 This is a schematic diagram of a specific electronic photo album generation process provided in some embodiments of the present application. The process includes three parts: training a classification model, determining labels, and generating an electronic photo album. The execution entity takes the smart housekeeper server as an example. Each part is described below:

[0109] Part 1: Training of the classification model, including the following steps:

[0110] S501: The first smart housekeeper server obtains any image sample in the sample set and each corresponding style.

[0111] S502: Training the original classification model based on the image samples and each corresponding style.

[0112] In the process of training the classification model, an offline method is generally adopted, and the image samples in the sample set are trained in advance through the first smart housekeeper server to obtain a trained classification model.

[0113] The second part is to determine the label. Based on the classification model trained by the first smart housekeeper server, the trained classification model is saved to the second smart housekeeper server, and the third label is determined by the second smart housekeeper server. The specific implementation includes the following steps:

[0114] S503: The second smart housekeeper server obtains the received photos and their corresponding collection time and collection location.

[0115] S504: The second smart housekeeper server uses an object detection algorithm to determine the types of objects contained in the photo.

[0116] S505: The second smart housekeeper server determines a first tag corresponding to the photo according to the type of the object contained in the photo.

[0117] S506: The second smart housekeeper server determines whether the type of object included in the photo is a person. If so, execute S507; otherwise, execute S511.

[0118] S507: The second smart housekeeper server uses a face detection algorithm to determine whether the photo contains a face. If so, execute S508; otherwise, execute S511.

[0119] S508: The second smart housekeeper server obtains the facial feature vector in the photo through a facial feature extraction algorithm.

[0120] S509: The second smart housekeeper server obtains the similarity between the facial feature vector and any pre-saved registered facial feature vector, and determines whether the maximum similarity is greater than a set first threshold. If so, execute S510; otherwise, execute S511.

[0121] S510: The second smart housekeeper server determines that the facial feature vector meets the preset conditions, and determines the second tag corresponding to the photo based on the identity identifier corresponding to the registered facial feature vector to which the maximum similarity value belongs.

[0122] S511: The second smart housekeeper server obtains the probability of the photo being of each style through the pre-trained classification model, and determines whether there is a probability greater than the set second threshold. If so, execute S512; otherwise, execute S513.

[0123] S512: The second intelligent housekeeper server determines the style corresponding to each probability greater than the set second threshold as the target style of the photo, and then executes S514.

[0124] S513: The second smart housekeeper server determines that the target style of the photo is a preset style.

[0125] S514: The second smart housekeeper server determines a third tag for the photo based on the target style of the photo.

[0126] Of course, the determination of the label can also be completed in the first smart housekeeper server. In this embodiment, the execution entity of the determination of the label is not limited.

[0127] The third part is the generation of the electronic photo album, which includes the following steps:

[0128] S515: The second smart housekeeper server receives an album generation instruction, which carries a target location, a target time period, a target quantity, and a target tag.

[0129] S516: The second smart housekeeper server updates the set quantity according to the target quantity.

[0130] S517: The second smart housekeeper server obtains a target number of target photos that match the target tag, target location and target time period based on the collection location, collection time and at least one tag corresponding to each pre-saved photo, as well as the target location, target time period and target tag carried in the album generation instruction.

[0131] S518: The second smart housekeeper server determines the duration of the background music based on the updated set quantity and the preset display duration of a single photo.

[0132] S519: The second smart housekeeper server determines the target background music style corresponding to each target tag based on the correspondence between the preset time period and the music style, and the correspondence between the preset tag and the music style.

[0133] S520: The second intelligent butler server determines any background music that meets the duration value in each target background music style as the target background music.

[0134] S521: The second smart housekeeper server generates an electronic photo album based on the target background music and each target photo.

[0135] Figure 6 A schematic diagram of a specific electronic photo album generation process provided in some embodiments of the present application, taking an electronic device receiving a photo album generation instruction and photos sent by another smart device as an example, the process includes:

[0136] S601: The electronic device receives photos sent by other smart devices.

[0137] Other smart devices are smart devices that have established communication with the electronic device in advance. They can send photos collected by themselves or photos sent by other smart devices to the electronic device. Specifically, other smart devices can send all received photos to the electronic device, or send some photos to the electronic device. Of course, other smart devices can also send a photo to the electronic device as soon as they receive it.

[0138] S602: The electronic device determines a tag corresponding to the photo.

[0139] Specifically, the process of the electronic device determining the tag has been described in the above embodiment and will not be repeated here.

[0140] S603: The electronic device obtains the collection time and collection location corresponding to the photo.

[0141] S604: The electronic device receives an album generation instruction sent by other smart devices.

[0142] Specifically, other smart devices are generally equipped with display screens that can receive trigger operations from users on the display interface, thereby generating corresponding instructions and sending them to electronic devices. Taking a smartphone as an example, an APP for generating electronic photo albums is installed on the smartphone. When the smartphone detects that the user has logged into the APP and selects the button for generating electronic photo albums on the display interface, a selection bar corresponding to the label, location, time period, and quantity will be displayed on the display interface. Through this selection bar, the user can determine the target location, target time period, target quantity, and at least one target label that meet their needs. When the user confirms that the selection is complete and clicks the corresponding submit button, the smartphone can generate a corresponding album generation instruction based on the target location, target time period, target quantity, and at least one target label selected on the display interface, and send the album generation instruction to the electronic device so that the electronic device can generate an electronic photo album based on the information contained in the album generation instruction.

[0143] Among them, the selection bar corresponding to the label on the display interface displays each label that can be selected. The label can be displayed in the form of text or in the form of a small icon. The user selects the label displayed on the display interface to determine the target label. The selection bar corresponding to the time period on the display interface displays two slidable time rollers or a time ruler. The user adjusts the time roller or time ruler on the display interface to determine the target time period. The selection bar corresponding to the location on the display interface displays each location that can be selected. The location can be displayed in the form of text or in the form of a small icon. The user selects the location displayed on the display interface to determine the target location. The selection bar corresponding to the quantity on the display interface displays the number of electronic photo albums that can be synthesized. The user can fill in the target number in the text input box corresponding to the number, or determine the target number by sliding the quantity ruler displayed on the display interface.

[0144] S605: The electronic device generates an electronic photo album.

[0145] The electronic device determines target photos and target background music that meet the user's needs based on the information contained in the above-mentioned album generation instruction, and generates an electronic photo album based on the target photos and target background music.

[0146] Specifically, the process of determining the target photo and the target background music has been described in the above embodiment and will not be repeated here.

[0147] This application also provides a device for generating an electronic photo album. Figure 7 This is a schematic diagram of a device for generating an electronic photo album provided in some embodiments of the present application, the device comprising:

[0148] An acquisition unit 71 is configured to acquire a set number of target photos that match the target tag based on at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, wherein the tag is determined based on the content contained in the photo;

[0149] A first determining unit 72 is configured to determine a duration of the background music according to the set number and a preset display duration of a single photo;

[0150] The second determining unit 73 is configured to determine the target background music style corresponding to each target tag according to a preset correspondence between the tag and the music style, and determine any background music in each target background music style that meets the duration value as the target background music;

[0151] The processing unit 74 is configured to generate an electronic photo album according to the target background music and each of the target photos.

[0152] For the concepts, explanations, detailed descriptions and other steps related to the technical solutions provided by the embodiments of the present invention involved in the electronic photo album generation device, please refer to the descriptions of these contents in the aforementioned method or other embodiments, which will not be repeated here.

[0153] Since the present application can obtain a set number of target photos matching the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the needs one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style, and any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of electronic album generation to a certain extent.

[0154] like Figure 8 This is a schematic diagram of an electronic device structure provided by some embodiments of the present application. Based on the above embodiments, the present application also provides an electronic device, such as Figure 8 As shown, it includes: a processor 81, a communication interface 82, a memory 83 and a communication bus 84, wherein the processor 81, the communication interface 82, and the memory 83 communicate with each other through the communication bus 84;

[0155] The memory 83 stores a computer program. When the program is executed by the processor 81, the processor 81 performs the following steps:

[0156] According to at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos matching the target tags are obtained, wherein the tags are determined based on the content contained in the photos;

[0157] Determining the duration of the background music according to the set number and the preset display duration of a single photo;

[0158] Determine the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style, and determine any background music that meets the duration value in each target background music style as the target background music;

[0159] An electronic photo album is generated according to the target background music and each of the target photos.

[0160] Since the principle of solving the problem by the above electronic device is similar to the method for generating an electronic photo album, the implementation of the above electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0161] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0162] The communication interface 82 is used for communication between the electronic device and other devices.

[0163] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk memory. Alternatively, the memory may be at least one storage device located away from the processor.

[0164] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0165] Since the present application can obtain a set number of target photos matching the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the needs one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style, and any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of electronic album generation to a certain extent.

[0166] Based on the above embodiments, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by a processor. When the program is executed on the processor, the processor implements the following steps:

[0167] According to at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos matching the target tags are obtained, wherein the tags are determined based on the content contained in the photos;

[0168] Determining the duration of the background music according to the set number and the preset display duration of a single photo;

[0169] Determine the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style, and determine any background music that meets the duration value in each target background music style as the target background music;

[0170] An electronic photo album is generated according to the target background music and each of the target photos.

[0171] Since the principle of solving the problem provided by the computer-readable medium is similar to the method for generating an electronic photo album, after the processor executes the computer program in the computer-readable medium, the steps implemented can be referred to the implementation of the method, and the repeated parts will not be repeated.

[0172] Since the present application can obtain a set number of target photos matching the target tag based on at least one target tag carried in the received album generation instruction, the user does not need to select photos that meet the needs one by one, which reduces the time spent on selecting photos. The duration value of the background music is determined based on the target number and the preset display duration of a single photo. The target background music style is determined based on the correspondence between the preset tag and the music style, and any background music that meets the duration value in each target background music style is determined as the target background music, further reducing the time spent on selecting background music, thereby improving the efficiency of electronic album generation to a certain extent.

[0173] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0177] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for generating an electronic photo album, characterized in that: The method comprises: According to at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos matching the target tags are obtained, wherein the tags are determined based on the content contained in the photos; Determining the duration of the background music according to the set number and the preset display duration of a single photo; Determine the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style, and determine any background music that meets the duration value in each target background music style as the target background music; generating an electronic photo album according to the target background music and each of the target photos; The tag is obtained in the following way: Determine the type of object contained in the received photo through an object detection algorithm; Determining first tags corresponding to the photos according to types of objects contained in the photos; If the type of object included in the photo is a person, and a face detection algorithm is used to determine that the photo contains a face, a face feature vector in the photo is obtained using a face feature extraction algorithm, and a similarity between the face feature vector and any pre-stored registered face feature vector is obtained; if the maximum similarity is greater than a set first threshold, it is determined that the face feature vector meets a preset condition, and a second label corresponding to the photo is determined based on the identity identifier corresponding to the registered face feature vector to which the maximum similarity belongs; The step of determining any background music that meets the duration value in each target background music style as the target background music includes: In the pre-saved background music, each candidate background music whose difference between any playing duration value and the duration value is within a certain value range is searched, and in each candidate background music, a target background music whose corresponding music style matches any target background music style is randomly selected.

2. The method according to claim 1, characterized in that The tag is obtained as follows: Using the pre-trained classification model, we can obtain the probability that the received photo is of each style. determining a target style of the photo according to each of the probabilities; A third label of the photo is determined according to the target style.

3. The method according to claim 2, characterized in that Determining the target photo type of the photo according to each probability includes: If there is a probability greater than the set second threshold, the style corresponding to the probability greater than the set second threshold is determined as the target style of the photo; otherwise, the target style of the photo is determined to be the preset style.

4. The method according to claim 2, characterized in that The classification model is trained as follows: Get any image sample in the sample set and each corresponding style; The original classification model is trained based on the image samples and each corresponding style.

5. The method according to claim 1, wherein If the album generation instruction also carries a target location, a target number, and a target time period, obtaining a set number of target photos that match the target tag based on at least one pre-stored tag corresponding to each photo and at least one target tag carried in the album generation instruction includes: updating the set quantity according to the target quantity; According to the collection location, collection time and at least one tag corresponding to each pre-saved photo, and the target location, the target time period and the target tag carried in the album generation instruction, the target number of target photos that match the target tag, the target location and the target time period are obtained.

6. The method according to claim 5, characterized in that The step of determining the target background music style corresponding to each target tag according to the preset correspondence between the tag and the music style includes: The target background music style corresponding to each target tag is determined according to the corresponding relationship between the preset time period and the music style, and the corresponding relationship between the preset tag and the music style.

7. A device for generating an electronic photo album, characterized in that: The device comprises: an acquisition unit, configured to acquire, based on at least one pre-stored tag corresponding to each photo and at least one target tag carried in the received album generation instruction, a set number of target photos that match the target tag, wherein the tag is determined based on the content contained in the photo; A first determining unit is configured to determine a duration of the background music according to the set number and a preset display duration of a single photo; A second determining unit is configured to determine a target background music style corresponding to each target tag according to a preset correspondence between the tag and the music style, and determine any background music in each target background music style that meets the duration value as the target background music; a processing unit, configured to generate an electronic photo album based on the target background music and each of the target photos; Wherein, the processing unit is further used to obtain the label in the following manner: determining the type of object contained in the received photo through an object detection algorithm; determining the first label corresponding to the photo according to the type of object contained in the photo; if the type of object contained in the photo includes a person, determining that the photo contains a face through a face detection algorithm, obtaining the face feature vector in the photo through a face feature extraction algorithm, and obtaining the similarity between the face feature vector and any pre-stored registered face feature vector; if the maximum similarity is greater than a set first threshold, determining that the face feature vector meets a preset condition, and determining the second label corresponding to the photo according to the identity identifier corresponding to the registered face feature vector to which the maximum similarity belongs; The processing unit is specifically used to search for each candidate background music in the pre-saved background music, wherein the difference between any playback duration value and the duration value is within a certain value range, and randomly select a target background music from each candidate background music, wherein the corresponding music style matches any target background music style.

8. An electronic device, characterized in that: The electronic device includes at least a processor and a memory, and the processor is configured to implement the steps of the method for generating an electronic photo album as claimed in any one of claims 1 to 6 when executing a computer program stored in the memory.

Images