Photo album generation method, apparatus, and device

Abstract

Embodiments of the present application disclose a photo album generation method, device and equipment. The method comprises: in response to a photo album generation request triggered by a user device, obtaining target information, the target information comprising photo album setting information input by the user device in the photo album generation request; obtaining a plurality of preset photos corresponding to the target information from a preset photo storage space; inputting the plurality of preset photos into a photo album layout generation model; fitting target layout information corresponding to the plurality of preset photos according to the graph structure information and visual information of the plurality of preset photos by the photo album layout generation model; and generating a photo album according to the target layout information. The embodiments of the present application can automatically generate a photo album with reasonable layout.

Description

Technical Field

[0001] This application belongs to the field of voiceprint recognition technology in the field of communications, and particularly relates to a method, apparatus and device for generating photo albums. Background Technology

[0002] With the rapid development of photography and internet technology, capturing life's moments with photos has become an indispensable part of people's daily lives, giving life a unique "sense of ritual." To better meet users' needs for online management and sharing of digital assets such as personal mobile phone albums and family albums, an innovative service—Story Album—has emerged.

[0003] A story album is a photo album that uses a combination of pictures and text to record and present a specific theme or event. Typically, a story album vividly portrays a specific storyline or theme through photographs, text descriptions, and illustrations. Story albums are often used to record special moments such as travel experiences, family activities, wedding anniversaries, and children's growth, and can also be used to narrate the development of a theme or event. Through story albums, people can present their memories and emotions in a fun and creative way, preserving precious memories and stories.

[0004] Currently, when generating story albums, users often need to manually select photos, which fails to meet the needs for intelligent and automated creation. Furthermore, the layout of photos in current story albums is rather monotonous, lacking variation and creativity, and thus lacking visual appeal, resulting in a poor user experience. Summary of the Invention

[0005] This application provides a method, apparatus, device, computer storage medium, and computer program product for generating photo albums, which can automatically generate photo albums with reasonable layouts.

[0006] In a first aspect, embodiments of this application provide a method for generating a photo album, including:

[0007] In response to a photo album creation request triggered by a user device, the target information is obtained, including the photo album settings information input by the user device in the photo album creation request;

[0008] Retrieve multiple preset photos corresponding to the target information from the preset photo storage space;

[0009] Input multiple preset photos into the album layout generation model;

[0010] By generating a photo album layout model, the target layout information corresponding to multiple preset photos is obtained by fitting the graph structure information and visual information of multiple preset photos.

[0011] Generate an album from multiple preset photos according to the target layout information.

[0012] In one optional implementation, the album layout generation model includes a first layout prediction network and a second layout prediction network; the graph structure information includes the Laplacian matrix and edge features corresponding to the preset photos;

[0013] By generating a photo album layout model, target layout information corresponding to multiple preset photos is obtained by fitting the graph structure and visual information of multiple preset photos, including:

[0014] The first layout prediction network processes the Laplacian matrix and edge features of each preset photo to obtain the bounding box and class label corresponding to each preset photo.

[0015] Based on the bounding box and class label corresponding to each preset photo, the initial layout information of each preset photo is obtained;

[0016] Based on the initial layout information and visual information of each preset photo, the target layout information corresponding to multiple preset photos is obtained by fitting through the second layout prediction network.

[0017] In one optional implementation, the method for determining the Laplacian matrix corresponding to the preset photo includes:

[0018] The first layout prediction network is used to extract the subject information, background information, and decoration information of each preset photo.

[0019] The first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix of each preset photo are calculated respectively. The first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information.

[0020] Based on the first, second, and third Laplacian matrices of each preset photo, generate the Laplacian matrix of each preset photo respectively.

[0021] In one optional implementation, the method for determining the visual information corresponding to the preset photograph includes:

[0022] Multi-scale sampling is performed on each preset photo to obtain the visual information of each preset photo.

[0023] In one optional implementation, based on the initial layout information and visual information of each preset photo, a second layout prediction network is used to fit multiple preset photos according to the target layout information, including performing the following steps through the second layout prediction network:

[0024] Based on the initial layout information and visual information of each preset photo, the target layout score of each preset photo is calculated. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and the adjacent photos, and the emotional difference loss value between the preset photo and the adjacent photos. The adjacent photos are other preset photos that are adjacent to the position of the preset photo in the initial layout information.

[0025] Based on the target layout score of each preset photo, and combined with the correspondence between the preset layout score and fitness value, the target fitness value corresponding to the initial layout information is calculated.

[0026] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to the preset fitness threshold, thus obtaining the target layout information.

[0027] In one optional implementation, the visual information includes information about the emotions of the people and the color information of the photos; based on the initial layout information of each preset photo and the visual information of each preset photo, a target layout score for each preset photo is calculated, including performing the following steps for each preset photo:

[0028] Calculate the layout position score of each preset photo based on the initial layout information of each preset photo;

[0029] Based on the color information of the preset photo and the color information of the preset photos adjacent to the preset photo, calculate the color difference loss value between the preset photo and the adjacent photos.

[0030] Based on the emotional information of the people in the preset photo and the emotional information of the people in the preset photos adjacent to the preset photo, calculate the emotional difference loss value between the preset photo and the adjacent photos.

[0031] In one optional implementation, the layout position score of the preset photo is calculated based on the initial layout information of the preset photo, including:

[0032] Based on the initial layout information, determine the distance between the preset photos and the preset photos adjacent to the preset photos; based on the distance and the preset distance threshold, calculate the layout position score of the preset photos.

[0033] In one optional implementation, in response to a user-triggered album generation request, target information is obtained, including:

[0034] In response to a user-triggered album creation request, obtain the voice information input by the user's device;

[0035] The speech information is recognized to obtain the target information.

[0036] In one optional implementation, after generating an album from multiple preset photos according to target layout information, the method further includes:

[0037] In response to a user device's selection of multiple preset photos, a target photo is determined from the multiple preset photos;

[0038] Obtain target description information for the target photo, including voice description information and / or video description information input by the user device;

[0039] Based on the target description information, a target control is generated in the album.

[0040] In one alternative implementation, after generating the target control in the photo album based on the target description information, the method further includes:

[0041] In response to the user device's selection of the target control, display the target description information.

[0042] Secondly, embodiments of this application provide a photo album generation device, comprising:

[0043] The acquisition module is used to respond to the album generation request triggered by the user device and acquire target information, including album setting information input by the user device in the album generation request;

[0044] The acquisition module is also used to acquire multiple preset photos corresponding to the target information from the preset photo storage space;

[0045] The input module is used to input multiple preset photos into the album layout generation model;

[0046] The fitting module is used to generate a model through the album layout. Based on the graph structure information and visual information of multiple preset photos, it fits the target layout information corresponding to multiple preset photos.

[0047] The generation module is used to generate an album from multiple preset photos according to the target layout information.

[0048] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions;

[0049] When the processor executes computer program instructions, it implements an album generation method as described in any optional embodiment of the first aspect of this application.

[0050] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the album generation method as described in any optional embodiment of the first aspect of this application.

[0051] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform an album generation method as described in any optional embodiment of the first aspect of this application.

[0052] The album generation method, apparatus, device, computer storage medium, and computer program product of this application embodiment can respond to an album generation request triggered by a user device and obtain target information, including album setting information input by the user device in the album generation request. Then, multiple preset photos corresponding to the target information are obtained from a preset photo storage space. This allows for the selection of only suitable photos according to the user's needs, eliminating the need for manual selection and improving the automation level of the album generation method. Subsequently, the multiple preset photos are input into an album layout generation model. Through the album layout generation model, target layout information corresponding to the multiple preset photos is fitted based on the graph structure information and visual information of the multiple preset photos. This allows for the adaptive generation of suitable layout information based on the graph structure information and visual information of the multiple preset photos, ensuring that the album layout meets the visual and narrative expression needs and improving the flexibility of the album layout. Finally, the multiple preset photos are used to generate an album according to the target layout information. This improves the coherence and expressiveness of the album, thereby enhancing the user experience. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a photo album generation method provided in one embodiment of this application;

[0055] Figure 2 This is a schematic diagram of the structure of an album layout generation model provided in another embodiment of this application;

[0056] Figure 3 This is a schematic diagram of the layout structure of a photo album page provided in another embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the structure of a photo album generation device provided in another embodiment of this application;

[0058] Figure 5 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0059] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0061] As described in the background section, currently, when generating story albums, users often need to manually select photos, which fails to meet the needs of intelligent and automated processes. Furthermore, the current story album layouts are rather monotonous, lacking variation and creativity, and thus lack visual appeal, resulting in a poor user experience.

[0062] The related technology relates to a method for generating story albums based on cloud storage. In this method, users can log in to a cloud storage application (APP) and select suitable images from a collection of photos stored in the cloud for generating a story album. Then, they can click the "Recommended Stories" card on the homepage to proceed to the next step. On this step, users can further select functions such as "Manage Photos," "Edit Title," and "Save to Video Story." If the user has enabled the smart album authorization switch, the cloud storage APP can generate the story album based on the user's actions.

[0063] However, the story album generation methods mentioned above still require users to manually select photos, lacking intelligent and automated features. Furthermore, these technologies typically use fixed templates for photo layout, resulting in a monotonous and uncreative approach.

[0064] In view of this, after in-depth thinking, the inventors have ingeniously proposed a method for generating photo albums, a method for receiving information, a device, an equipment, a computer storage medium, and a computer program product.

[0065] The album generation method provided in this application will be described below with reference to the accompanying drawings, through specific embodiments and application scenarios. The album generation method provided in this application can be executed by an album generation device, or a portion of the album generation device used to execute the album generation method. This application uses an album generation device executing the album generation method as an example to describe in detail the album generation method provided in this application.

[0066] The following is in conjunction with the appendix Figure 1 The album generation method provided in the embodiments of this application will be described in detail.

[0067] Figure 1 A flowchart illustrating an embodiment of the album generation method provided in this application is shown. Figure 1 As shown, the album generation method may specifically include the following steps S110 to 150.

[0068] S110, in response to the album generation request triggered by the user device, obtain target information, the target information including album setting information input by the user device in the album generation request.

[0069] In step S110, the album generation request can include an album generation request triggered by the user through actions such as clicking or pressing, for example, an album generation request triggered by the user clicking the "Recommended Stories" card in the cloud drive APP. Obtaining target information can include obtaining album setting information input by the user's device through the user input interface, which can include text input interface, voice input interface, option input interface, etc., and is not limited here. The album setting information can include the user's requirements for the story album, such as the theme, number of photos, type, content, layout style, etc.

[0070] S120: Obtain multiple preset photos corresponding to the target information from the preset photo storage space.

[0071] In step S120, the photo storage space may include local photo storage space, such as a local photo album; the photo storage space may also include cloud storage space, such as a cloud photo album. Multiple preset photos may include multiple preset photos that match the target information.

[0072] S130: Input multiple preset photos into the album layout generation model.

[0073] S140: By generating a photo album layout model, the target layout information corresponding to multiple preset photos is obtained by fitting the graph structure information and visual information of multiple preset photos.

[0074] In step S140, the graph structure information may include the neighboring node features and edge features of the photograph. For example, the neighboring node features may include information about other photographs adjacent to the photograph and located above, below, to the left, or right of it; for instance, it may be the Laplacian matrix corresponding to the photograph. Visual information may include information conveyed visually, such as information about the emotions of people in the photograph, color information, and content information.

[0075] S150 generates an album from multiple preset photos according to the target layout information.

[0076] The album generation method of this application embodiment can respond to an album generation request triggered by a user device and obtain target information, including album setting information input by the user device in the album generation request. Then, multiple preset photos corresponding to the target information are obtained from a preset photo storage space. This allows for the selection of only suitable photos according to the user's needs, eliminating the need for manual selection and improving the automation level of the album generation method. Subsequently, the multiple preset photos are input into an album layout generation model. Through the album layout generation model, target layout information corresponding to the multiple preset photos is fitted based on the graph structure information and visual information of the multiple preset photos. This allows for the adaptive generation of suitable layout information based on the graph structure information and visual information of the multiple preset photos, ensuring that the album layout meets the visual and narrative expression needs and improving the flexibility of the album layout. Finally, the multiple preset photos are used to generate an album according to the target layout information. This improves the coherence and expressiveness of the album, thereby enhancing the user experience.

[0077] In one embodiment, the album layout generation model includes a first layout prediction network and a second layout prediction network. Graph structure information includes the Laplacian matrix and edge features corresponding to preset photos.

[0078] By generating an album layout model, target layout information corresponding to multiple preset photos is obtained by fitting the graph structure and visual information of multiple preset photos. Specifically, this may include:

[0079] The first layout prediction network processes the Laplacian matrix and edge features of each preset photo to obtain the bounding box and class label corresponding to each preset photo.

[0080] Based on the bounding box and class label corresponding to each preset photo, the initial layout information of each preset photo is obtained.

[0081] Based on the initial layout information and visual information of each preset photo, the target layout information corresponding to multiple preset photos is obtained by fitting through the second layout prediction network.

[0082] In the above embodiments, the Laplace matrix has a meaning known in the art and can be calculated using methods known in the art. As an example, the Laplace matrix can be represented as shown in Equation 1.

[0083] Δ = ID -1 / 2 AD -1 / 2 Formula 1

[0084] In Equation 1, Δ can represent the Laplacian matrix of the photo; I can represent the original photo; D can represent the degree matrix corresponding to the photo; and A can represent the adjacency matrix corresponding to the features of the photo.

[0085] According to the above implementation method, the Laplacian matrix and edge features of each preset photo are processed by the first layout prediction network. This allows for the combination of the topological structure and connectivity of each preset photo, along with image variation information, to obtain the bounding boxes and class labels corresponding to each preset photo, thereby generating initial layout information. Then, the initial layout is further optimized by incorporating the visual information of each preset photo. This allows for flexible adjustment of the photo layout based on the visual communication effect of each preset photo, resulting in more visually harmonious photos. This improves the flexibility and rationality of the photo layout, thereby enhancing the visual appeal of the album and improving the user experience when browsing the album.

[0086] In one embodiment, the method for determining the Laplacian matrix corresponding to the preset photo may include:

[0087] The first layout prediction network is used to extract the subject information, background information, and decoration information of each preset photo.

[0088] The first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix are calculated for each preset photo. The first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information.

[0089] Based on the first, second, and third Laplacian matrices of each preset photo, generate the Laplacian matrix of each preset photo respectively.

[0090] In the above embodiments, the main subject information may include person information or main scenery information. Main scenery information can represent the scenery information contained in a photograph that primarily presents local scenery. For example, if a photograph contains people, and the people occupy a prominent position in the photograph (e.g., the people are located in the center of the photograph, or the percentage of area occupied by the people in the photograph is greater than a preset threshold, etc.), the main subject information is person information. If the photograph does not contain people, or the people do not occupy a prominent position in the photograph, the main subject information is main scenery information. Decoration information may include information about the decorations worn by the people in the photograph. Background information may include the photograph's background, for example, it may include other information in the photograph besides the main subject information and decoration information. For example, if an image only presents the scenery of the "Li River" and there are no people in the photograph, then the "Li River" scenery information included in the photograph can be considered the main scenery information. Conversely, if a photograph contains people, and the people occupy a prominent position in the photograph, then even if the background information of the photograph is the "Li River" scenery, the "Li River" scenery information included in such a photograph will not be considered the main scenery information, but rather the background information.

[0091] In one example, calculating the first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix for each preset photo may include: classifying the people, main scenery, background information, and decorative information in each preset photo; and determining the first Laplacian matrix corresponding to the people or main scenery in the preset photo, the second Laplacian matrix corresponding to the background information, and the third Laplacian matrix corresponding to the decorative information.

[0092] In one example, generating the Laplacian matrix of each preset photo based on its first, second, and third Laplacian matrices can include: flipping the first and second Laplacian matrices. The Laplacian matrix of each preset photo is then generated based on the flipped results of the first and second Laplacian matrices and the third Laplacian matrix.

[0093] According to the above implementation method, by classifying and extracting graph structure-related information from the photograph, subject information, background information, and decorative object information are obtained. Then, the Laplacian matrix corresponding to each of the subject information, background information, and decorative object information is calculated separately, thereby generating the Laplacian matrix of the photograph. In this way, rich structural information can be extracted from the photograph, which is beneficial for generating a reasonable and coherent scene layout.

[0094] In one embodiment, the method for determining the visual information corresponding to the preset photograph may include:

[0095] Multi-scale sampling is performed on each preset photo to obtain the visual information of each preset photo.

[0096] According to the above implementation method, visual information is obtained by sampling each preset photo at multiple scales. This allows for the subsequent fitting of target layout information with the visual information, which helps to clearly express the storyline, enhances user focus when browsing the album, and improves the coherence and expressiveness of the album's narrative.

[0097] In one embodiment, based on the initial layout information and visual information of each preset photo, a second layout prediction network is used to fit multiple preset photos according to the target layout information. Specifically, this may include performing the following steps through the second layout prediction network:

[0098] Based on the initial layout information and visual information of each preset photo, the target layout score of each preset photo is calculated. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and the adjacent photos, and the emotional difference loss value between the preset photo and the adjacent photos. The adjacent photos are other preset photos that are adjacent to the position of the preset photo in the initial layout information.

[0099] Based on the target layout score of each preset photo, and combined with the correspondence between the preset layout score and fitness value, the target fitness value corresponding to the initial layout information is calculated.

[0100] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to the preset fitness threshold, thus obtaining the target layout information.

[0101] According to the above implementation method, by calculating the layout position score of each preset photo, the suitability of the position of each preset photo in the initial layout can be measured. By calculating the color difference loss value between the preset photo and adjacent photos, the consistency of color in the photos in the initial layout can be measured. By calculating the emotional difference loss value between the preset photo and adjacent photos, the consistency of emotion in the photos in the initial layout can be measured. Based on the layout position score, color difference loss value, and emotional difference loss value of each preset photo, a target fitness value for the initial layout is calculated, and the initial layout information is adjusted based on this target fitness value. In this way, the layout position, color distribution, and emotional distribution of each preset photo in the final album layout are all more reasonable. This can improve visual harmony and the coherence of narrative expression, which is conducive to improving the expressiveness and attractiveness of the album.

[0102] In one embodiment, visual information may include information about a person's emotions and color information of the photograph. Based on the initial layout information and visual information of each preset photograph, a target layout score is calculated for each preset photograph. Specifically, this may involve performing the following steps for each preset photograph:

[0103] Calculate the layout position score of each preset photo based on the initial layout information of each preset photo.

[0104] Based on the color information of the preset photo and the color information of the preset photos adjacent to it, calculate the color difference loss value between the preset photo and the adjacent photos.

[0105] Based on the emotional information of the people in the preset photo and the emotional information of the people in the preset photos adjacent to the preset photo, calculate the emotional difference loss value between the preset photo and the adjacent photos.

[0106] According to the above implementation method, by calculating the layout position score of the preset photos using initial layout information, the suitability of the photo layout can be quantified, which is beneficial for evaluating and reasonably adjusting the position of the preset photos in the album. By calculating the color difference loss value between the preset photos and adjacent photos, the color continuity of the photos in the initial layout can be intuitively represented, which is beneficial for evaluating and reasonably adjusting the album layout based on the color difference loss value, thereby improving the color continuity of the album layout. By calculating the emotional difference loss value between the preset photos and adjacent photos, the emotional continuity of the photos in the initial layout can be intuitively represented, which is beneficial for evaluating and reasonably adjusting the album layout based on the emotional difference loss value, thereby facilitating the clear and coherent expression of the storyline. In this way, the intelligence level and visual appeal of album generation can be further improved.

[0107] In one embodiment, calculating the layout position score of the preset photo based on the initial layout information of the preset photo may specifically include:

[0108] Based on the initial layout information, determine the distance between the preset photos and the preset photos adjacent to the preset photos.

[0109] Calculate the layout position score of the preset photo based on the distance and the preset distance threshold.

[0110] In the above embodiments, the preset distance threshold may include the preset minimum distance between adjacent photos.

[0111] According to the above implementation method, the layout position score of the preset photos is calculated based on the distance between adjacent photos and a preset distance threshold. This can quantify the suitability of the photo layout, thereby facilitating the evaluation and reasonable adjustment of the position of the preset photos in the album.

[0112] For example, the layout location score, color difference loss value, and mood difference loss value can be calculated using Equations 2 to 4 as follows.

[0113]

[0114] In Equation 2, P i The layout score for photo i can be used to indicate how appropriate the layout of photo i is; D can represent the layout orientation in the story album, such as east, west, south, and north; T i D This represents the actual distance between photo i and its neighboring photos in the D direction; This can represent the minimum distance between photo i and other photos in its D direction. It can be a preset value.

[0115]

[0116] In Equation 3, L 色彩 (i)i can represent the color difference loss value between photo i and its neighboring photos; s[E(i)] can represent the color information of photo i; It can represent the color information of the adjacent photo j in the D direction of photo i.

[0117]

[0118] In Equation 4, L 情绪 (i) can represent the emotional difference loss value between photo i and the photos in its adjacent positions; E[E(i)] can represent the emotional information of the person in photo i; It can represent the emotional information of people in adjacent photos j in the D direction of photo i.

[0119] For example, after calculating the layout position score, color difference loss and emotion difference loss of each preset photo using Equations 2 to 4, the target fitness value F corresponding to the initial layout can be calculated according to Equation 5 below, based on the target layout score of each preset photo.

[0120] F=[∑p i +L 色彩 (i)+L 情绪 (i)+1] -1 Formula 5

[0121] In one embodiment, such as Figure 2As shown, the album layout generation model may include a first layout prediction network 10 and a second layout prediction network 20. The first layout prediction network 10 may include a multi-head attention mechanism (Q, K, V, E), a normalized layout template, a softmax function, a sum function, a concat function, a linear function, a feedforward neural network (FFN), a residual connection layer (add), and a normalization operation layer (norm). The second layout prediction network 20 may include a feature extraction network, multiple convolutional layers with different kernels, an upsampling layer, a pooling layer, a concat function layer, and a decoding layer.

[0122] In one example, multiple preset photos can be input into the album layout generation model. The album layout generation model can first extract subject information, background information, and decorative information from the photos based on the photo content using a first layout prediction network 10. The subject information, background information, and decorative information in each preset photo are then classified, and the corresponding Laplacian matrices for each subject information, background information, and decorative information are determined. The representation of the Laplacian matrix can be referred to Equation 1, and will not be repeated here.

[0123] Subsequently, based on the Laplacian matrices corresponding to the subject and background information, the subject and background information can be flipped using a preset flip function λ. Based on the flipped result and the combined Laplacian matrices corresponding to the decorative information, the Laplacian matrices for each preset photograph can be obtained.

[0124] Next, the Laplacian matrix and edge features of each preset image can be processed by the first layout prediction network 10 to obtain the bounding box and class label corresponding to each preset image. Specifically, the Attention(Q, K, V, E) result of each preset image under the multi-head attention mechanism can be determined through a multi-head attention mechanism. Based on the output of the multi-head attention mechanism, the bounding box and class label of each preset image are obtained. The definition of the multi-head attention mechanism can be found in Equation 6.

[0125]

[0126] In Equation 6, It can represent the learnable projection matrix; k can represent the number of heads in the multi-head attention mechanism, k∈{0,…,N} heads};h i h can represent the feature vector of photo i; j e can represent the feature vector of photo j; ij d can represent the edge features of a photo; d can represent the number of photo categories; d k The number of categories of associated photos can be represented by Equation 7.

[0127]

[0128] Next, based on the bounding box and class label of each photo and the preset specification layout template parameters, then according to... Figure 2 The content shown is processed sequentially using the softmax function, sum function, concat function, linear function, feedforward neural network (FFN), residual connection layer, and layer normalization operation layer to obtain the initial layout information of the image. The initial layout information can be represented by Equation 8.

[0129] L layout =L box +L label +L iou Formula 8

[0130] In Equation 8, L layout It can represent the initial layout information of a photograph; L box It can represent the loss of coordinates on the bounding box of the image; L label It can represent the cross-entropy on the label of a photo; L iou This can represent the distance loss between the bounding box of a photo and the edge of the story album layout page.

[0131] Next, the initial layout information can be passed to a 1*1 convolutional layer in the second layout prediction network 20. This 1*1 convolutional layer can perform convolution operations based on visual information such as the emotional information of people in the photo, the color information of the photo, and the content information of the photo obtained by the feature extraction network, as well as the initial layout information of the photo. Then, an upsampling operation is performed on the result of the convolution operation to obtain the first result. The decoding layer in the second layout prediction network 20 performs convolution operations based on multiple preset photos to obtain the second result. Then, the first and second results can be fused using the concat function. After that, a 3*3 convolution operation is performed on the fused result, followed by a second upsampling operation to obtain the prediction result of the final layout of each preset photo in the story album, that is, to obtain the target layout information corresponding to multiple preset photos.

[0132] Specifically, the second layout prediction network 20 can combine the initial layout information with the emotion, color, and content information extracted by the feature extraction network to calculate the layout position score, color difference loss value, and emotion difference loss value between preset photos in adjacent positions in the initial layout. The specific calculation process can be shown in Equations 2 to 4 above, and will not be elaborated here. Next, the target fitness value F corresponding to the initial layout can be calculated with reference to Equation 5. When the target fitness value F is greater than or equal to the preset fitness threshold, the target layout information of the story album is output. Otherwise, the model parameters of the second layout prediction network 20 are iteratively updated according to the color difference loss value, emotion difference loss value, and layout position score between photos in adjacent positions in the initial layout to adjust the initial layout information until the calculated target fitness value is greater than or equal to the preset fitness threshold, then the update stops, and the target layout information of the story album is output.

[0133] In one embodiment, in response to a user-triggered album generation request, target information is obtained, which may specifically include:

[0134] In response to a user-triggered album creation request, obtain the voice information input from the user's device.

[0135] The speech information is recognized to obtain the target information.

[0136] For example, after a user triggers an album generation request, a voice input window can be displayed to the user. This voice input window can include a voice input device symbol, such as a microphone icon. The user can click or press the voice input device symbol to provide feedback to the server regarding their needs for the story album. For instance, a user could click the voice input device symbol and say, "Please find 5 funny solo photos to generate a funny and enjoyable story album." Or, they could say, "Please generate a travel story album based on photos taken in XXX location, with more group photos and fewer solo photos," and so on.

[0137] In the above embodiments, the recognition of voice information can be achieved by methods known in the art, which will not be elaborated here. For example, key information such as the theme, number, type, content, and layout style of the story album can be extracted from the voice information using voice recognition technology to obtain the target information.

[0138] According to the above implementation method, users describe their needs for the story album via voice, which, compared to manual input, quickly converts their intentions into text, avoiding the tedious process of manual input. Furthermore, through voice description, users only need to describe their needs to the cloud server, which then selects suitable photos and generates a story album matching those needs. This eliminates the need for users to manually select photos one by one, significantly improving the intelligence and automation of the story album generation process, thereby enhancing album generation efficiency and user experience.

[0139] In one embodiment, after generating an album from multiple preset photos according to target layout information, the method may further include:

[0140] In response to a user device's selection of multiple preset photos, a target photo is determined from the multiple preset photos.

[0141] Obtain target description information for the target photo, including voice description information and / or video description information input by the user device.

[0142] Based on the target description information, a target control is generated in the album.

[0143] In related technologies, text descriptions are often added to photos in story albums to capture certain memorable moments. When there is a lot of story behind a memorable moment, a large amount of text may be needed to describe it, thus taking up a large area of ​​the photo, destroying the aesthetics and simplicity of the photo, affecting the overall visual effect, and making the photo appear cluttered or chaotic.

[0144] In the above embodiments, the target photo can be a photo selected by the user that corresponds to a memorable moment. The target control may include a button or other component associated with target description information. The target control can be located anywhere in the album. For example, the target control can be located on the border of the target photo or on the target photo. The target control can also be located in other locations in the album and associated with the target photo through an icon or line.

[0145] According to the above implementation method, target controls can replace text descriptions, thus avoiding the problem of viewers being distracted and the visual effect and appeal of the photo album being reduced when adding too much text description of the story behind the photos. Furthermore, using target controls instead of text descriptions can also prevent excessive text descriptions from ruining the aesthetics and simplicity of the photos, thereby improving the overall visual effect of the album.

[0146] In one embodiment, after generating the target control in the photo album based on the target description information, the method may further include:

[0147] In response to the user device's selection of the target control, display the target description information.

[0148] In the above embodiments, the user device may include any user device with photo album viewing permissions. The method of displaying the target description information can be selected according to the form of the target description information. For example, when the target description information is voice information, it can be displayed by playing the audio of the target description information; when the target description information is video information, it can jump to the video playback window and display the target description information through the video playback window.

[0149] According to the above implementation method, target description information can be displayed in response to the user device's selection operation of the target control. In this way, the album's storyline can be enriched while maintaining its simplicity and aesthetics.

[0150] To facilitate understanding, the following will continue to use the example of generating "Highlights" in the album to explain the above implementation method in detail.

[0151] In one example, after the album is generated, the user can select a "Highlight Moment" photo from the album and then enter a description of the actual situation corresponding to the "Highlight Moment." The cloud server can then generate an audio or video story section from the description and embed it into the appropriate location of the relevant photo in the album. Later, when the user browses the album and wants to learn the story behind the "Highlight Moment" photo, they can trigger the corresponding audio or video story section to listen to or view the story.

[0152] The above-mentioned wonderful moments can be defined by the user, such as unforgettable moments, the happiest moments, the saddest moments, etc., and there are no restrictions here.

[0153] As an example, such as Figure 3 As shown, a page in the album can include four photos: Photo 1, Photo 2, Photo 3, and a "Highlight Photo." The "Highlight Photo" shows a photo where the user "laughed and cried." In practice, when the album is created and the user views this photo, they usually easily recall why they "laughed and cried" at the time. However, as time goes by, the user may forget why they "laughed and cried," or when the user shares the album with a third party, that third party may also be curious about why the user "laughed and cried." To avoid this, an audio or video story section can be generated for the "Highlight Photo" to allow users to learn the story behind the moment later. For example, as shown... Figure 3 As shown, the audio story section or video story section can be set on the photo of the highlight moment in the form of control 100.

[0154] After generating control 100, when browsing the story album, users can click on control 100 to jump to a video playback window, playing the story behind the photo of that special moment, or listen to the corresponding audio story to explain the background of the photo. This approach avoids adding excessive text descriptions to the photos, thus enhancing their visual appeal and reducing the risk of distraction. It also prevents excessive text from ruining the aesthetics and simplicity of the photos, making them appear cluttered or chaotic. Furthermore, some viewers may not want to spend time and effort reading lengthy text descriptions, especially when browsing photos, preferring to intuitively appreciate the information conveyed by the photos. Adding controls not only avoids over-reliance on text descriptions but also allows viewers to choose whether to view the story behind the special moment. This further enhances the user experience.

[0155] Based on the same inventive concept as the album generation method, this application also provides an album generation apparatus.

[0156] like Figure 4 As shown, the album generation device 200 may include a first acquisition module 201, an input module 202, a fitting module 203, and a first generation module 204.

[0157] The first acquisition module 201 is used to acquire target information in response to a photo album generation request triggered by a user device. The target information includes the photo album settings information input by the user device in the photo album generation request.

[0158] The first acquisition module 201 is also used to acquire multiple preset photos corresponding to the target information from a preset photo storage space.

[0159] The input module 202 is used to input multiple preset photos into the album layout generation model.

[0160] The fitting module 203 is used to generate a model through the album layout, and to fit the target layout information corresponding to multiple preset photos based on the graph structure information and visual information of multiple preset photos.

[0161] The first generation module 204 is used to generate an album from multiple preset photos according to the target layout information.

[0162] The album generation apparatus of this application embodiment can respond to an album generation request triggered by a user device and obtain target information, including album setting information input by the user device in the album generation request. Then, it retrieves multiple preset photos corresponding to the target information from a preset photo storage space. This allows for the selection of only suitable photos according to the user's needs, eliminating the need for manual selection and improving the automation level of the album generation method. Subsequently, the multiple preset photos are input into an album layout generation model. Through the album layout generation model, target layout information corresponding to the multiple preset photos is fitted based on the graph structure information and visual information of the multiple preset photos. This allows for the adaptive generation of suitable layout information based on the graph structure information and visual information of the multiple preset photos, ensuring that the album layout meets the visual and narrative expression needs and improving the flexibility of the album layout. Finally, the multiple preset photos are used to generate an album according to the target layout information. This improves the coherence and expressiveness of the album, thereby enhancing the user experience.

[0163] In one embodiment, the album layout generation model may include a first layout prediction network and a second layout prediction network. Graph structure information may include the Laplacian matrix and edge features corresponding to preset photos.

[0164] The fitting module is used to generate a model based on the album layout. It fits the target layout information corresponding to multiple preset photos based on the graph structure and visual information of those photos. Specifically, it can include:

[0165] The processing submodule is used to process the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain the bounding box and class label corresponding to each preset photo.

[0166] The processing submodule is also used to obtain the initial layout information of each preset photo based on the bounding box and class label corresponding to each preset photo.

[0167] The fitting submodule is used to fit the target layout information corresponding to multiple preset photos through the second layout prediction network based on the initial layout information and visual information of each preset photo.

[0168] In one embodiment, the apparatus may further include:

[0169] The extraction module is used to extract the subject information, background information and decoration information of each preset photo before processing the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain the bounding box and class label of each preset photo.

[0170] The calculation module is used to calculate the first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix for each preset photo. The first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information.

[0171] The second generation module is used to generate the Laplacian matrix of each preset photo based on the first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix of each preset photo.

[0172] In one embodiment, the apparatus may further include:

[0173] The sampling module is used to perform multi-scale sampling on each preset photo before fitting multiple preset photos according to the target layout information through the second layout prediction network based on the initial layout information and visual information of each preset photo, so as to obtain the visual information of each preset photo.

[0174] In one embodiment, the fitting submodule is used to fit multiple preset photos according to the target layout information based on the initial layout information and visual information of each preset photo through a second layout prediction network. Specifically, it may include: the fitting submodule is used to perform the following steps through the second layout prediction network:

[0175] Based on the initial layout information and visual information of each preset photo, the target layout score of each preset photo is calculated. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and the adjacent photos, and the emotional difference loss value between the preset photo and the adjacent photos. The adjacent photos are other preset photos that are adjacent to the position of the preset photo in the initial layout information.

[0176] Based on the target layout score of each preset photo, and combined with the correspondence between the preset layout score and fitness value, the target fitness value corresponding to the initial layout information is calculated.

[0177] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to the preset fitness threshold, thus obtaining the target layout information.

[0178] In one embodiment, visual information may include information about a person's emotions and color information of the photograph. The fitting submodule is used to calculate the target layout score of each preset photograph using the second layout prediction network, based on the initial layout information and visual information of each preset photograph. Specifically, it may include: the fitting submodule, used to perform the following steps for each preset photograph using the second layout prediction network:

[0179] Calculate the layout position score of each preset photo based on the initial layout information of each preset photo.

[0180] Based on the color information of the preset photo and the color information of the preset photos adjacent to it, calculate the color difference loss value between the preset photo and the adjacent photos.

[0181] Based on the emotional information of the people in the preset photo and the emotional information of the people in the preset photos adjacent to the preset photo, calculate the emotional difference loss value between the preset photo and the adjacent photos.

[0182] In one embodiment, the fitting submodule is used to calculate the layout position score of each preset photo based on the initial layout information of the preset photo using the second layout prediction network. Specifically, it may include:

[0183] Based on the initial layout information, determine the distance between the preset photos and the preset photos adjacent to them. Calculate the layout position score of the preset photos based on the distances and a preset distance threshold.

[0184] In one embodiment, the acquisition module is used to acquire target information in response to a user-triggered album generation request, which may specifically include:

[0185] The Get submodule is used to respond to user-triggered album generation requests and obtain voice information input from the user's device.

[0186] The recognition submodule is used to recognize speech information and obtain target information.

[0187] In one embodiment, the apparatus may further include:

[0188] The determination module is used to determine the target photo from the multiple preset photos after generating an album according to the target layout information, in response to the user device's selection operation on the multiple preset photos.

[0189] The second acquisition module acquires target description information of the target photo, which includes voice description information and / or video description information input by the user device.

[0190] The third generation module is used to generate target controls in the album based on the target description information.

[0191] In one embodiment, the apparatus may further include:

[0192] The display module is used to generate a target control in the album based on the target description information, and then display the target description information in response to the user's device selection operation on the target control.

[0193] The album generation device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0194] Figure 5 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0195] An electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0196] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0197] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0198] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0199] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the album generation methods in the above embodiments.

[0200] As an example, the electronic device may also include a communication interface 303 and a bus 310. Wherein, such as Figure 5 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0201] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0202] Bus 310 includes hardware, software, or both, that couples components of the photo album generation device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0203] The electronic device can execute the album generation method in the embodiments of this application, thereby achieving a combination Figure 1 and Figure 4 The described method and apparatus for generating photo albums.

[0204] Furthermore, in conjunction with the album generation methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the album generation methods in the above embodiments.

[0205] This application also provides a computer program product, including a computer program that, when executed, implements any of the album generation methods described in the above embodiments.

[0206] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0207] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0208] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0209] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0210] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for generating a photo album, characterized in that, include: In response to a photo album generation request triggered by a user device, target information is obtained, the target information including photo album settings input by the user device in the photo album generation request; Retrieve multiple preset photos corresponding to the target information from a preset photo storage space; Input the multiple preset photos into the album layout generation model; The album layout generation model is used to fit the target layout information corresponding to the multiple preset photos based on the graph structure information and visual information of the multiple preset photos. Generate an album from the multiple preset photos according to the target layout information; The album layout generation model includes a first layout prediction network and a second layout prediction network; the graph structure information includes the Laplacian matrix and edge features corresponding to the preset photos; The step of using the album layout generation model to fit the target layout information corresponding to the multiple preset photos based on the graph structure information and visual information of the multiple preset photos includes: The first layout prediction network processes the Laplacian matrix and edge features of each preset photo to obtain the bounding box and class label corresponding to each preset photo. Based on the bounding box and class label corresponding to each preset photo, the initial layout information of each preset photo is obtained; Based on the initial layout information and visual information of each preset photo, the target layout information corresponding to the multiple preset photos is obtained by fitting through the second layout prediction network. The step of fitting the multiple preset photos according to the target layout information using the second layout prediction network, based on the initial layout information and visual information of each preset photo, includes performing the following steps using the second layout prediction network: Based on the initial layout information and visual information of each preset photo, a target layout score is calculated for each preset photo. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and adjacent photos, and the emotional difference loss value between the preset photo and adjacent photos. The adjacent photos are other preset photos that are adjacent to the position of the preset photo in the initial layout information. Based on the target layout score of each preset photo, and combined with the correspondence between the preset layout score and fitness value, the target fitness value corresponding to the initial layout information is calculated. The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

2. The method according to claim 1, characterized in that, The methods for determining the Laplacian matrix corresponding to the preset photo include: The first layout prediction network is used to extract the main body information, background information and decoration information of each preset photo. Calculate the first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix for each preset photo, wherein the first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information. Based on the first, second, and third Laplacian matrices of each preset photo, generate the Laplacian matrix of each preset photo respectively.

3. The method according to claim 1, characterized in that, The methods for determining the visual information corresponding to the preset photo include: Multi-scale sampling is performed on each preset photo to obtain the visual information of each preset photo.

4. The method according to claim 1, characterized in that, The visual information includes information about the emotions of the people and the color information of the photos; based on the initial layout information of each preset photo and the visual information of each preset photo, the target layout score of each preset photo is calculated, including performing the following steps for each preset photo: Calculate the layout position score of the preset photo based on the initial layout information of the preset photo; Based on the color information of the preset photo and the color information of the preset photos adjacent to the preset photo, calculate the color difference loss value between the preset photo and the adjacent photos; Based on the emotional information of the people in the preset photo and the emotional information of the people in the preset photos adjacent to the preset photo, calculate the emotional difference loss value between the preset photo and the adjacent photos.

5. The method according to claim 4, characterized in that, The step of calculating the layout position score of the preset photo based on the initial layout information of the preset photo includes: Based on the initial layout information, the distance between the preset photo and the preset photos adjacent to the preset photo is determined; based on the distance and a preset distance threshold, the layout position score of the preset photo is calculated.

6. The method according to any one of claims 1-5, characterized in that, The process of obtaining target information in response to a photo album generation request triggered by a user device includes: In response to a user-triggered album creation request, obtain the voice information input by the user's device; The voice information is recognized to obtain the target information.

7. The method according to any one of claims 1-5, characterized in that, After generating an album from the plurality of preset photos according to the target layout information, the method further includes: In response to a user device's selection operation of the plurality of preset photos, a target photo is determined from the plurality of preset photos; Obtain target description information of the target photo, the target description information including voice description information and / or video description information input by the user device; Based on the target description information, a target control is generated in the album.

8. The method according to claim 7, characterized in that, After generating the target control in the album based on the target description information, the method further includes: In response to the user device's selection of the target control, the target description information is displayed.

9. A photo album generation device, characterized in that, include: The acquisition module is used to acquire target information in response to a photo album generation request triggered by a user device. The target information includes photo album settings information input by the user device in the photo album generation request. The acquisition module is also used to acquire multiple preset photos corresponding to the target information from a preset photo storage space; The input module is used to input the multiple preset photos into the album layout generation model; The fitting module is used to generate a model through the album layout and fit the target layout information corresponding to the multiple preset photos based on the graph structure information and visual information of the multiple preset photos. The generation module is used to generate an album from the multiple preset photos according to the target layout information; The album layout generation model includes a first layout prediction network and a second layout prediction network; the graph structure information includes the Laplacian matrix and edge features corresponding to the preset photos; The step of using the album layout generation model to fit the target layout information corresponding to the multiple preset photos based on the graph structure information and visual information of the multiple preset photos includes: The first layout prediction network processes the Laplacian matrix and edge features of each preset photo to obtain the bounding box and class label corresponding to each preset photo. Based on the bounding box and class label corresponding to each preset photo, the initial layout information of each preset photo is obtained; Based on the initial layout information and visual information of each preset photo, the target layout information corresponding to the multiple preset photos is obtained by fitting through the second layout prediction network. The step of fitting the multiple preset photos according to the target layout information using the second layout prediction network, based on the initial layout information and visual information of each preset photo, includes performing the following steps using the second layout prediction network: Based on the initial layout information and visual information of each preset photo, a target layout score is calculated for each preset photo. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and adjacent photos, and the emotional difference loss value between the preset photo and adjacent photos. The adjacent photos are other preset photos that are adjacent to the position of the preset photo in the initial layout information. Based on the target layout score of each preset photo, and combined with the correspondence between the preset layout score and fitness value, the target fitness value corresponding to the initial layout information is calculated. The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

10. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the album generation method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the album generation method as described in any one of claims 1-8.

12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the album generation method as described in any one of claims 1-8.

Images