Photo album generation method, device and equipment

Abstract

The embodiment of the invention discloses a photo album generation method, device and equipment. The method comprises the steps that in response to a photo album generation request triggered by user equipment, target information is obtained, and the target information comprises photo album setting information input by the user equipment in the photo album generation request; obtaining a plurality of preset photos corresponding to the target information from a preset photo storage space; inputting a plurality of preset photos into a photo album layout generation model; through a photo album layout generation model, according to the graph structure information and the visual information of the plurality of preset photos, performing fitting to obtain target layout information corresponding to the plurality of preset photos; and generating a photo album from the plurality of preset photos according to the target layout information. According to the embodiment of the invention, the photo album with reasonable layout can be automatically generated.

Description

Technical Field

[0001] The present application belongs to the field of voiceprint recognition technology in the field of communications, and in particular, relates to a method, device and equipment for generating an album. Background Art

[0002] With the rapid development of photography and Internet technology, using photos to freeze life moments has become an indispensable part of people's daily lives, giving life a unique "ritual sense". In order to better meet users' needs for online management and sharing of digital assets such as personal mobile phone albums and family albums, an innovative service - Story Album came into being.

[0003] A story album is an album that records and presents a specific theme or event in the form of a combination of pictures and text. Usually, a story album vividly presents a specific storyline or theme through photos, text descriptions, illustrations, etc. Story albums are often used to record special moments such as travel experiences, family activities, wedding anniversaries, and children's growth. They can also be used to tell the development process of a certain theme or event. Through story albums, people can present their memories and emotions in an interesting and creative way, leaving behind precious memories and stories.

[0004] At present, when generating story albums, users are often required to manually select photos, which cannot meet the needs of intelligence and automation. In addition, the layout of photos in current story albums is relatively monotonous, lacking variety and creativity, and lacking visual appeal, resulting in a poor user experience. Summary of the invention

[0005] The embodiments of the present application provide a photo album generation method, apparatus, device, computer storage medium and computer program product, which can automatically generate a photo album with a reasonable layout.

[0006] In a first aspect, an embodiment of the present application provides a method for generating an album, comprising:

[0007] In response to a photo album generation request triggered by a user device, acquiring target information, the target information including photo album setting information input by the user device in the photo album generation request;

[0008] Acquire multiple preset photos corresponding to the target information from a preset photo storage space;

[0009] Input multiple preset photos into the album layout generation model;

[0010] The album layout generation model is used to fit the target layout information corresponding to the multiple preset photos according to the graph structure information and visual information of the multiple preset photos;

[0011] Generate an album of multiple preset photos according to the target layout information.

[0012] In an optional implementation, the album layout generation model includes a first layout prediction network and a second layout prediction network; the graph structure information includes a Laplacian matrix and edge features corresponding to the preset photos;

[0013] The album layout generation model is used to fit the target layout information corresponding to the multiple preset photos according to the graph structure information and visual information of the multiple preset photos, including:

[0014] Processing the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain a bounding box and a class label corresponding to each preset photo;

[0015] According to the bounding boxes and class labels corresponding to the preset photos, initial layout information of the preset photos is obtained;

[0016] According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout information corresponding to the plurality of preset photos is fitted through the second layout prediction network.

[0017] In an optional implementation, a method for determining a Laplacian matrix corresponding to a preset photo includes:

[0018] Extracting subject information, background information, and decoration information of each preset photo through a first layout prediction network;

[0019] Calculate the first Laplace matrix, the second Laplace matrix and the third Laplace matrix of each preset photo respectively, wherein the first Laplace matrix is ​​calculated based on the subject information, the second Laplace matrix is ​​calculated based on the background information, and the third Laplace matrix is ​​calculated based on the decoration information;

[0020] The Laplacian matrix of each preset photo is generated according to the first Laplacian matrix, the second Laplacian matrix and the third Laplacian matrix of each preset photo.

[0021] In an optional implementation, a method for determining visual information corresponding to a preset photo includes:

[0022] Multi-scale sampling is performed on each preset photo to obtain visual information of each preset photo.

[0023] In an optional implementation, according to the initial layout information of each preset photo and the visual information of each preset photo, a plurality of preset photos are fitted according to the target layout information through a second layout prediction network, including performing the following steps through the second layout prediction network:

[0024] Calculate the target layout score of each preset photo according to the initial layout information of each preset photo and the visual information of each preset photo, wherein the target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and adjacent photos, and the emotion difference loss value between the preset photo and adjacent photos, wherein the adjacent photos are other preset photos adjacent to the preset photo position in the initial layout information;

[0025] Calculate the target fitness value corresponding to the initial layout information according to the target layout score of each preset photo and the corresponding relationship between the preset layout score and the fitness value;

[0026] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

[0027] In an optional implementation, the visual information includes character emotion information and photo color information; and according to the initial layout information of each preset photo and the visual information of each preset photo, the target layout score of each preset photo is calculated, including performing the following steps for each preset photo:

[0028] Calculating a layout position score of each preset photo according to the initial layout information of each preset photo;

[0029] Calculating a color difference loss value between the preset photo and the adjacent photos according to the photo color information of the preset photo and the photo color information of the preset photos adjacent to the preset photo;

[0030] According to the character emotion information of the preset photo and the character emotion information of the preset photos adjacent to the preset photo, the emotion difference loss value between the preset photo and the adjacent photos is calculated.

[0031] In an optional implementation, calculating the layout position score of the preset photo according to the initial layout information of the preset photo includes:

[0032] According to the initial layout information, the distance between the preset photo and the preset photo adjacent to the preset photo is determined; and according to the distance and a preset distance threshold, the layout position score of the preset photo is calculated.

[0033] In an optional implementation, in response to a user-triggered album generation request, obtaining target information includes:

[0034] In response to a request for generating an album triggered by a user, obtaining voice information input by a user device;

[0035] Recognize the voice information and obtain the target information.

[0036] In an optional implementation, after generating an album based on the target layout information for the plurality of preset photos, the method further includes:

[0037] In response to a selection operation of a user device on a plurality of preset photos, determining a target photo from the plurality of preset photos;

[0038] Acquire target description information of the target photo, where the target description information includes voice description information and / or video description information input by a user device;

[0039] Generate a target control in the album based on the target description information.

[0040] In an optional implementation, after generating the target control in the album based on the target description information, the method further includes:

[0041] In response to a selection operation of a target control by a user device, target description information is displayed.

[0042] In a second aspect, an embodiment of the present application provides a device for generating an album, including:

[0043] An acquisition module, configured to acquire target information in response to a photo album generation request triggered by a user device, wherein the target information includes photo album setting information input by the user device in the photo album generation request;

[0044] The acquisition module is further used to acquire a plurality of preset photos corresponding to the target information from a preset photo storage space;

[0045] An input module, used for inputting a plurality of preset photos into the album layout generation model;

[0046] A fitting module is used to generate a model through an album layout, and to fit target layout information corresponding to a plurality of preset photos according to the graph structure information and visual information of the plurality of preset photos;

[0047] The generation module is used to generate an album from multiple preset photos according to the target layout information.

[0048] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0049] When the processor executes the computer program instructions, it implements the photo album generation method of any optional implementation manner of the first aspect of the present application.

[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a method for generating an album according to any optional implementation of the first aspect of the present application is implemented.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a method for generating an album as in any optional implementation of the first aspect of the present application.

[0052] The album generation method, device, equipment, computer storage medium and computer program product of the embodiment of the present application can obtain target information in response to the album generation request triggered by the user device, and the target information includes the album setting information input by the user device in the album generation request. Then, multiple preset photos corresponding to the target information are obtained from the preset photo storage space. In this way, only suitable photos can be selected according to the needs of the user, without the need for the user to make manual selections, thereby improving the degree of automation of the album generation method. Subsequently, the multiple preset photos are input into the album layout generation model. Through the album layout generation model, the target layout information corresponding to the multiple preset photos is fitted according to the graph structure information and visual information of the multiple preset photos. In this way, suitable layout information can be adaptively generated for the graph structure information and visual information of the multiple preset photos, so that the album layout meets the expression requirements of the visual and storyline, and the flexibility of the album layout is improved. Then, the multiple preset photos are generated into an album according to the target layout information. In this way, the coherence and expressiveness of the album can be improved, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 It is a flowchart of a method for generating an album provided by an embodiment of the present application;

[0055] Figure 2 is a structural diagram of a photo album layout generation model provided by another embodiment of the present application;

[0056] Figure 3 This is a schematic diagram of the layout structure of an album page provided by another embodiment of the present application;

[0057] Figure 4 is a structural schematic diagram of a photo album generating device provided in yet another embodiment of the present application;

[0058] Figure 5 It is a structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0059] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0061] As described in the background technology, currently, when generating a story album, users are often required to manually select photos, which cannot meet the needs of intelligence and automation. In addition, the layout of photos in the current story album is relatively monotonous, lacking variety and creativity, and lacking visual appeal, resulting in a poor user experience.

[0062] The related technology relates to a method for generating a story album based on cloud disk storage. In this method, the user can log in to the cloud disk application (Application, APP) and select images suitable for generating a story album from the photo collection stored in the cloud disk. Then, you can click the "Recommended Story" card on the homepage to enter the next operation page. In the operation page, further select functions such as "Manage Photos", "Edit Titles", and "Save to Video Stories". If the user turns on the smart album authorization switch, the cloud disk APP can generate a story album based on the above operations of the user.

[0063] However, in the story album generation method of the above-mentioned related art, users are still required to manually select photos, etc., which lacks the characteristics of intelligence and automation. In addition, the above-mentioned related art usually arranges photos according to a fixed template, and the layout method is still relatively monotonous, lacking changes and creativity.

[0064] In view of this, the inventor, after in-depth thinking, cleverly proposed a photo album generation method, information receiving method, device, equipment, computer storage medium and computer program product.

[0065] The following is an introduction to the album generation method provided in the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings. The album generation method provided in the embodiment of the present application, the device that executes the sending may be an album generation device, or a partial module in the album generation device that is used to execute the album generation method. In the embodiment of the present application, the album generation method performed by the album generation device is taken as an example to describe in detail the album generation method provided in the embodiment of the present application.

[0066] The following is combined with Figure 1 The album generation method provided in the embodiment of the present application is described in detail.

[0067] Figure 1 FIG. 1 is a flow chart showing a method for generating an album according to an embodiment of the present application. Figure 1 As shown, the album generation method may specifically include the following steps S110 to S150.

[0068] S110, in response to the album generation request triggered by the user device, obtaining target information, where the target information includes the album setting information input by the user device in the album generation request.

[0069] In step S110, the album generation request may include an album generation request triggered by a user through an operation such as clicking or pressing, for example, an album generation request triggered by a user clicking a "recommended story" card in a cloud disk APP. Acquiring target information may include acquiring album setting information input by a user device through a user input interface, and the user input interface may include a text input interface, a voice input interface, an option input interface, etc., which are not limited here. The album setting information may include user demand information for a story album, for example, it may include information such as the theme, number of photos, type, content, layout style, etc. of the story album.

[0070] S120, acquiring a plurality of preset photos corresponding to the target information from a preset photo storage space.

[0071] In step S120, the photo storage space may include a local photo storage space, such as a local photo album; the photo storage space may also include a cloud storage space, such as a cloud photo album. The plurality of preset photos may include a plurality of preset photos that match the target information.

[0072] S130, inputting a plurality of preset photos into an album layout generation model.

[0073] S140, using an album layout generation model, fitting target layout information corresponding to the plurality of preset photos according to the graph structure information and visual information of the plurality of preset photos.

[0074] In step S140, the graph structure information may include adjacent node features and edge features of the photo. Exemplarily, the adjacent node features may include information about other photos adjacent to the photo and located above, below, left, and right of the photo, such as the Laplacian matrix corresponding to the photo. Visual information may include information conveyed visually, such as emotional information, color information, and content information of the person in the photo.

[0075] S150, generating an album based on the target layout information for the plurality of preset photos.

[0076] The album generation method of the embodiment of the present application can obtain target information in response to the album generation request triggered by the user device, and the target information includes the album setting information input by the user device in the album generation request. Then, multiple preset photos corresponding to the target information are obtained from the preset photo storage space. In this way, only suitable photos can be selected according to the needs of the user, without the need for the user to make manual selections, thereby improving the degree of automation of the album generation method. Subsequently, the multiple preset photos are input into the album layout generation model. Through the album layout generation model, the target layout information corresponding to the multiple preset photos is fitted according to the graph structure information and visual information of the multiple preset photos. In this way, suitable layout information can be adaptively generated for the graph structure information and visual information of the multiple preset photos, so that the album layout meets the expression requirements of the visual and storyline, and the flexibility of the album layout is improved. Then, the multiple preset photos are generated into an album according to the target layout information. In this way, the coherence and expressiveness of the album can be improved, thereby improving the user experience.

[0077] In one embodiment, the album layout generation model includes a first layout prediction network and a second layout prediction network. The graph structure information includes a Laplacian matrix and edge features corresponding to a preset photo.

[0078] By generating an album layout model, target layout information corresponding to multiple preset photos is obtained by fitting according to the graph structure information and visual information of multiple preset photos, which may specifically include:

[0079] The Laplacian matrix and edge features of each preset photo are processed through the first layout prediction network to obtain a bounding box and a class label corresponding to each preset photo.

[0080] According to the bounding boxes and class labels corresponding to the preset photos, the initial layout information of the preset photos is obtained.

[0081] According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout information corresponding to the plurality of preset photos is fitted through the second layout prediction network.

[0082] In the above implementation, the Laplacian matrix has a well-known meaning in the art and can be calculated by a method known in the art. As an example, the Laplacian matrix can be expressed as shown in Formula 1.

[0083] Δ=ID -1 / 2 AD -1 / 2 Formula 1

[0084] In Formula 1, Δ may represent the Laplacian matrix of the photo; I may represent the original photo; D may represent the degree matrix corresponding to the photo; and A may represent the adjacency matrix corresponding to the features of the photo.

[0085] According to the above implementation, by processing the Laplacian matrix and edge features of each preset photo through the first layout prediction network, the bounding box and class label corresponding to each preset photo can be obtained in combination with the topological structure and connectivity of each preset photo and the image change information of the photo, thereby generating the initial layout information. Then, the initial layout is further optimized in combination with the visual information of each preset photo. In this way, it is conducive to flexibly adjusting the layout of the photos according to the visual communication effect of each preset photo, so that the photos are more visually coordinated. In this way, the flexibility and rationality of the photo layout can be improved, thereby improving the visual appeal of the album, and thus improving the user experience when browsing the album.

[0086] In one embodiment, a method for determining a Laplacian matrix corresponding to a preset photo may include:

[0087] The subject information, background information and decoration information of each preset photo are extracted respectively through the first layout prediction network.

[0088] A first Laplacian matrix, a second Laplacian matrix and a third Laplacian matrix of each preset photo are calculated respectively, wherein the first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information.

[0089] The Laplacian matrix of each preset photo is generated according to the first Laplacian matrix, the second Laplacian matrix and the third Laplacian matrix of each preset photo.

[0090] In the above-mentioned embodiment, the subject information may include character information or main scenery information. The main scenery information may represent the scenery information contained in the photo that mainly presents the local scenery. Exemplarily, when the photo contains a character and the character occupies a prominent position in the photo (for example, the character is located in the central area of ​​the photo, or the percentage of the area occupied by the character in the photo is greater than a preset threshold, etc.), the subject information is the character information. When the photo does not contain a character, or the character does not occupy a prominent position in the photo, the subject information is the main scenery information. The decoration information may include information about the decorations worn by the character in the photo containing the character. The background information may include the background of the photo, for example, it may include other information in the photo except the subject information and the decoration information. For example, if a certain picture only presents the scenery of "Lijiang River" and there is no character in the photo, the scenery information of "Lijiang River" included in the photo can be considered as the main scenery information. On the contrary, if there is a character in the photo and the character occupies a prominent position in the photo, then even if the background information of the photo is the scenery of "Lijiang River", the scenery information of "Lijiang River" contained in such a photo will not be considered as the main scenery information, but will be considered as the background information.

[0091] In one example, respectively calculating the first Laplace matrix, the second Laplace matrix, and the third Laplace matrix of each preset photo may include: classifying the characters, main scenery, background information, and decoration information in each preset photo; and respectively determining the first Laplace matrix corresponding to the characters or main scenery in the preset photo, the second Laplace matrix corresponding to the background information, and the third Laplace matrix corresponding to the decoration information.

[0092] In one example, generating the Laplacian matrix of each preset photo according to the first Laplacian matrix, the second Laplacian matrix, and the third Laplacian matrix of each preset photo may include: flipping the first Laplacian matrix and the second Laplacian matrix. Based on the flipped results of the first Laplacian matrix and the second Laplacian matrix and the third Laplacian matrix, generating the Laplacian matrix of each preset photo.

[0093] According to the above implementation, by classifying and extracting the relevant information of the graph structure of the photo, the subject information, background information and decoration information are obtained. Then, the Laplace matrices corresponding to the subject information, background information and decoration information are calculated respectively, and then the Laplace matrix of the photo is generated. In this way, rich structural information can be extracted from the photo, which is conducive to generating a reasonable and coherent scene layout.

[0094] In one embodiment, a method for determining visual information corresponding to a preset photo may include:

[0095] Multi-scale sampling is performed on each preset photo to obtain visual information of each preset photo.

[0096] According to the above implementation, visual information is obtained by performing multi-scale sampling on each preset photo. In this way, the subsequent combination of visual information and target layout information is helpful to clearly express the storyline, improve the user's concentration when browsing the album, and improve the coherence and expressiveness of the album narrative.

[0097] In one embodiment, according to the initial layout information of each preset photo and the visual information of each preset photo, fitting multiple preset photos according to the target layout information through the second layout prediction network may specifically include performing the following steps through the second layout prediction network:

[0098] According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout score of each preset photo is calculated. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and adjacent photos, and the emotion difference loss value between the preset photo and adjacent photos. The adjacent photos are other preset photos adjacent to the preset photo position in the initial layout information.

[0099] According to the target layout scores of the preset photos and the corresponding relationship between the preset layout scores and the fitness values, the target fitness value corresponding to the initial layout information is calculated.

[0100] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

[0101] According to the above implementation, by calculating the layout position score of each preset photo, the suitability of the position of each preset photo in the initial layout can be measured. By calculating the color difference loss value between the preset photo and the adjacent photos, the consistency of the photo color in the initial layout can be measured. By calculating the emotion difference loss value between the preset photo and the adjacent photos, the consistency of the photo emotion in the initial layout can be measured. According to the layout position score, color difference loss value and emotion difference loss value of each preset photo, the target fitness value of the initial layout is calculated, and then the initial layout information is adjusted based on the target fitness value. In this way, the layout position, color distribution and emotion distribution of each preset photo in the final album layout can be more reasonable. In this way, the visual coordination and the consistency of the plot expression can be improved, which is conducive to improving the expressiveness and attractiveness of the album.

[0102] In one embodiment, the visual information may include character emotion information and photo color information. According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout score of each preset photo is calculated, which may specifically include performing the following steps for each preset photo:

[0103] According to the initial layout information of each preset photo, a layout position score of each preset photo is calculated.

[0104] According to the photo color information of the preset photo and the photo color information of the preset photos adjacent to the preset photo, a color difference loss value between the preset photo and the adjacent photos is calculated.

[0105] According to the character emotion information of the preset photo and the character emotion information of the preset photos adjacent to the preset photo, the emotion difference loss value between the preset photo and the adjacent photos is calculated.

[0106] According to the above-mentioned implementation, by calculating the layout position score of the preset photos through the initial layout information, the suitability of the photo layout can be quantified, which is conducive to evaluating and reasonably adjusting the position of the preset photos in the album. By calculating the color difference loss value between the preset photos and the adjacent photos, the consistency of the photo colors in the initial layout can be intuitively characterized, which is conducive to evaluating and reasonably adjusting the album layout according to the color difference loss value, thereby improving the color consistency of the album layout. By calculating the emotional difference loss value between the preset photos and the adjacent photos, the consistency of the photo emotions in the initial layout can be intuitively characterized, which is conducive to evaluating and reasonably adjusting the album layout according to the emotional difference loss value, thereby facilitating a clear and coherent expression of the storyline. In this way, the intelligence and visual appeal of album generation can be further improved.

[0107] In one embodiment, calculating the layout position score of the preset photo according to the initial layout information of the preset photo may specifically include:

[0108] According to the initial layout information, a distance between the preset photo and the preset photo adjacent to the preset photo is determined.

[0109] A layout position score of the preset photo is calculated according to the distance and a preset distance threshold.

[0110] In the above implementation, the preset distance threshold may include a preset minimum distance between adjacent photos.

[0111] According to the above implementation, the layout position score of the preset photos is calculated based on the distance between adjacent photos and the preset distance threshold, so that the suitability of the photo layout can be quantified, which is conducive to evaluating and reasonably adjusting the position of the preset photos in the album.

[0112] Exemplarily, the layout position score, the color difference loss value, and the emotion difference loss value may be calculated by the following equations 2 to 4, respectively.

[0113]

[0114] In formula 2, P i can represent the layout position score of photo i, which can be used to indicate the suitability of the layout position of photo i; D can represent the layout direction in the story album, such as east, west, south and north; T i D It indicates the actual distance between photo i and its adjacent photos in the D direction; It can represent the minimum distance between photo i and its photos in the D direction, Can be a preset value.

[0115]

[0116] In formula 3, L 色彩 (i) i can represent the color difference loss value between photo i and its adjacent photos; s[E(i)] can represent the color information of photo i; It can represent the color information of the adjacent photo j in the D direction of photo i.

[0117]

[0118] In formula 4, L 情绪 (i) can represent the emotional difference loss value between photo i and its adjacent photos; E[E(i)] can represent the emotional information of the person in photo i; It can represent the emotional information of the person in the adjacent photo j in the D direction of the photo i.

[0119] Exemplarily, after calculating the layout position score, color difference loss and emotion difference loss of each preset photo through equations 2 to 4, the target fitness value F corresponding to the initial layout can be calculated according to the target layout score of each preset photo according to the following equation 5.

[0120] F=[∑p i +L 色彩 (i)+L 情绪 (i)+1] -1 Formula 5

[0121] In one embodiment, Figure 2As shown, the album layout generation model may include a first layout prediction network 10 and a second layout prediction network 20. The first layout prediction network 10 may include a multi-head attention mechanism (Q, K, V, E), a standard layout template, a softmax function, a sum function, a concat function, a linear function linear, a feedforward neural network FFN, a residual connection layer (add), and a normalization operation layer (norm), etc. The second layout prediction network 20 may include a feature extraction network, multiple convolution layers with different convolution kernels, an upsampling layer, a pooling layer, a concat function layer, a decoding layer, etc.

[0122] In one example, a plurality of preset photos may be input into the album layout generation model. The album layout generation model may first extract subject information, background information, and decoration information in the photo based on the photo content through the first layout prediction network 10. The subject information, background information, and decoration information in each preset photo are classified, and the Laplacian matrices corresponding to the subject information, background information, and decoration information are determined respectively. The representation of the Laplacian matrix may refer to Formula 1, which is not repeated here.

[0123] Subsequently, based on the Laplace matrices corresponding to the subject information and the background information, the subject information and the background information can be flipped by a preset flip function λ. Based on the flipped result, the Laplace matrix corresponding to the decoration information can be combined to obtain the Laplace matrix of each preset photo.

[0124] Next, the Laplacian matrix and edge features of each preset photo can be processed by the first layout prediction network 10 to obtain the bounding box and class label corresponding to each preset photo. Specifically, the result Attention (Q, K, V, E) of each preset photo under the multi-head attention mechanism can be determined by the multi-head attention mechanism. According to the result output by the multi-head attention mechanism, the bounding box and class label of each preset photo are obtained. The definition of the multi-head attention mechanism can be found in Formula 6.

[0125]

[0126] In formula 6, can represent a learnable projection matrix; k can represent the number of heads of the multi-head attention mechanism, k∈{0,…,N heads};h i It can represent the feature vector of photo i; h j can represent the feature vector of photo j; e ij can represent the edge features of the photo; d can represent the number of photo categories; d k The number of categories of associated photos of a specific category can be represented, which can be calculated by Formula 7.

[0127]

[0128] Next, you can layout the template parameters according to the bounding box and class label of each photo and the preset specification, and then follow Figure 2 The content shown in the figure is successively processed by the softmax function, the sum function, the concat function, the linear function, the feedforward neural network FFN, the residual connection layer and the layer normalization operation layer, and then the initial layout information of the photo is obtained. The initial layout information can be expressed by formula 8.

[0129] L layout =L box +L label +L iou Formula 8

[0130] In formula 8, L layout It can represent the initial layout information of the photo; L box It can represent the loss of coordinates on the bounding box of the photo; L label It can represent the cross entropy on the labels of the photos; L iou It can represent the distance loss of the bounding box of the photo from the border of the story album layout page.

[0131] Then, the initial layout information can be passed to the 1*1conv layer in the second layout prediction network 20. The 1*1conv layer can perform a convolution operation based on the visual information such as the emotional information of the characters in the photo, the color information of the photo, the content information of the photo, and the initial layout information of the photo obtained by the feature extraction network. Then, the result of the convolution operation is upsampled to obtain a first result. The decoding layer in the second layout prediction network 20 performs a convolution operation based on multiple preset photos to obtain a second result. Then, the first result and the second result can be fused through the concat function. After that, a 3*3 convolution operation is performed on the fused result, and then a secondary upsampling operation is performed to obtain the prediction result of the final layout of each preset photo in the story album, that is, the target layout information corresponding to multiple preset photos is obtained.

[0132] Specifically, the second layout prediction network 20 can combine the initial layout information and the emotional information, color information, and content information extracted by the feature extraction network to calculate the layout position score, color difference loss value, and emotional difference loss value between the preset photos in adjacent positions in the initial layout. The specific calculation process can be shown in Formulas 2 to 4 above, which will not be repeated here. Then, the target fitness value F corresponding to the initial layout can be calculated with reference to Formula 5. When the target fitness value F is greater than or equal to the preset fitness threshold, the target layout information of the story album is output. Otherwise, the model parameters of the second layout prediction network 20 are iteratively updated according to the color difference loss value, emotional difference loss value, and layout position score between the photos in adjacent positions in the initial layout to adjust the initial layout information until the calculated target fitness value is greater than or equal to the preset fitness threshold, then the update is stopped and the target layout information of the story album is output.

[0133] In one embodiment, in response to a user-triggered album generation request, obtaining target information may specifically include:

[0134] In response to a request for generating an album triggered by a user, voice information input by a user device is obtained.

[0135] Recognize the voice information and obtain the target information.

[0136] Exemplarily, after the user triggers the album generation request, a voice input window may be displayed to the user. The voice input window may include a voice input device symbol, such as a microphone logo. The user may click or press the voice input device symbol to feedback to the server his or her setting requirements for the story album. For example, the user may click the voice input device symbol and then say, "Please find 5 funny single-person photos to generate a funny and happy story album." Or, you may say, "Please generate a travel story album based on the photos taken in XXX, with more group photos and fewer single-person photos in the story album," and so on.

[0137] In the above implementation, the recognition of voice information can be achieved by methods known in the art, which will not be described in detail here. For example, key information such as the theme, number, type, content, layout style, etc. of the story album can be extracted from the voice information by voice recognition technology to obtain target information.

[0138] According to the above implementation, the user describes the needs of the story album by voice, which can convert the user's intention into text form more quickly than the manual input method, avoiding the tedious process of manual input. In addition, through the voice description method, the user only needs to describe the needs to the cloud server, and then the cloud server can select appropriate photos according to the needs described by the user and generate a story album that matches the needs. In this way, the user no longer needs to manually select photos one by one, which can significantly improve the intelligence and automation of the story album generation process, thereby improving the efficiency of album generation and user experience.

[0139] In one embodiment, after generating an album with multiple preset photos according to the target layout information, the method may further include:

[0140] In response to a selection operation of a plurality of preset photos by a user device, a target photo is determined from the plurality of preset photos.

[0141] Target description information of the target photo is obtained, where the target description information includes voice description information and / or video description information input by a user device.

[0142] Generate a target control in the album based on the target description information.

[0143] In the related art, in order to record some wonderful moments, text descriptions are usually added to photos in the story album. When the story behind the wonderful moment is long, a large amount of text may be needed to describe it, which occupies a large area of ​​the photo, destroying the beauty and simplicity of the photo, affecting the overall visual effect, and making the photo look messy or chaotic.

[0144] In the above implementation, the target photo may be a photo selected by the user and corresponding to a wonderful moment. The target control may include a button or other component associated with the target description information. The target control may be located at any position in the album. For example, the target control may be located at the border of the target photo, or on the target photo. The target control may also be located at other positions in the album and associated with the target photo through an icon or a line.

[0145] According to the above implementation, the target control can be used to replace the text description, thereby avoiding the problem of distracting the viewer of the story album and reducing the visual effect and attractiveness of the photo by adding too much text to describe the story behind the wonderful photo. In addition, by replacing the text description with the target control, it is also possible to avoid excessive text descriptions from destroying the beauty and simplicity of the photo, thereby improving the overall visual effect of the album.

[0146] In one embodiment, after generating a target control in the album based on the target description information, the method may further include:

[0147] In response to a selection operation of a target control by a user device, target description information is displayed.

[0148] In the above implementation, the user device may include any user device with album viewing permission. The method of displaying the target description information may be selected according to the form of the target description information. For example, when the target description information is voice information, the target description information may be displayed by playing the audio of the target description information; when the target description information is video information, the target description information may be displayed by jumping to the video playback window through the video playback window.

[0149] According to the above implementation, the target description information can be displayed in response to the user device selecting the target control, so that the story of the album can be enriched while maintaining the simplicity and aesthetics of the album.

[0150] For ease of understanding, the above implementation method will be described in detail below by taking the generation of “wonderful moments” in an album as an example.

[0151] In one example, after generating an album, the user can select a "wonderful moment" photo from the album, and then enter a description of the actual situation corresponding to the "wonderful moment". The cloud server can generate an audio story section or a video story section from the description content, and embed the audio story section or the video story section into the appropriate position of the relevant photos in the album. When the user subsequently browses the story album, if he wants to know the story behind the "wonderful moment" photo, he can trigger the audio story section or the video story section corresponding to the "wonderful moment" photo to listen to or view the story behind it.

[0152] The above-mentioned wonderful moment photos can be defined by the user himself, for example, they can be unforgettable moments, happiest moments, saddest moments, etc. recorded by the photos, which are not limited here.

[0153] As an example, Figure 3 As shown, a page of an album may include four photos, namely, Photo 1, Photo 2, Photo 3, and Wonderful Moment Photos. Wonderful Moment Photos are photos of users "crying with laughter". In actual applications, when the album is generated and the user browses the photo, it is usually easy for the user to think of why they "crying with laughter" at that time. However, as time goes by, the user may forget why they "crying with laughter" at that time, or when the user shares the story album with other third parties, the third party may also be curious about why the user "crying with laughter" at that time. In order to avoid this situation, an audio story section or a video story section can be generated for the wonderful moment photos so that the story behind the moment can be understood later. For example, as Figure 3 As shown, the audio story section or the video story section can be set on the wonderful moment photo in the form of a control 100.

[0154] After the control 100 is generated, when browsing the story album, the user can click the control 100 to jump to the video playback window, play the story behind the wonderful moment photo, or play the corresponding audio story to explain the story behind the wonderful moment photo. In this way, on the one hand, it is possible to avoid adding too much text to describe the story behind the wonderful photo, thereby enhancing the visual effect and attractiveness of the photo and reducing the risk of distracting the viewer's attention. On the other hand, it is also possible to avoid the problem that too much text description destroys the beauty and simplicity of the photo, affects the overall visual effect, and makes the photo appear messy or chaotic. Furthermore, some viewers may not be willing to spend time and energy to read a large amount of text descriptions, especially when browsing photos, and are more inclined to intuitively appreciate the information conveyed by the photo. By adding controls, not only can excessive reliance on text to describe the content of the photo be avoided, but viewers can also choose whether to view the story behind the wonderful moment. In this way, the user experience can be further improved.

[0155] Based on the same inventive concept as the photo album generation method, the embodiment of the present application also provides a photo album generation device.

[0156] like Figure 4 As shown, the album generating device 200 may include a first acquiring module 201 , an input module 202 , a fitting module 203 and a first generating module 204 .

[0157] The first acquisition module 201 is used to acquire target information in response to a photo album generation request triggered by a user device, where the target information includes photo album setting information input by the user device in the photo album generation request.

[0158] The first acquisition module 201 is further used to acquire a plurality of preset photos corresponding to the target information from a preset photo storage space.

[0159] The input module 202 is used to input a plurality of preset photos into the album layout generation model.

[0160] The fitting module 203 is used to fit the target layout information corresponding to the plurality of preset photos according to the graph structure information and visual information of the plurality of preset photos through the album layout generation model.

[0161] The first generating module 204 is used to generate an album from a plurality of preset photos according to target layout information.

[0162] The album generation device of the embodiment of the present application can obtain target information in response to the album generation request triggered by the user device, and the target information includes the album setting information input by the user device in the album generation request. Then, multiple preset photos corresponding to the target information are obtained from the preset photo storage space. In this way, only suitable photos can be selected according to the needs of the user, without the need for the user to make manual selections, thereby improving the degree of automation of the album generation method. Subsequently, the multiple preset photos are input into the album layout generation model. Through the album layout generation model, the target layout information corresponding to the multiple preset photos is fitted according to the graph structure information and visual information of the multiple preset photos. In this way, suitable layout information can be adaptively generated for the graph structure information and visual information of the multiple preset photos, so that the album layout meets the expression requirements of the visual and storyline, and the flexibility of the album layout is improved. Then, the multiple preset photos are generated into an album according to the target layout information. In this way, the coherence and expressiveness of the album can be improved, thereby improving the user experience.

[0163] In one embodiment, the album layout generation model may include a first layout prediction network and a second layout prediction network. The graph structure information may include a Laplacian matrix and edge features corresponding to a preset photo.

[0164] The fitting module is used to generate a model through the album layout, and to fit the target layout information corresponding to the multiple preset photos according to the graph structure information and visual information of the multiple preset photos, which may specifically include:

[0165] The processing submodule is used to process the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain a bounding box and a class label corresponding to each preset photo.

[0166] The processing submodule is further used to obtain initial layout information of each preset photo according to the bounding box and class label corresponding to each preset photo.

[0167] The fitting submodule is used to fit the target layout information corresponding to the plurality of preset photos through the second layout prediction network according to the initial layout information of each preset photo and the visual information of each preset photo.

[0168] In one embodiment, the apparatus may further include:

[0169] The extraction module is used to extract the subject information, background information and decoration information of each preset photo through the first layout prediction network before processing the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain the bounding box and class label corresponding to each preset photo.

[0170] The calculation module is used to calculate the first Laplace matrix, the second Laplace matrix and the third Laplace matrix of each preset photo respectively, wherein the first Laplace matrix is ​​calculated based on the subject information, the second Laplace matrix is ​​calculated based on the background information, and the third Laplace matrix is ​​calculated based on the decoration information.

[0171] The second generating module is used to generate the Laplacian matrix of each preset photo according to the first Laplacian matrix, the second Laplacian matrix and the third Laplacian matrix of each preset photo.

[0172] In one embodiment, the apparatus may further include:

[0173] The sampling module is used to perform multi-scale sampling on each preset photo to obtain the visual information of each preset photo before fitting multiple preset photos according to the target layout information through the second layout prediction network based on the initial layout information of each preset photo and the visual information of each preset photo.

[0174] In one embodiment, the fitting submodule is used to fit the target layout information of multiple preset photos through the second layout prediction network according to the initial layout information of each preset photo and the visual information of each preset photo. Specifically, it may include: the fitting submodule is used to perform the following steps through the second layout prediction network:

[0175] According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout score of each preset photo is calculated. The target layout score includes the layout position score of the preset photo, the color difference loss value between the preset photo and adjacent photos, and the emotion difference loss value between the preset photo and adjacent photos. The adjacent photos are other preset photos adjacent to the preset photo position in the initial layout information.

[0176] According to the target layout scores of the preset photos and the corresponding relationship between the preset layout scores and the fitness values, the target fitness value corresponding to the initial layout information is calculated.

[0177] The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

[0178] In one embodiment, the visual information may include character emotion information and photo color information. The fitting submodule is used to calculate the target layout score of each preset photo according to the initial layout information of each preset photo and the visual information of each preset photo through the second layout prediction network, and specifically may include: the fitting submodule is used to perform the following steps for each preset photo through the second layout prediction network:

[0179] According to the initial layout information of each preset photo, a layout position score of each preset photo is calculated.

[0180] According to the photo color information of the preset photo and the photo color information of the preset photos adjacent to the preset photo, a color difference loss value between the preset photo and the adjacent photos is calculated.

[0181] According to the character emotion information of the preset photo and the character emotion information of the preset photos adjacent to the preset photo, the emotion difference loss value between the preset photo and the adjacent photos is calculated.

[0182] In one embodiment, the fitting submodule is used to calculate the layout position score of each preset photo according to the initial layout information of the preset photo through the second layout prediction network, which may specifically include:

[0183] According to the initial layout information, the distance between the preset photo and the preset photo adjacent to the preset photo is determined, and the layout position score of the preset photo is calculated according to the distance and a preset distance threshold.

[0184] In one embodiment, the acquisition module is used to respond to the album generation request triggered by the user and acquire the target information, which may specifically include:

[0185] The acquisition submodule is used to obtain the voice information input by the user device in response to the album generation request triggered by the user.

[0186] The recognition submodule is used to recognize the voice information and obtain the target information.

[0187] In one embodiment, the apparatus may further include:

[0188] The determination module is used to determine a target photo from the plurality of preset photos in response to a selection operation of the plurality of preset photos by a user device after the plurality of preset photos are generated into an album according to the target layout information.

[0189] The second acquisition module acquires target description information of the target photo, where the target description information includes voice description information and / or video description information input by the user device.

[0190] The third generating module is used to generate a target control in the album based on the target description information.

[0191] In one embodiment, the apparatus may further include:

[0192] The display module is used to display the target description information in response to a selection operation of the target control by the user device after generating the target control in the album based on the target description information.

[0193] The photo album generation device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0194] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0195] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0196] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0197] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 302 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0198] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0199] The processor 301 implements any one of the photo album generation methods in the above embodiments by reading and executing computer program instructions stored in the memory 302 .

[0200] As an example, the electronic device may further include a communication interface 303 and a bus 310. Figure 5 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.

[0201] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0202] Bus 310 includes hardware, software or both, and the parts of album generation equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0203] The electronic device can execute the album generation method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 4 The described method and device for generating a photo album.

[0204] In addition, in combination with the album generation method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the album generation methods in the above embodiment is implemented.

[0205] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, implements any one of the methods for generating an album in the above embodiments.

[0206] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0207] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0208] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0209] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0210] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for generating a photo album, characterized in that: include: In response to a photo album generation request triggered by a user device, acquiring target information, the target information including photo album setting information input by the user device in the photo album generation request; Acquire a plurality of preset photos corresponding to the target information from a preset photo storage space; Inputting the plurality of preset photos into an album layout generation model; By using the album layout generation model, according to the graph structure information and visual information of the plurality of preset photos, target layout information corresponding to the plurality of preset photos is fitted; The plurality of preset photos are used to generate an album according to the target layout information.

2. The method according to claim 1, characterized in that: The album layout generation model includes a first layout prediction network and a second layout prediction network; the graph structure information includes a Laplacian matrix and edge features corresponding to preset photos; The step of fitting target layout information corresponding to the plurality of preset photos by the album layout generation model according to the graph structure information and visual information of the plurality of preset photos includes: Processing the Laplacian matrix and edge features of each preset photo through the first layout prediction network to obtain a bounding box and a class label corresponding to each preset photo; According to the bounding boxes and class labels corresponding to the preset photos, initial layout information of the preset photos is obtained; According to the initial layout information of each preset photo and the visual information of each preset photo, the target layout information corresponding to the plurality of preset photos is fitted through the second layout prediction network.

3. The method according to claim 2, characterized in that The method for determining the Laplacian matrix corresponding to the preset photo includes: Extracting subject information, background information, and decoration information of each preset photo through the first layout prediction network; Calculating a first Laplacian matrix, a second Laplacian matrix, and a third Laplacian matrix of each preset photo respectively, wherein the first Laplacian matrix is ​​calculated based on the subject information, the second Laplacian matrix is ​​calculated based on the background information, and the third Laplacian matrix is ​​calculated based on the decoration information; The Laplacian matrix of each preset photo is generated according to the first Laplacian matrix, the second Laplacian matrix and the third Laplacian matrix of each preset photo.

4. The method according to claim 2, characterized in that: The method for determining the visual information corresponding to the preset photo includes: Multi-scale sampling is performed on each preset photo to obtain visual information of each preset photo.

5. The method according to claim 2, characterized in that: The fitting of the plurality of preset photos according to the target layout information is performed by the second layout prediction network according to the initial layout information of each preset photo and the visual information of each preset photo, including performing the following steps by the second layout prediction network: Calculate a target layout score for each preset photo according to the initial layout information of each preset photo and the visual information of each preset photo, wherein the target layout score includes a layout position score of the preset photo, a color difference loss value between the preset photo and adjacent photos, and a mood difference loss value between the preset photo and adjacent photos, wherein the adjacent photos are other preset photos adjacent to the preset photo position in the initial layout information; Calculate the target fitness value corresponding to the initial layout information according to the target layout score of each preset photo and the corresponding relationship between the preset layout score and the fitness value; The initial layout information is adjusted according to the target fitness value until the target fitness value is greater than or equal to a preset fitness threshold, thereby obtaining the target layout information.

6. The method according to claim 5, characterized in that The visual information includes character emotion information and photo color information; and according to the initial layout information of each preset photo and the visual information of each preset photo, a target layout score of each preset photo is calculated, including performing the following steps for each preset photo: Calculating a layout position score of each preset photo according to the initial layout information of each preset photo; Calculating a color difference loss value between the preset photo and the adjacent photos according to the photo color information of the preset photo and the photo color information of the preset photos adjacent to the preset photo; According to the character emotion information of the preset photo and the character emotion information of the preset photos adjacent to the preset photo, the emotion difference loss value between the preset photo and the adjacent photos is calculated.

7. The method according to claim 6, characterized in that The calculating the layout position score of the preset photo according to the initial layout information of the preset photo includes: According to the initial layout information, the distance between the preset photo and a preset photo adjacent to the preset photo is determined; and according to the distance and a preset distance threshold, the layout position score of the preset photo is calculated.

8. The method according to any one of claims 1 to 7, characterized in that The step of obtaining target information in response to a request for generating an album triggered by a user includes: In response to a request for generating an album triggered by a user, obtaining voice information input by a user device; The voice information is recognized to obtain the target information.

9. The method according to any one of claims 1 to 7, characterized in that: After generating an album with the plurality of preset photos according to the target layout information, the method further includes: In response to a selection operation of the plurality of preset photos by a user device, determining a target photo from the plurality of preset photos; Acquire target description information of the target photo, where the target description information includes voice description information and / or video description information input by the user device; Based on the target description information, a target control is generated in the photo album.

10. The method according to claim 9, characterized in that After generating a target control in the album based on the target description information, the method further includes: In response to a selection operation of the target control by the user device, the target description information is displayed.

11. A device for generating a photo album, characterized in that: include: An acquisition module, configured to acquire target information in response to a photo album generation request triggered by a user device, wherein the target information includes photo album setting information input by the user device in the photo album generation request; The acquisition module is further used to acquire a plurality of preset photos corresponding to the target information from a preset photo storage space; An input module, used for inputting the plurality of preset photos into the album layout generation model; A fitting module, configured to fit target layout information corresponding to the plurality of preset photos through the album layout generation model according to the graph structure information and visual information of the plurality of preset photos; A generating module is used to generate an album from the plurality of preset photos according to the target layout information.

12. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for determining the generation of an album as described in any one of claims 1-10 is implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method for generating an album according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the photo album generation method as described in any one of claims 1-10.

Images