Microbrochure automatic generation method, device and equipment and storage medium
Generate copywriting, background pictures and background music of electronic brochures through pre-trained models, and combine cosine similarity to filter materials and templates, solving the problems of low manual design efficiency and high error rate, and achieving efficient and personalized micro-book generation .
Patent Information
- Application Number
- CN202411151459.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-12
AI Technical Summary
The creation of existing e-brands mainly relies on manual design and production, resulting in low design efficiency and high error rate.
The pre-trained text generation model, picture generation model and audio generation model are used to generate microbill copy, background image and background music based on the user's input needs, and the candidate materials and templates are determined through cosine similarity calculation. After the user selects, the microblock will be automatically generated. book.
It reduces manual operations, improves the efficiency of micro-book generation, reduces the error rate, and realizes personalized micro-book generation.
Smart Images

Figure CN118945437A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing, and more specifically, to a method, device, equipment and storage medium for automatically generating a micro-library. Background Art
[0002] With the development of Internet technology and the popularity of mobile devices, the dissemination of information is no longer limited to traditional paper brochures. Electronic brochures have an important position in modern marketing and publicity due to their advantages such as easy updating and dissemination, strong interactivity, environmental protection and energy saving. Electronic brochures can be used for exhibitions, publicity, sample displays, etc. Electronic brochures display and disseminate multiple elements such as text, pictures, videos, audio, animation, etc. through digital media (such as computers, tablets, smart phones, etc.).
[0003] At present, the creation of electronic brochures mainly relies on manual design and production. First, the relevant text, audio and video materials are collected manually, and then the pages of the electronic brochure are manually produced. The manual design method is inefficient and has a high error rate. Summary of the invention
[0004] In view of this, the present invention provides a method, device, equipment and storage medium for automatically generating a micro-album, which is used to solve the problem that the existing method mainly relies on manual design and production, resulting in low design efficiency and high error rate.
[0005] To achieve the above objectives, the proposed solution is as follows: A method for automatically generating a micro-book library, comprising: Obtain the material file set and requirement text entered by the user, and vectorize and store each material file; The pre-trained text generation model, image generation model, and audio generation model generate micro-book copy, micro-book background image, and micro-book background music respectively according to the required text; Calculate the cosine similarity between the required text and each material file, and determine the candidate material file set based on the cosine similarity; Calculate the cosine similarity between the requirement text and each template, and determine the candidate template set based on the cosine similarity; The candidate material file set and the candidate template set are displayed respectively, so that the user can select the target material and the target template according to the design requirements; A micro-booklet set is generated based on the micro-booklet copy, the micro-booklet background image, the micro-booklet background music, the target material and the target template, wherein the micro-booklet set includes at least one micro-booklet.
[0006] Preferably, before calculating the cosine similarity between the requirement text and each material file, the method further includes: Preprocess the demand text to obtain a set of text terms; Based on the word set, the TF-IDF vectors of the requirement text, each material file and template are calculated respectively.
[0007] Preferably, each material file is vectorized and stored, including: Identify the file type of each material file in the material file set, where the file type includes audio, video, picture and text; Extract file information from corresponding material files according to file type; Vectorize and store file information; Preferably, extracting file information from a corresponding source file according to the file type includes: If the file type is audio, extract the music information and voice information in the material file, and convert the voice information into text information; If the file type is a picture, the text information and picture content information in the source file are extracted; If the file type is video, the audio in the source file is extracted, and each frame of the source file is sampled to obtain the audio file and the image corresponding to each frame, and the file information in the audio file and the image of each frame is extracted according to the file type.
[0008] Preferably, it also includes: Display the micro-book; In response to the user's editing operation, the micro-booklet copy, micro-booklet background image, micro-booklet background music and micro-booklet layout of the micro-booklet are edited.
[0009] Preferably, the text generation model is a Transformer model; The image generation model is the Stable Diffusion model; The audio generation model is the Stable Audio Open model.
[0010] Preferably, it also includes: An identification code is generated based on the micro-book, so that other users can view the micro-book content by scanning the identification code.
[0011] A device for automatically generating a micro-book library, comprising: A text acquisition unit, used to acquire a set of material files and a required text input by a user, and vectorize and store each material file; A content generation unit is used to generate micro-book copy, micro-book background image and micro-book background music according to the required text based on the pre-trained text generation model, image generation model and audio generation model; A material file screening unit, used to calculate the cosine similarity between the requirement text and each material file, and determine a candidate material file set according to the cosine similarity; The template screening unit is used to calculate the cosine similarity between the requirement text and each template, and determine the candidate template set according to the cosine similarity; A display unit, used to display the candidate material file set and the candidate template set respectively, so that the user can select the target material and the target template according to the design requirements; The micro-booklet generating unit is used to generate a micro-booklet set based on the micro-booklet copy, micro-booklet background picture, micro-booklet background music, target material and target template, wherein the micro-booklet set includes at least one micro-booklet.
[0012] A micro-book library automatic generation device, comprising: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the aforementioned micro-library automatic generation method.
[0013] A storage medium stores a computer program, which, when executed by a processor, implements the various steps of the aforementioned micro-library automatic generation method.
[0014] It can be seen from the above technical scheme that the micro-book library automatic generation method provided by the present invention generates micro-book copy, micro-book background image and micro-book background music according to the demand text through the text generation model, the picture generation model and the audio generation model. The cosine similarity between the demand text and each material file, and between the demand text and each template is calculated, and then the candidate material file set and the candidate template set are determined according to the cosine similarity. The user selects the target material and the target template according to the design requirements. The micro-book is generated based on the micro-book copy, the micro-book background image, the micro-book background music, the target material and the target template. In the micro-book library automatic generation method of the present invention, the user only needs to input the material file and the demand, and select the material file and the template, and the copy, background image and background music can be automatically generated by the pre-trained model, and then the micro-book is automatically generated based on the material and template selected by the user. Manual operation is reduced, the generation efficiency of the micro-book is improved, and the error rate in the micro-book generation process is reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.
[0016] Figure 1 A flowchart of a method for automatically generating a micro-book library provided in this embodiment; Figure 2 A schematic diagram of the structure of a micro-book library automatic generation device provided in this embodiment; Figure 3 A hardware structure block diagram of a micro-library automatic generation device provided in this embodiment. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0018] The micro-book library includes at least one column, and each column includes at least one micro-book. Figure 1 A method for automatically generating a micro-book library provided in this embodiment is introduced. Figure 1 As shown, the method includes: Step S01, obtaining a set of material files and a requirement text input by a user, and vectorizing and storing each material file.
[0019] Specifically, the user can input the requirement text of the micro-book design and the collection of collected material files, wherein the material file collection includes at least one material file. The material files can be file types such as audio, video, picture and text. For example: If you want to design a micro-book for a school, the user can input a collection of material files containing school information and education field data, as well as the specific requirement text of the micro-book "Generate a micro-book product for an online campus open day, and give a comprehensive, illustrated introduction to the school. It is required to include sections such as school philosophy, major events, discipline construction, grade growth, teaching and research, school history exploration, home-school co-education, and achievements collection. The page style is Chinese style".
[0020] Each material file in the material file set input by the user is vectorized and stored. First, the file type of each material file in the material file set is identified. Then, the file information in the corresponding material file is extracted according to the file type. Finally, the file information is vectorized and stored. Among them, the file types include audio, video, picture and text.
[0021] Based on this, the process of extracting file information from the corresponding material file according to the file type is as follows: If the file type is audio, music recognition and voice recognition are performed on the material file to extract the music information and voice information in the material file, and the voice information is converted into text information in the form of text; If the file type is a picture, the text information in the picture can be extracted by optical character recognition, and / or target detection can be performed on the material file to extract the picture content information of the material file, and the picture content information can be converted into text information; If the file type is a video, the audio in the source file is extracted, and each frame of the source file is sampled to obtain the audio file and the image corresponding to each frame, and the file information in the audio file and the image of each frame is extracted according to the file type to obtain the text information of the source file. That is, music recognition and speech recognition are performed on the audio file, the music information and speech information in the audio file are extracted, and the speech information is converted into text information in the form of text; the text information in each image is extracted through optical character recognition, and / or target detection is performed on each image, the picture content information of each image is extracted, and converted into text information.
[0022] Step S02, the pre-trained text generation model, image generation model, and audio generation model generate micro-book copy, micro-book background image, and micro-book background music respectively according to the required text.
[0023] Specifically, the pre-trained model can be used to automatically generate the text, background image and background music of the micro-book. The text generation model, the image generation model and the audio generation model are used to generate the micro-book text, the micro-book background image and the micro-book background music of the micro-book according to the input demand text. Among them, the text generation model can be a Transformer model, the image generation model is a Stable Diffusion model, and the audio generation model is a Stable Audio Open model.
[0024] Step S03: Calculate the cosine similarity between the requirement text and each material file, and determine a candidate material file set according to the cosine similarity.
[0025] Specifically, the cosine similarity between the requirement text and the vectorized stored material files is calculated. Based on the cosine similarity, a set of candidate material files that are closer to the user's design requirements is determined. Since the video and audio type material files are voice recognized before storage, the audio and video type materials can also be retrieved through the keywords in the requirement text. Therefore, the type of candidate material files can be any of pictures, audio, video, and text. For example: the first N material files with the highest similarity can be used as candidate materials; materials with a similarity higher than M can also be used as candidate materials.
[0026] Step S04, calculating the cosine similarity between the requirement text and each template, and determining a candidate template set according to the cosine similarity.
[0027] Specifically, similar to the aforementioned process of obtaining a set of candidate material files, a plurality of candidate templates having a high similarity to the requirement text input by the user are determined according to the calculated cosine similarity.
[0028] Step S05: display the candidate material file set and the candidate template set respectively.
[0029] Specifically, the screened candidate material file set and candidate template set are displayed respectively, and the user selects suitable candidate material files and candidate templates as target materials and target templates according to the displayed contents.
[0030] Step S06, generating a micro-booklet set based on the micro-booklet copy, micro-booklet background image, micro-booklet background music, target material and target template.
[0031] Specifically, the micro-booklet copy, micro-booklet background image, micro-booklet background music and target material are filled into the corresponding positions in the target template to generate a micro-booklet. One or more micro-booklets are generated according to the required text input by the user to obtain a micro-booklet set. For example, the HTML5 web page code of the target template can be copied, and the copy, picture, background music and material files in the web page code can be replaced by the micro-booklet copy, micro-booklet background image, micro-booklet background music and target material to obtain a micro-booklet set that meets the user's design requirements.
[0032] The micro-book library automatic generation method provided in this embodiment generates micro-book copy, micro-book background image and micro-book background music according to the demand text through a text generation model, a picture generation model and an audio generation model. The cosine similarity between the demand text and each material file, and between the demand text and each template is calculated, and then the candidate material file set and the candidate template set are determined according to the cosine similarity. The user selects the target material and the target template according to the design requirements. The micro-book is generated based on the micro-book copy, the micro-book background image, the micro-book background music, the target material and the target template. In the micro-book library automatic generation method of the present invention, the user only needs to input the material file and the requirements, and select the material file and the template, and the copy, background image and background music can be automatically generated by the pre-trained model micro-book library, and then the micro-book is automatically generated based on the material and template selected by the user. Manual operation is reduced, the generation efficiency of the micro-book is improved, and the error rate in the micro-book generation process is reduced.
[0033] The micro-booklet automatic generation method provided in this embodiment generates a corresponding micro-booklet in a personalized manner based on the target material and target candidate template selected by the user, and the target material is a material file in a material file set collected by the user according to the requirements, not a material file stored in the original database. This avoids the situation where the contents of the generated micro-booklets are relatively similar, and improves the personalization of the generated micro-booklets.
[0034] Next, this embodiment introduces the process of generating corresponding contents in each model in step S02, where the pre-trained text generation model, picture generation model, and audio generation model generate micro-book copy, micro-book background image, and micro-book background music according to the required text. The process is as follows: 1. The process of generating micro-book copy includes: 1. Extract features from the requirement text; 2. The text generation model generates corresponding micro-book copy based on the features.
[0035] 2. The process of generating the micro-book background image includes: 1. Text processing: Perform natural language processing operations such as word segmentation and part-of-speech tagging on the input demand text to extract key text information; 2. Feature extraction: convert the processed text information into vector representation as input for the image generation model; 3. Image generation: The image generation model converts the vector representation into image data and generates a micro-book background image that matches the text description.
[0036] 3. The process of generating micro album background music includes: 1. Generate text prompts for the audio generation model based on the required text; 2. Generate the corresponding micro-book background music according to the content of the text prompt.
[0037] Furthermore, this embodiment introduces the training process of each model, and the process is as follows: 1. The training process of the text generation model is as follows: The collection of material files provided by the user is used as a data set, and the collected data is cleaned. The initial parameters of the model are set, including the learning rate, number of iterations, etc. First, a general Chinese data set is used to pre-train the text generation model and generate training texts. The model effect of the text generation model is evaluated based on the training texts. Then, based on the model effect, all parameters of the text generation model are fine-tuned. The fine-tuned text generation model generates training texts again, and the model effect after fine-tuning all parameters is evaluated based on the training texts. Finally, based on the model effect after parameter fine-tuning, the instructions are fine-tuned, and the training texts are generated using the fine-tuned instructions. The model effect after fine-tuning the instructions is evaluated based on the training texts, and the training is stopped when the text generation model reaches the preset model effect. For example: If the text generation model is used to generate the school's micro-booklet copy, the data set can be the school information and education data entered by the user.
[0038] 2. The training process of the image generation model is as follows: First, fine-tune the image generation model. The fine-tuning process includes: preparing the data set, selecting the basic model, configuring the training environment, setting the training parameters, starting training, adjusting the training strategy, evaluating the model performance, and iterating optimization.
[0039] 3. The training process of the audio generation model is as follows: First, convert the custom audio samples into a format supported by the model, and perform necessary preprocessing operations such as noise reduction and standardization to obtain a data set of audio samples. Then, prepare corresponding text prompts based on the content of the audio samples, and configure the fine-tuning parameters of the audio generation model according to the task requirements and hardware conditions, including learning rate, batch size, number of iterations, etc. Then, load the preprocessed audio samples and text prompts into the audio generation model, and let the audio generation model be trained on the data set of custom audio samples. During the training process, the training effect of the audio generation model can be judged by observing the training curve and evaluation indicators. Finally, after fine-tuning, use the validation set to evaluate the audio generation model. According to the evaluation results, adjust the fine-tuning parameters and optimize the model.
[0040] This embodiment trains the text generation model, the image generation model and the audio generation model. The trained model can generate personalized micro-book copy, micro-book background image and micro-book background music according to the needs of users.
[0041] On the basis of the above-mentioned embodiment, before calculating the cosine similarity between the requirement text and each material file in step S03, the following steps may be further performed: Preprocess the demand text to obtain a set of text terms.
[0042] Based on the word set, the TF-IDF vectors of the requirement text, each material file and template are calculated respectively.
[0043] Specifically, preprocessing is performed based on the demand text entered by the user. The preprocessing process includes operations such as word segmentation, stop word removal, and stem extraction, and the demand text is represented as a set of terms. Then, based on the words in the set of terms, the TF-IDF values of the demand text, each material file, and the template are calculated respectively to obtain the TF-IDF vector of the demand text, the TF-IDF vector of each material file, and the TF-IDF vector of each template.
[0044] Based on this, the cosine similarity between the requirement text and each material file, and the cosine similarity between the requirement text and each template can be calculated based on the TF-IDF vector of the requirement text, the TF-IDF vector of each material file, and the TF-IDF vector of each template.
[0045] In addition, after generating the micro-booklet set in step S06, you can also: Display the micro-book. In response to the user's editing operation, edit the micro-book copy, micro-book background image, micro-book background music and micro-book layout of the micro-book. The micro-book can be displayed through a WYSIWYG visual editing interface. Users can modify the micro-book copy, micro-book background image, micro-book background music and layout through this page. Users can also preview the display effect of the modified micro-book at any time.
[0046] In addition, an identification code can be generated based on the micro-booklet so that other users can view the content of the micro-booklet by scanning the identification code. Generate an identification code of a type such as a QR code or a barcode corresponding to the micro-booklet. Users can use electronic devices to scan the corresponding identification code to view the content of the corresponding micro-booklet. For better publicity and reading, users can also print the identification code on the corresponding paper document.
[0047] The following is a description of the micro-album automatic generation device provided by this embodiment. The micro-album automatic generation device described below and the micro-album automatic generation method described above can refer to each other.
[0048] First combine Figure 2 , introduce the automatic generation device of micro-book library, such as Figure 2 As shown, the micro-booklet automatic generation device may include: The text acquisition unit 100 is used to acquire the material file set and the required text input by the user, and vectorize and store each material file; The content generation unit 200 is used to generate micro-book copy, micro-book background image and micro-book background music according to the required text based on the pre-trained text generation model, image generation model and audio generation model; The material file screening unit 300 is used to calculate the cosine similarity between the requirement text and each material file, and determine the candidate material file set according to the cosine similarity; The template screening unit 400 is used to calculate the cosine similarity between the requirement text and each template, and determine the candidate template set according to the cosine similarity; The display unit 500 is used to display the candidate material file set and the candidate template set respectively, so that the user can select the target material and the target template according to the design requirements; The micro-booklet generating unit 600 is used to generate a micro-booklet set based on the micro-booklet copy, micro-booklet background image, micro-booklet background music, target material and target template, wherein the micro-booklet set includes at least one micro-booklet.
[0049] The micro-album automatic generation device provided in this embodiment can be applied to the micro-album automatic generation equipment. Figure 3 The hardware structure diagram of the micro-book library automatic generation device is shown. Figure 3, the hardware structure of the device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4; In this embodiment, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4; The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the present embodiment, etc.; The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory; The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement various processing flows in the aforementioned micro-library automatic generation solution.
[0050] This embodiment also provides a storage medium, which can store a program suitable for execution by a processor, and the program is used to implement each processing flow in the aforementioned micro-library automatic generation solution.
[0051] Finally, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0052] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0053] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for automatically generating a micro-book library, characterized in that: include: Obtain the material file set and requirement text entered by the user, and vectorize and store each material file; The pre-trained text generation model, image generation model, and audio generation model generate micro-book copy, micro-book background image, and micro-book background music respectively according to the required text; Calculate the cosine similarity between the required text and each material file, and determine the candidate material file set based on the cosine similarity; Calculate the cosine similarity between the requirement text and each template, and determine the candidate template set based on the cosine similarity; The candidate material file set and the candidate template set are displayed respectively, so that the user can select the target material and the target template according to the design requirements; A micro-booklet set is generated based on the micro-booklet copy, the micro-booklet background image, the micro-booklet background music, the target material and the target template, wherein the micro-booklet set includes at least one micro-booklet.
2. According to the method for automatically generating a micro-booklet library according to claim 1, it is characterized in that: Before calculating the cosine similarity between the required text and each material file, it also includes: Preprocess the demand text to obtain a set of text terms; Based on the word set, the TF-IDF vectors of the requirement text, each material file and template are calculated respectively.
3. The method for automatically generating a micro-booklet according to claim 1, characterized in that: Vectorize and store each material file, including: Identify the file type of each material file in the material file set, where the file type includes audio, video, picture and text; Extract file information from corresponding material files according to file type; Vectorize and store file information.
4. The method for automatically generating a micro-booklet according to claim 3, characterized in that: Extract file information from the corresponding source file according to the file type, including: If the file type is audio, extract the music information and voice information in the material file, and convert the voice information into text information; If the file type is a picture, the text information and picture content information in the source file are extracted; If the file type is video, the audio in the source file is extracted, and each frame of the source file is sampled to obtain the audio file and the image corresponding to each frame, and the file information in the audio file and the image of each frame is extracted according to the file type.
5. The method for automatically generating a micro-booklet according to claim 1, characterized in that: Also includes: Display the micro-book; In response to the user's editing operation, the micro-booklet copy, micro-booklet background image, micro-booklet background music and micro-booklet layout of the micro-booklet are edited.
6. The method for automatically generating a micro-booklet according to claim 1, characterized in that: The text generation model is the Transformer model; The image generation model is the Stable Diffusion model; The audio generation model is the Stable Audio Open model.
7. The method for automatically generating a micro-booklet according to any one of claims 1 to 6, characterized in that: Also includes: An identification code is generated based on the micro-book, so that other users can view the micro-book content by scanning the identification code.
8. A device for automatically generating a micro-book library, characterized in that: include: A text acquisition unit, used to acquire a set of material files and a required text input by a user, and vectorize and store each material file; A content generation unit is used to generate micro-book copy, micro-book background image and micro-book background music according to the required text based on the pre-trained text generation model, image generation model and audio generation model; A material file screening unit, used to calculate the cosine similarity between the requirement text and each material file, and determine a candidate material file set according to the cosine similarity; The template screening unit is used to calculate the cosine similarity between the requirement text and each template, and determine the candidate template set according to the cosine similarity; A display unit, used to display the candidate material file set and the candidate template set respectively, so that the user can select the target material and the target template according to the design requirements; The micro-booklet generating unit is used to generate a micro-booklet set based on the micro-booklet copy, micro-booklet background picture, micro-booklet background music, target material and target template, wherein the micro-booklet set includes at least one micro-booklet.
9. A micro-booklet automatic generation device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the micro-library automatic generation method as described in any one of claims 1-7.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the method for automatically generating a micro-album as described in any one of claims 1 to 7 is implemented.
