Method for matching related multimedia materials according to text

Abstract

The invention provides a method for matching related multimedia materials according to texts, which is used for acquiring a retrieval library file according to a material information file and comprises the following steps of: loading the material information file and loading material information into a server memory; executing a material processing flow to obtain a key value pair array; obtaining a Trie tree according to the key value pair array, and storing the Trie tree as a retrieval library file; text processing and material retrieval acquisition: executing text preprocessing and segmentation, and executing structured segmentation on the text by using a first regular expression to acquire a structured text and a segmented title; according to a second regular expression (-) Lt; and =. ! U ! - Performing text sentence segmentation on the structured text to obtain a sentence list; executing a title and sentence connection and combination instruction according to the sentence list and the segmented titles to obtain title and sentence key value pairs; and executing the multimedia content generation instruction according to the material library file. The problem of low efficiency of artificially synthesizing the multimedia materials is solved.

Description

Technical Field

[0001] The present invention belongs to the field of text illustration, and particularly relates to a method for matching relevant multimedia materials according to text. Background Art

[0002] In modern content production, especially in the creation of multimedia content such as pictures, texts, and videos, it is often necessary to match relevant pictures, audio, short videos and other multimedia materials according to the text content to enhance the expressiveness of the content. However, in actual operation, creators usually need to manually search, screen and insert these materials after understanding the text content, which not only consumes a lot of time, but also easily results in problems such as unmatched materials or uneven material quality. Summary of the Invention

[0003] The purpose of this embodiment is to provide a method for matching relevant multimedia materials according to text, which is used to solve the problem of low efficiency in manually synthesizing multimedia materials.

[0004] A method for matching relevant multimedia materials according to text includes:

[0005] S1: Obtain a retrieval library file according to the material information file, including the steps of:

[0006] Load the material information file and load the material information into the server memory;

[0007] Execute the material processing process to obtain a key-value pair array;

[0008] Obtain a Trie tree according to the key-value pair array, including converting the keyword key into a character array and inserting it word by word from the root node;

[0009] Use the pickle library to serialize the Trie tree and save it as a retrieval library file;

[0010] S2: Text processing and material retrieval acquisition, including the steps of:

[0011] Execute text preprocessing and segmentation, and use the first regular expression to perform structured segmentation on the text to obtain structured text and segmentation titles. The first regular expression includes [\r\n]+\\d+\\.\\s*([^\r\n]+)[\r\n]+;

[0012] Execute text sentence segmentation according to the second regular expression (?<=。|!|?|!|\\?) and the structured text to obtain a sentence list;

[0013] Execute the title sentence connection and combination instruction according to the sentence list and the segmentation title to obtain a title sentence key-value pair;

[0014] Obtain the material result string corresponding to the key by using the BinTrie algorithm and the callback interface method according to the title sentence key-value pair and the retrieval library file;

[0015] Traverse each sentence node according to the material result, execute the field mapping comparison instruction to obtain the material library file and save it;

[0016] S3: Execute the multimedia content generation instruction according to the material library file.

[0017] Furthermore, the execution of the material processing process includes annotating and editing the material, and the annotation and editing include keyword and tag editing.

[0018] Furthermore, the key-value pair array includes keyword-address key-values, where the address key-value includes the server relative path or the http link address.

[0019] Furthermore, the title sentence connection combination instruction includes string connection, separated by an underscore in the middle.

[0020] Furthermore, the material result string is a JSON string.

[0021] Furthermore, the field mapping comparison instruction includes traversing each sentence node, performing field mapping comparison, constructing a fused structured object in memory and outputting it as a file.

[0022] Furthermore, the multimedia content generation instruction includes a video generation instruction and a graphic and text generation instruction.

[0023] Furthermore, the video generation instruction includes using FFmpeg to generate video segments from text and materials and merging them into a complete video.

[0024] Furthermore, the graphic and text generation instruction includes using PDFBox to create a graphic and text article, and also includes using PDImageXObject to perform image drawing.

[0025] Furthermore, the graphic and text generation instruction also includes using PDDocument and PDPage to perform text generation.

[0026] A method for matching relevant multimedia materials according to text provided by the present invention solves the problem of low efficiency in manually synthesizing multimedia materials.

[0027] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, provides a detailed description as follows. Description of the Drawings

[0028] Figure 1 : A method step diagram for matching relevant multimedia materials according to text provided by an embodiment of the present invention;

[0029] Figure 2 : The Trie tree for retrieval used to match relevant multimedia materials according to the text provided by the embodiments of the present invention. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present invention will be described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] The present invention provides a method for matching relevant multimedia materials according to the text (see Figure 1 ), and the specific implementation method includes steps of material library construction, text processing, and material retrieval;

[0032] S1: Obtain a retrieval library file according to the material information file. In this embodiment, the following steps are included

[0033] S1.1 Load the material information file and load the material information into the server memory to ensure that the system can quickly respond to multimedia material query requests. In this embodiment, the material information file is a text file, and the file reading method is used to load the material information. Optionally, it can be implemented using Python statements; the sample code is as follows:

[0034]

[0035] S1.2 Execute the material processing process, perform annotation and editing (keywords, tags) on the materials; form the material name according to fixed rules, and obtain the key-value pair array; the constructed keyword-address key-value pairs facilitate efficient search later, such as: {ibuprofen dispersible tablets: address 1}, {ibuprofen suspension: address 2}; where the address includes the server relative path, http link, etc. The Ai annotation involves multiple aspects. In this embodiment, the material annotation is manual text annotation. Optionally, for image annotation, it is bounding box annotation, polygon annotation, and key point annotation. It also includes video frame annotation and video segment annotation for videos, and named entity recognition annotation and sentiment analysis annotation for texts.

[0036] S1.3 Obtain the Trie tree according to the key-value pair array generated in step 1.2, and construct the character tree according to the following logic: convert the keyword key into a character array and insert it word by word from the root node; the structure generated by this step is as Figure 2 shown. Specifically, when implementing, use the BinTrie binary tree algorithm to quickly retrieve and obtain the Trie tree according to the keyword; the example code is:

[0037]

[0038]

[0039]

[0040] S1.4 Serialize the Trie tree using the pickle library and save it as a retrieval library file to provide data support for keyword search and material association in subsequent steps. The implementation code in this embodiment is as follows:

[0041] Example of saving the Trie tree as a serialized file using the pickle library:

[0042]

[0043]

[0044] S2: Text processing and material retrieval:

[0045] S2.1 Perform text preprocessing and segmentation, and use the regular expression library to perform structured segmentation of the text to obtain structured text and segmentation titles; for example, the input text is as follows:

[0046] I am the first paragraph.

[0047] 1. I am title 1. I am sentence 1. I am sentence 2.

[0048] 2. I am title 2. I am sentence 1. I am sentence 2. I am sentence 3.

[0049] I am the last paragraph.

[0050] In this embodiment, the file reading method is used to obtain the text matching content. Specifically, when implementing, read the file and execute the regular expression [\r\n]+\\d+\\.\\s*([^\r\n]+)[\r\n]+ to extract the segmented content and generate the following structured text:

[0051]

[0052] The JAVA implementation code in this embodiment is as follows:

[0053]

[0054]

[0055]

[0056]

[0057] S2.2 Perform text sentence segmentation according to the regular expression (?<=。|!|?|!|\\?) and the structured text to obtain a sentence list. For example, the first paragraph content is segmented into ["I am sentence 1", "I am sentence 2."]. The Java example code in this embodiment is as follows:

[0058]

[0059]

[0060] S2.3 Execute the title sentence connection and combination instruction according to the sentence list and the segmented title in S2.1 to obtain the title sentence key-value pair. In the specific implementation, the combined string format is "Title X"_"Sentence", separated by an underscore in the middle; the code is as follows:

[0061]

[0062]

[0063] S2.4 Use the title sentence key-value pair obtained in 2.3 and the retrieval library file to obtain the material result string corresponding to the key through the custom callback interface using the BinTrie algorithm. In this embodiment, the result string is obtained through the custom callback interface using the BinTrie algorithm. The result string is in JSON format. In the specific implementation, the custom interface is parseText(String text, AhoCorasickDoubleArrayTrie.IHit <v>processor), the structure is as follows:

[0064]

[0065] IHit <v>The interface is a callback interface that defines a hit method, which is called when a keyword matches the text. By implementing the IHit interface, it is possible to process content related to the keyword when the keyword is matched. During retrieval, the keyword is split into a character array, and starting from the first character, it is retrieved character by character from the root node to the leaf node. If each character matches a child node and the last character matches a leaf node, the index successfully matches the material. Whenever a keyword is matched, the IHit.hit method is called, and the starting and ending positions of the match, as well as the associated material address, are passed into the hit method. Through the hit method of the IHit interface, it is possible to dynamically process the material information matched when a keyword is detected in the text, perform extended rule processing, and additionally specify tags (such as the elderly and children as tags): a. When no tags are specified in the text, only materials without tags are matched; b. When tags are specified in the text, only materials with the same tags are matched

[0066] S2.5 Traverse each sentence node according to the material result to execute the field mapping comparison instruction to obtain the material library file. After the matching is completed, receive all the matching results output in S2.4, traverse each sentence node, perform field mapping comparison, construct a fused and generated structured object in memory, further serialize it into a new structured JSON string (final_result) and output it as a material library file, and the final content is as follows:

[0067]

[0068]

[0069] S3: Execute the multimedia content generation instruction according to the material library file, including: video generation instruction and graphic and text generation instruction;

[0070] In this embodiment, the video generation instruction includes using FFmpeg to generate video segments from text and materials and merge them into a complete video.

[0071] Traverse contentArray to generate segments. Obtain the material address and content corresponding to the paragraph, and then use the drawtext command of ffmpeg to generate video segments. Finally, use the concat parameter to merge the segments to generate a complete video.

[0072] In this embodiment, the graphic and text generation instruction includes using PDFBox to create a graphic and text article, draw with the help of PDDocument, PDPage, PDImageXObject, etc., add text and corresponding pictures, and output a PDF file. Specific application examples:

[0073]

[0074]

[0075]

[0076]

[0077] A method for matching relevant multimedia materials according to text provided by the present invention solves the problem of low efficiency in manually synthesizing multimedia materials.

[0078] The above are only embodiments of the present invention and are not intended to limit the protection scope of the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.< / v> < / v>

Claims

1. A method for matching relevant multimedia materials according to text, characterized in that, Including: S1: Obtain the retrieval library file according to the material information file, including the steps of: Loading the material information file and loading the material information into the server memory; Executing the material processing flow to obtain an array of key-value pairs; Obtaining a Trie tree according to the array of key-value pairs, including converting the keyword key into a character array and inserting it character by character from the root node; Using the pickle library to serialize the Trie tree and save it as a retrieval library file; S2: Text processing and material retrieval acquisition, including the steps of: Executing text preprocessing and segmentation, and using the first regular expression to perform structured segmentation of the text to obtain structured text and segmentation titles, where the first regular expression includes [\r\n]+\\d+\\.\\s*([^\r\n]+)[\r\n]+; Performing text sentence segmentation on the structured text according to the second regular expression (?<=。|!|?|!|\\?) to obtain a list of sentences; Executing the title sentence connection combination instruction according to the list of sentences and the segmentation titles to obtain title sentence key-value pairs; According to the title sentence key-value pairs and the retrieval library file, obtaining the material result string corresponding to the key by using the BinTrie algorithm and in the way of using a callback interface; Traversing each sentence node according to the material result to execute the field mapping comparison instruction to obtain and save the material library file; S3: Executing the multimedia content generation instruction according to the material library file.

2. The method for matching relevant multimedia materials according to the text as claimed in claim 1, wherein The execution of the material processing flow includes performing annotation and editing on the material, where the annotation and editing include keyword and tag editing.

3. The method for matching relevant multimedia materials according to the text as claimed in claim 1, wherein, The array of key-value pairs includes keyword-address key-values, where the address key-values include the server relative path or the http link address.

4. The method for matching relevant multimedia materials according to the text as claimed in claim 1, characterized in that, The title sentence connection combination instruction includes string connection, with an underscore in the middle for separation.

5. The method for matching relevant multimedia materials according to the text as claimed in claim 1, wherein The material result string is a JSON string.

6. The method for matching relevant multimedia materials according to text as claimed in claim 1, wherein The field mapping comparison instruction includes traversing each sentence node, performing field mapping comparison, constructing a fused structured object in memory and outputting it as a file.

7. The method for matching relevant multimedia materials according to the text as claimed in claim 1, wherein The multimedia content generation instruction includes a video generation instruction and a graphic and text generation instruction.

8. The method for matching relevant multimedia materials according to text as claimed in claim 7, wherein The video generation instruction includes using FFmpeg to generate video segments from the text and materials and merging them into a complete video.

9. The method for matching relevant multimedia materials according to text as claimed in claim 7, wherein The graphic and text generation instruction includes using PDFBox to create a graphic and text article, and also includes using PDImageXObject to perform image drawing.

10. The method for matching relevant multimedia materials according to the text as claimed in claim 7, wherein The graphic and text generation instruction also includes using PDDocument and PDPage to perform text generation.

Images