Multi-modal file label automatic generation method

Abstract

The invention discloses a multi-modal file label automatic generation method, and relates to the technical field of label generation, and the method comprises the steps: firstly constructing a multi-level file label category set as a label matching reference library, then uploading a target multi-modal file, preprocessing a text extracted according to the file, and constructing a file representation vector; finally, the computing power of the current equipment is judged, labels are generated according to scenes, if the computing power is limited, the labels are extracted step by step, and a multi-level candidate label set is generated; if the computing power is sufficient, combining the multi-level label category sets, and directly extracting a candidate label set; thirdly, modifying the word segmentation result by calculating the cohesion degree of adjacent words in the initial word segmentation result, and automatically extracting the multi-level file label again; and finally, calculating a comprehensive cosine similarity value of the generated candidate file tags, and outputting an optimal file tag. According to the invention, effective utilization of resources can be realized, and the accuracy of the generated file tag is improved.

Description

Technical Field

[0001] This invention relates to the field of tag generation technology, and in particular to a method for automatically generating tags for multimodal files. Background Technology

[0002] In existing technologies, the generation of labels for multimodal files largely relies on manual customization based on analysis needs and experience. This process can involve tedious steps such as manually reading file content, extracting information, and filtering and matching labels. This high level of human involvement and low efficiency not only consumes significant manpower but also struggles to meet the demands of generating labels for batches of multimodal files. Our designed method uses a pre-constructed multi-level set of file label categories and file representation vectors to calculate cosine similarity, automating the matching and generation of file labels. This effectively reduces manpower input and improves label generation efficiency. Furthermore, existing technologies do not address the differences in computing resources. If a uniform label generation method is used on devices with varying computing power configurations, two drawbacks can occur: first, insufficient computing resources can hinder or even prevent task execution; second, excessive computing resources can lead to redundant processes and low resource utilization efficiency. Summary of the Invention

[0003] The main objective of this invention is to provide a method for automatically generating multimodal file tags.

[0004] Another objective of this invention is to provide an automatic multimodal document tag generation device.

[0005] The third objective of this invention is to provide an electronic device.

[0006] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0007] To achieve the above objectives, a first aspect of the present invention provides a method for automatically generating multimodal document tags, comprising:

[0008] Construct a multi-level set of file tag categories containing first-level tags, second-level tags, and third-level tags, and construct corresponding tag representation vectors for each level of tag category; Receive and parse multimodal files, extract text content from the files and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weights of the candidate words; Based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted; the comprehensive similarity is calculated based on the similarity of the tags at each level in the tag path, and the tag path with the highest comprehensive similarity is output as the file annotation result; The cohesion of adjacent words in the word segmentation results is calculated and the candidate word set is updated. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

[0009] Optionally, a multi-level set of file tag categories is constructed, including first-level tags, second-level tags, and third-level tags, and corresponding tag representation vectors are constructed for each level of tag category, including: A set of primary tag categories is pre-defined, and corresponding tag description information is configured for each primary tag category; Based on the set of primary tag categories, each primary tag category is logically expanded to generate a corresponding set of secondary tag categories; Based on the set of secondary tag categories, each secondary tag category is logically expanded to generate a corresponding set of tertiary tag categories; The tag description information corresponding to the first-level tag, second-level tag and third-level tag is vectorized to generate tag representation vectors corresponding to each level of tag category.

[0010] Optionally, receive and parse multimodal files, extract the text content of the files, and generate corresponding text data, including: Receive the multimodal file to be processed and determine the corresponding parsing method according to the file type; When the multimodal file is a document file, the file content is directly parsed to extract text information; When the multimodal file is an image, audio, or video file, text information related to the file content is extracted through content parsing or metadata acquisition. The extracted text information is processed in a unified format to generate text data for subsequent processing.

[0011] Optionally, the text data is segmented and candidate words are selected. A file representation vector is constructed based on the word frequency weights of the candidate words, including: The text data is segmented to obtain a segmentation result consisting of multiple words; The words in the word segmentation results are labeled with part-of-speech information, and candidate words are selected according to preset part-of-speech filtering rules; The frequency of each candidate word in the corresponding text data is counted, and the word frequency weight of the candidate words is calculated based on the frequency of occurrence; Candidate words with word frequency weight values ​​are sorted, and the candidate words ranked at the top of the preset ranking are used as feature words of the text data. A corresponding file representation vector is constructed, where each element in the vector represents a word frequency weight value corresponding to a feature value.

[0012] Optionally, when computing power is limited, based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted, including: Calculate the cosine similarity between the file representation vector and the representation vectors of each first-level tag in the first-level tag category set, and select candidate first-level tags according to preset filtering rules; Based on the set of secondary label categories corresponding to the candidate primary labels, calculate the cosine similarity between the file representation vector and each secondary label representation vector, and select candidate secondary labels; Based on the set of tertiary tag categories corresponding to the candidate secondary tags, calculate the cosine similarity between the file representation vector and each tertiary tag representation vector, and select candidate tertiary tags; Combine candidate first-level tags, candidate second-level tags, and candidate third-level tags to form at least one complete tag path.

[0013] Optionally, when sufficient computing power is available, based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted, including: The first-level tag category set, the second-level tag category set, and the third-level tag category set are merged into a unified tag category set; Calculate the cosine similarity between the file representation vector and each tag representation vector in the unified tag category set, and select a candidate tag set according to a preset filtering rule; The candidate tag set is filtered to include at least one complete tag path consisting of first-level tags, second-level tags, and third-level tags.

[0014] Optionally, the cohesion of adjacent words in the word segmentation results is calculated and the candidate word set is updated. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file, including: Calculate the cohesion of adjacent words in the word segmentation results; Adjacent words with a cohesion greater than a preset threshold are grouped into word groups, and the candidate word set is updated based on the word groups; The word frequency weights of the candidate words are recalculated based on the updated candidate word set, and the file representation vector is reconstructed. The tag path extraction and comprehensive similarity calculation steps are repeated based on the reconstructed file representation vector; Compare the overall similarity between the label paths obtained before and after the update, and output the label path with the higher overall similarity as the final annotation result of the file.

[0015] To achieve the above objectives, a second aspect of the present invention provides an automatic multimodal document tag generation device, comprising: The first module is used to construct a multi-level set of file tag categories, including first-level tags, second-level tags, and third-level tags, and to construct corresponding tag representation vectors for each level of tag category; The second module is used to receive and parse multimodal files, extract the text content of the files and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weight of the candidate words; The third module is used to extract at least one tag path consisting of first-level tags, second-level tags, and third-level tags based on the cosine similarity between the file representation vector and the tag representation vectors at each level; calculate the comprehensive similarity based on the similarity of the tags at each level in the tag path, and output the tag path with the highest comprehensive similarity as the file annotation result; The fourth module is used to calculate the cohesion of adjacent words in the word segmentation results and update the candidate word set. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

[0016] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory to implement the method described in the first aspect.

[0017] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0018] The embodiments of this invention have the following beneficial effects: Firstly, it can replace the manual tag generation process, significantly saving labor costs, and is adaptable to batch multimodal file processing, significantly improving tag generation efficiency. Secondly, this method is adaptable to devices with different computing power configurations, avoiding task obstruction when resources are insufficient and process redundancy when resources are abundant, thus improving the compatibility and resource utilization efficiency of the embodiments of this invention. Furthermore, this method modifies the word segmentation results by using phrase cohesion calculated based on word frequency and co-occurring word frequency, automatically re-extracting file tags, and outputting the optimal file tags by comprehensively comparing the cosine similarity of the two automatically extracted file tags, avoiding the unreliability of tag extraction caused by inaccurate traditional word segmentation methods. Finally, the multi-level file tags generated by this method can be applied to multiple fields such as hierarchical protection, intelligent retrieval, and workflow automation, possessing broad application prospects and practical value. Attached Figure Description

[0019] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a first manifestation of a multimodal file tag automatic generation method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a second manifestation of an automatic multimodal file tag generation method provided in an embodiment of the present invention; Figure 3 This is an architecture diagram of a multimodal file tag automatic generation system provided in an embodiment of the present invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] In existing technologies, the generation of multimodal file tags mostly relies on manual customization, which is not only costly in terms of manpower but also difficult to adapt to the needs of batch multimodal file tag generation. Secondly, existing technologies generally do not have an adaptation design for the differences in computing resources. If a uniform tag generation method is used to run on devices with different computing power configurations, problems such as task execution being hindered when computing resources are insufficient and process redundancy and low resource utilization efficiency when computing resources are abundant can easily occur. Finally, traditional word segmentation methods are prone to inaccurate word segmentation when processing text from different domains, which in turn affects the reliability of automated file tag extraction results.

[0023] To address this issue, this application proposes a multimodal file tag automatic generation method. The method designs an automatic tag generation process tailored to varying computing resources, employing different strategies to generate file tags under different computing scenarios, thus achieving efficient resource utilization. Furthermore, traditional word segmentation methods may exhibit inaccuracies when dealing with text from different domains. The method designed in this application introduces the calculation of phrase cohesion, performing secondary word segmentation before calculating file tags, thereby improving the accuracy of the generated file tags.

[0024] The following describes an automatic multimodal file tag generation method according to an embodiment of the present invention, with reference to the accompanying drawings.

[0025] Example 1 This embodiment provides a method for automatically generating multimodal file tags. For example... Figure 1 and Figure 2 As shown, the method includes the following steps: S1. Construct a multi-level set of file tag categories containing first-level tags, second-level tags, and third-level tags, and construct corresponding tag representation vectors for each level of tag category.

[0026] In the embodiments of this application, a multi-level file tag category set is constructed, which includes a first-level tag category set, a second-level tag category set, and a third-level tag category set. Each set corresponds to a specific tag category, and there are certain logical relationships between tag categories at different levels. Specifically, the second-level tag category is a further expansion of the first-level tag category, while the third-level tag category is a further refinement and expansion of the second-level tag category. Through this multi-level tag category design, different types of files can be classified and labeled more accurately, and the refinement and accuracy of the tags can be improved.

[0027] Specifically, in this embodiment of the invention, a primary tag category set, TagSet_1, is pre-defined. This set includes multiple primary tag categories, each corresponding to a broader document category. For example, primary tag categories could be set to "Technical," "Arts and Literature," or "Finance," etc. Each primary tag category has corresponding tag description information to further define the document content type represented by that category.

[0028] After constructing the first-level tag category set, the next step is to construct the second-level tag category set, TagSet_2. Second-level tag categories are further logical expansions of the first-level tag categories. For example, under the first-level tag "Technology," second-level tags can be subdivided into "Software Technology," "Hardware Technology," "Network Technology," etc.; under the first-level tag "Arts and Literature," second-level tags can be subdivided into "Literary Works," "Film Art," "Musical Art," etc. Each second-level tag category also corresponds to descriptive information, which explains the specific meaning and scope of application of the second-level tag in detail.

[0029] Furthermore, for each second-level tag category, third-level tag categories are constructed. Third-level tag categories are further refinements and expansions of the second-level tag categories. For example, under the second-level tag "Software Technology," third-level tags can be subdivided into "Programming Languages," "Operating Embodiments of this Invention," "Database," etc.; under the second-level tag "Literary Works," third-level tags can be subdivided into "Novels," "Poetry," "Drama," etc. Third-level tag categories provide more refined file classification identifiers, making file classification more hierarchical and precise.

[0030] To ensure that each tag category can be effectively processed by the computer, each tag category needs to be converted into a vector representation that the computer can understand. The construction of tag vectors is one of the key steps in this method. In this embodiment, the tag description information corresponding to first-level, second-level, and third-level tags is vectorized. Vectorization transforms the tag category description information into numerical vectors, enabling the computer to perform tag matching and classification through vector computation.

[0031] In practice, tag description information is processed using Natural Language Processing (NLP) techniques. One possible implementation uses a bag-of-words model, TF-IDF (Term Frequency-Inverse Document Frequency), or word vector models (such as Word2Vec and GloVe) to represent the tag description information. After vectorization, the resulting tag representation vectors effectively reflect the similarity between tag categories, thus providing a foundation for subsequent tag matching and classification.

[0032] Specifically, let i = 1, 2, 3, representing the first-level, second-level, and third-level tag categories. The tag categories in each set can be represented as TagSet_i = {T_1, T_2, ..., T_n}, where n represents the number of tag categories in the set. Each tag category T_i corresponds to a vector [t_1, t_2, ..., t_m], where m represents the dimension of the tag vector. In this way, each tag category can be stored in vector form, facilitating subsequent calculations and processing.

[0033] S2, receive and parse the multimodal file, extract the text content of the file and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weight of the candidate words.

[0034] In one embodiment of this application, a multimodal file to be processed is first received, and the corresponding parsing method is determined according to the file type. Multimodal files can include various types, such as document files, image files, audio files, and video files. Different parsing methods are used to extract text information from different file types.

[0035] Specifically: For document files, the file content is directly parsed to extract the text information. For image, audio, and video files, different technologies are used to extract text information related to the file content. For image files, Optical Character Recognition (OCR) technology can be used to convert the text in the image into text; for audio files, speech recognition technology can be used to convert speech into text; for video files, subtitles can be extracted or other content analysis techniques can be used to obtain relevant text information. All extracted text information will be uniformly formatted to facilitate subsequent processing and analysis.

[0036] Next, the extracted text data is processed through word segmentation. This process breaks the text data down into individual words, and each word is labeled with its part-of-speech tagging. The results of this tagging aid in subsequent candidate word selection. Typically, nouns are selected as candidate words based on specific part-of-speech rules, as nouns generally better represent the key information in the text.

[0037] Next, frequency statistics are performed on the selected candidate words to calculate the frequency of each candidate word in the text. Words with higher frequencies usually occupy more important positions in the text, and therefore contribute more to the semantic expression of the text. Based on the frequency of each candidate word, its word frequency weight can be calculated. The formula for calculating word frequency weight is as follows:

[0038] Where w_q_p represents the frequency of candidate word w_q in text d_p. If there are t candidate words in text d_p, then This represents the sum of the frequencies of all candidate words in the text d_p. As you can see, the more times a word appears, the higher its frequency weight. In this way, each word can be assigned a weight based on its frequency, reflecting its importance in the text.

[0039] After calculating the term frequency weights, all candidate words with corresponding term frequency weights are sorted, and the top m candidate words are selected as feature words. Feature words are typically words that effectively describe the text's topic and content, playing a crucial role in document representation. Finally, using these feature words and their corresponding term frequency weights, a document representation vector is constructed. This vector describes the text's content, with each vector element representing the term frequency weight value corresponding to a feature word. The document representation vector can be represented as: FileVec=[f_1, f_2,...,f_m] Where f_1, f_2, ..., f_m are the word frequency weights of the first m feature words, and m is the number of feature words selected.

[0040] The above steps complete the conversion from multimodal files to file representation vectors, laying the foundation for subsequent tasks such as tag generation, text classification, and similarity calculation. This method effectively extracts key information from text and converts it into vector form, enabling computers to perform further automated processing and analysis.

[0041] S3. Based on the cosine similarity between the file representation vector and the label representation vectors at all levels, extract at least one label path consisting of first-level labels, second-level labels, and third-level labels; calculate the comprehensive similarity based on the similarity of each level of labels in the label path, and output the label path with the largest comprehensive similarity as the file annotation result.

[0042] In this embodiment, the tag path that best matches the file content is extracted based on the cosine similarity between the file representation vector and the tag representation vectors at each level. The tag path consists of first-level tags, second-level tags, and third-level tags, representing the hierarchical classification of the file. By calculating the similarity of tags at each level in the tag path, the tag path with the highest overall similarity is finally output as the file's annotation result.

[0043] (1) When computing power is limited, this embodiment of the invention will extract the tag path by means of step-by-step screening. First, the cosine similarity between the file representation vector and each tag category in the first-level tag category set is calculated, and the top few first-level tags with the highest similarity to the file representation vector are selected as candidate tags. Then, based on the selected first-level tags, the cosine similarity between the corresponding second-level tag set and the file representation vector is calculated, and the top few second-level tags with the highest similarity are selected. Finally, based on the selected second-level tags, the cosine similarity between the third-level tags and the file representation vector is calculated, and the top few third-level tags with the highest similarity are selected.

[0044] In each level of label category, the cosine similarity calculation formula is:

[0045] in, This represents the label vector of the j-th label category. This represents the value of the j-th feature in the file representation vector, where m is the dimension of the label vector. The closer the cosine value is to 1, the higher the similarity between the file and the label.

[0046] In one embodiment of the present invention, the body step includes: 1. Extract primary labels: Calculate the cosine similarity between the file representation vector and the representation vector of each label in the primary label category set. Select the top 20% of primary labels as candidate labels based on the similarity, and record the cosine similarity value corresponding to each label.

[0047] 2. Extract secondary labels: Based on the selected primary labels, calculate the cosine similarity between the file representation vector and the representation vector of each label in the corresponding secondary label category set. Select the 20% of secondary labels with the highest similarity as candidate labels and record the corresponding cosine similarity values.

[0048] 3. Extracting Level 3 Labels: Based on the selected Level 2 labels, calculate the cosine similarity between the file representation vector and the representation vector of each label in the corresponding Level 3 label category set. Select the 20% of Level 3 labels with the highest similarity as candidate labels and record the corresponding cosine similarity values.

[0049] 4. Combine tag paths: Combine the selected first-level tags, second-level tags, and third-level tags to form at least one complete tag path. Each tag path corresponds to a triple {first-level tag, second-level tag, third-level tag}.

[0050] (2) When computing power is sufficient, this embodiment of the invention will improve computing efficiency and expand the tag space by merging tag sets at all levels. The specific steps are as follows: 1. Merge Tag Category Sets: Merge the first-level, second-level, and third-level tag category sets into a unified tag category set, TagSet. This allows for the simultaneous calculation of the cosine similarity between the file representation vector and all tag category sets.

[0051] 2. Extract candidate tag set: Traverse the merged tag category set TagSet, calculate the cosine similarity between each tag category and the file representation vector, and select candidate tag sets according to preset filtering rules, such as selecting candidate third-level tag sets based on a certain proportion (e.g., the top 20%). During filtering, the candidate set must contain at least one complete third-level tag path, that is, at least one triple {first-level tag, second-level tag, third-level tag}.

[0052] Finally, regardless of whether computing power is limited or sufficient, this embodiment of the invention will perform a comprehensive similarity calculation on the generated multiple tag paths. The comprehensive similarity of each tag path is calculated using the following formula:

[0053] Where cos1, cos2, and cos3 represent the cosine similarity values ​​corresponding to the first-level, second-level, and third-level labels, respectively, and α, β, and γ are the corresponding weight parameters, satisfying the following conditions: Depending on the actual needs, the weight parameters α, β, and γ can be adaptively configured to optimize the matching effect of the label path.

[0054] Finally, this embodiment of the invention selects the triple {first-level label, second-level label, third-level label} with the highest comprehensive similarity value as the output annotation result for the file. This annotation result can provide accurate labels for subsequent file processing, supporting more efficient file management and information retrieval.

[0055] S4 calculates the cohesion of adjacent words in the word segmentation results and updates the candidate word set. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

[0056] In this embodiment, the cohesion of adjacent words in the word segmentation results is further calculated, and the candidate word set is updated based on the calculation results. The updated candidate word set will be used to reconstruct the file representation vector, and the label path extraction and comprehensive similarity calculation steps will be repeated. Finally, this embodiment of the invention will select the label path with the highest comprehensive similarity based on the similarity value as the final annotation result of the file.

[0057] First, this embodiment of the invention calculates the cohesion of adjacent words in the word segmentation results. Cohesion measures the probability that two words together form a phrase; the higher the cohesion, the higher the probability that the two words appear together and form a phrase. The formula for calculating the cohesion of adjacent words is:

[0058] Where η_q_p represents the frequency of candidate words w_q and w_(q+1) appearing together in text d_p, and θ is a set co-occurrence threshold used to filter infrequent word groups. t is the total number of words in text d_p, w_q_p represents the frequency of word w_q in text d_p, and w_(q+1)p represents the frequency of word w(q+1) in text d_p. If the co-occurrence frequency of two words is lower than the set threshold θ, then these two words are considered unsuitable as word groups and are therefore removed from the candidate word set, thereby reducing the influence of occasional word groups.

[0059] By calculating cohesion, this embodiment of the invention selects adjacent words with cohesion greater than a preset threshold, combines them into new word groups, and adds these word groups to the candidate word set. Simultaneously, words that are part of these word groups in the original candidate word set are removed. In this way, this embodiment of the invention can remove some low-frequency, meaningless words, thereby optimizing the candidate word set and avoiding the misselection of uncommon or irrelevant words.

[0060] After updating the candidate word set, this embodiment of the invention needs to recalculate the word frequency weight of each candidate word. Assuming ξ new word groups are obtained as candidate words, the formula for recalculating the word frequency weight is:

[0061] Where w_q_p represents the frequency of word w_q in text d_p, and ∑_(k=1)^(t-ξ) w_k_p represents the sum of the frequencies of all candidate words in text d_p (after updating the candidate word set, the ξ word groups are no longer included in the original candidate word set). By recalculating the word frequency weights, this embodiment of the invention can more accurately represent the relative importance of each candidate word in the file, thereby improving the accuracy of the file representation vector.

[0062] Based on the updated candidate word set and its word frequency weights, this embodiment of the invention reconstructs the document representation vector. This new document representation vector will better reflect the content and topic of the text and provide higher-quality data for subsequent tag path extraction and similarity calculation.

[0063] Once the file representation vector is updated, this embodiment of the invention will repeat the aforementioned tag path extraction and comprehensive similarity calculation steps. First, the cosine similarity between the updated file representation vector and the tag representation vectors at each level is calculated to extract new tag paths. Then, the comprehensive similarity is calculated based on the new tag paths. The comprehensive similarity value is obtained by weighting the cosine similarity values ​​of the first-level, second-level, and third-level tags, as shown in the following formula:

[0064] Finally, this embodiment of the invention compares the overall similarity between the tag paths obtained before and after the update. By comparing the overall similarity values ​​calculated twice, the tag paths {first-level tags, second-level tags, and third-level tags} with higher similarity are selected as the final annotation results for the file. This ensures that the final output tag paths are more accurate and reflect the core features of the file content.

[0065] Through this series of steps, this embodiment of the invention can optimize the candidate word set by utilizing the cohesion of adjacent words, construct a more accurate document representation vector based on the updated candidate word set, and finally select the label path with the highest comprehensive similarity as the document annotation result. This approach not only improves the accuracy of the labels but also further optimizes the efficiency of label path extraction, ensuring high-quality annotation results even with limited computing resources.

[0066] Example 2 This invention also provides a multimodal file tag automatic generation system to implement the multimodal file tag automatic generation method described in Embodiment 1. Its technical architecture is as follows: Figure 3 As shown, the system is divided into three layers: a multimodal file upload layer, an automatic tag generation layer, and a tag application layer.

[0067] Multimodal file upload layer: As the system's data input layer, it supports the upload of multimodal files such as image files, document files, and video files, providing a data source for the subsequent automated generation of tags.

[0068] Automatic tag generation layer: As the core processing layer of the system, it receives files uploaded by the upload layer and sequentially performs text preprocessing (including file text content extraction and text segmentation) and file representation vector construction (including word frequency weight calculation and feature word extraction). Finally, it automatically extracts and outputs multi-level file tags containing first-level tags, second-level tags, and third-level tags.

[0069] Tag Application Layer: As the system's output application layer, based on the generated multi-level tags and combined with actual application needs, it can realize applications such as hierarchical protection, intelligent retrieval, and workflow automation. Regarding hierarchical protection, users can determine the file data security level based on the security attribute tag type corresponding to the file, using preset discrimination rules. Then, according to the different security level classification standards, appropriate security protection measures are implemented for files of the corresponding level, ultimately achieving the goal of hierarchical and classified security protection for multimodal files. In terms of intelligent retrieval, multimodal files that meet the requirements can be found through file tags, upgrading traditional filename-based or simple description-based retrieval to understanding and matching the semantic content of the file, improving the user's information retrieval efficiency, and realizing cross-modal retrieval. Regarding workflow automation, workflow triggering rules can be preset based on the attributes and hierarchical information of multi-level tags. When a file generates tags, the matching workflow process is automatically triggered, improving the automation level of workflow operation and realizing intelligent management of the entire file lifecycle, including automatic archiving, intelligent distribution, process monitoring, and other fully automated operations.

[0070] Example 3 This invention also provides a multimodal document tag automatic generation device, the device comprising: The first module is used to construct a multi-level set of file tag categories, including first-level tags, second-level tags, and third-level tags, and to construct corresponding tag representation vectors for each level of tag category; The second module is used to receive and parse multimodal files, extract the text content of the files and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weight of the candidate words; The third module is used to extract at least one tag path consisting of first-level tags, second-level tags, and third-level tags based on the cosine similarity between the file representation vector and the tag representation vectors at each level; calculate the comprehensive similarity based on the similarity of the tags at each level in the tag path, and output the tag path with the highest comprehensive similarity as the file annotation result; The fourth module is used to calculate the cohesion of adjacent words in the word segmentation results and update the candidate word set. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

[0071] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0072] Example 4 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0073] Example 5 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0074] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0076] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for automatically generating multimodal document tags, characterized in that, Includes the following steps: Construct a multi-level set of file tag categories containing first-level tags, second-level tags, and third-level tags, and construct corresponding tag representation vectors for each level of tag category; Receive and parse multimodal files, extract text content from the files and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weights of the candidate words; Based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted; the comprehensive similarity is calculated based on the similarity of the tags at each level in the tag path, and the tag path with the highest comprehensive similarity is output as the file annotation result; The cohesion of adjacent words in the word segmentation results is calculated and the candidate word set is updated. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

2. The method according to claim 1, characterized in that, Construct a multi-level set of file tag categories containing first-level, second-level, and third-level tags, and construct corresponding tag representation vectors for each level of tag category, including: A set of primary tag categories is pre-defined, and corresponding tag description information is configured for each primary tag category; Based on the set of primary tag categories, each primary tag category is logically expanded to generate a corresponding set of secondary tag categories; Based on the set of secondary tag categories, each secondary tag category is logically expanded to generate a corresponding set of tertiary tag categories; The tag description information corresponding to the first-level tag, second-level tag and third-level tag is vectorized to generate tag representation vectors corresponding to each level of tag category.

3. The method according to claim 2, characterized in that, Receive and parse multimodal files, extract the text content, and generate corresponding text data, including: Receive the multimodal file to be processed and determine the corresponding parsing method according to the file type; When the multimodal file is a document file, the file content is directly parsed to extract text information; When the multimodal file is an image, audio, or video file, text information related to the file content is extracted through content parsing or metadata acquisition. The extracted text information is processed in a unified format to generate text data for subsequent processing.

4. The method according to claim 3, characterized in that, The text data is segmented and candidate words are selected. A file representation vector is constructed based on the word frequency weights of the candidate words, including: The text data is segmented to obtain a segmentation result consisting of multiple words; The words in the word segmentation results are labeled with part-of-speech information, and candidate words are selected according to preset part-of-speech filtering rules; The frequency of each candidate word in the corresponding text data is counted, and the word frequency weight of the candidate words is calculated based on the frequency of occurrence; Candidate words with word frequency weight values ​​are sorted, and the candidate words ranked at the top of the preset ranking are used as feature words of the text data. A corresponding file representation vector is constructed, where each element in the vector represents a word frequency weight value corresponding to a feature value.

5. The method according to claim 4, characterized in that, When computing power is limited, based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted, including: Calculate the cosine similarity between the file representation vector and the representation vectors of each first-level tag in the first-level tag category set, and select candidate first-level tags according to preset filtering rules; Based on the set of secondary label categories corresponding to the candidate primary labels, calculate the cosine similarity between the file representation vector and each secondary label representation vector, and select candidate secondary labels; Based on the set of tertiary tag categories corresponding to the candidate secondary tags, calculate the cosine similarity between the file representation vector and each tertiary tag representation vector, and select candidate tertiary tags; Combine candidate first-level tags, candidate second-level tags, and candidate third-level tags to form at least one complete tag path.

6. The method according to claim 4, characterized in that, When computing power is sufficient, based on the cosine similarity between the file representation vector and the tag representation vectors at each level, at least one tag path consisting of first-level tags, second-level tags, and third-level tags is extracted, including: The first-level tag category set, the second-level tag category set, and the third-level tag category set are merged into a unified tag category set; Calculate the cosine similarity between the file representation vector and each tag representation vector in the unified tag category set, and select a candidate tag set according to a preset filtering rule; The candidate tag set is filtered to include at least one complete tag path consisting of first-level tags, second-level tags, and third-level tags.

7. The method according to any one of claims 5 or 6, characterized in that, The cohesion of adjacent words in the word segmentation results is calculated and the candidate word set is updated. Based on the updated candidate word set, the file representation vector is reconstructed, and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file, including: Calculate the cohesion of adjacent words in the word segmentation results; Adjacent words with a cohesion greater than a preset threshold are grouped into word groups, and the candidate word set is updated based on the word groups; The word frequency weights of the candidate words are recalculated based on the updated candidate word set, and the file representation vector is reconstructed. The tag path extraction and comprehensive similarity calculation steps are repeated based on the reconstructed file representation vector; Compare the overall similarity between the label paths obtained before and after the update, and output the label path with the higher overall similarity as the final annotation result of the file.

8. A multimodal document tag automatic generation device, characterized in that, include: The first module is used to construct a multi-level set of file tag categories, including first-level tags, second-level tags, and third-level tags, and to construct corresponding tag representation vectors for each level of tag category; The second module is used to receive and parse multimodal files, extract the text content of the files and generate corresponding text data; perform word segmentation on the text data and filter candidate words, and construct a file representation vector based on the word frequency weight of the candidate words; The third module is used to extract at least one tag path consisting of first-level tags, second-level tags, and third-level tags based on the cosine similarity between the file representation vector and the tag representation vectors at each level; calculate the comprehensive similarity based on the similarity of the tags at each level in the tag path, and output the tag path with the highest comprehensive similarity as the file annotation result; The fourth module is used to calculate the cohesion of adjacent words in the word segmentation results and update the candidate word set. Based on the updated candidate word set, the file representation vector is reconstructed and the label path extraction and comprehensive similarity calculation steps are repeated. Finally, the label path with the highest similarity is output as the final annotation result of the file.

9. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images