Cataloguing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a cataloguing method and device, electronic equipment and a readable storage medium. A cataloguing system can obtain a target text; the target text comprises a first text, directory data and a second text; determining a text sample according to the first text and the directory data; based on a mapping function, mapping the text sample and the second text into a matrix set corresponding to a third text and a mask; constructing a parallel structure information block, wherein the parallel structure information block is composed of task examples and a fourth text; and inputting the parallel structure information group into a multi-modal model for directory prediction, and determining a target directory corresponding to the fourth text. Therefore, the cataloguing system can construct the parallel structure information group according to the target text, and input the parallel structure information group into the multi-modal model for catalogue prediction, so that the cataloguing efficiency and the cataloguing accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and data management technology, and in particular to a cataloging method, device, electronic device and readable storage medium. Background Art

[0002] At present, the characteristics of data assets are massive, multi-source, heterogeneous and fragmented, which greatly increases the workload of data asset managers. Traditional manual cataloging and management methods are still the mainstream of data asset cataloging, but this method is not only time-consuming and labor-intensive, but also difficult to ensure the optimization of cataloging effect and quality.

[0003] Although existing self-supervised learning frameworks can learn rich features, these features are often not directly applicable to specific downstream tasks, but need to be adapted by fine-tuning on labeled datasets. However, the fine-tuning process is time-consuming and will weaken the generalization ability of the model, thus affecting the cataloging efficiency. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present invention provides a cataloging method, device, electronic device and readable storage medium.

[0005] In a first aspect, the present invention provides a cataloguing method, the method comprising:

[0006] Acquire a target text; the target text includes a first text, directory data, and a second text;

[0007] Determining a text sample according to the first text and the directory data;

[0008] Based on a mapping function, mapping the text sample and the second text to a matrix set corresponding to a third text and a mask;

[0009] Constructing a parallel structure information group, wherein the parallel structure information group is composed of a task example and a fourth text;

[0010] The parallel structure information group is input into a multimodal model for directory prediction to determine a target directory corresponding to the fourth text.

[0011] Optionally, obtaining the target text includes:

[0012] Obtain multimodal data information;

[0013] Converting the multimodal data information into a structured data summary;

[0014] The structured data summary is cleaned and integrated to obtain a target text that meets preset quality requirements.

[0015] Optionally, the constructing and type structure information group includes:

[0016] adding a first tag to the task example, and adding a second tag to the fourth text;

[0017] The union structure information group is trained according to the first label and the second label.

[0018] Optionally, the method further comprises:

[0019] Extracting the probability of each feature node corresponding to the first text by a linear discriminant analysis algorithm;

[0020] Calculating the similarity of each feature node;

[0021] Obtaining a tree structure according to the similarity through a hierarchical clustering algorithm, and dividing the tree structure;

[0022] Determining a community structure corresponding to the first text based on the segmented tree structure;

[0023] An output text corresponding to the first text is determined according to the community structure.

[0024] Optionally, the method further comprises:

[0025] In the model training stage, the matrix set is input into the multimodal model to perform directory prediction, and an output text corresponding to the third text is obtained;

[0026] Based on the output text, a target directory corresponding to the mask is extracted.

[0027] Optionally, after obtaining the target text, the following steps are included:

[0028] A storage method is determined according to the features and scene information carried by the target text, and the target text is stored based on the storage method.

[0029] In a second aspect, the present invention provides a cataloging device, the device comprising:

[0030] An acquisition module, used for acquiring a target text; the target text includes a first text, directory data, and a second text;

[0031] A determination module, configured to determine a text sample according to the first text and the directory data;

[0032] A mapping module, used for mapping the text sample and the second text to a matrix set corresponding to a third text and a mask based on a mapping function;

[0033] A construction module, used for constructing a parallel structure information group, wherein the parallel structure information group consists of a task example and a fourth text;

[0034] The determination module is further used to input the parallel structure information group into a multimodal model for directory prediction, and determine a target directory corresponding to the fourth text.

[0035] Optionally, the acquisition module includes:

[0036] A first acquisition submodule is used to acquire multimodal data information;

[0037] A conversion module, used for converting the multimodal data information into a structured data summary;

[0038] The module is used to clean and integrate the structured data summary to obtain a target text that meets the preset quality requirements.

[0039] Optionally, the building block further includes:

[0040] An adding module, configured to add a first mark to the task example and a second mark to the fourth text;

[0041] A training module is used to train the union structure information group according to the first label and the second label.

[0042] Optionally, the device further comprises:

[0043] An extraction module, used to extract the probability of each feature node corresponding to the first text through a linear discriminant analysis algorithm;

[0044] A calculation module, used to calculate the similarity of each feature node;

[0045] A segmentation module, used to obtain a tree structure according to the similarity through a hierarchical clustering algorithm, and segment the tree structure;

[0046] A determination submodule is used to determine the community structure corresponding to the first text based on the segmented tree structure; and determine the output text corresponding to the first text according to the community structure.

[0047] Optionally, the device further comprises:

[0048] Obtaining a submodule, used for inputting the matrix set into a multimodal model to perform directory prediction in a model training phase, and obtaining an output text corresponding to the third text;

[0049] The extraction submodule is used to extract a target directory corresponding to the mask based on the output text.

[0050] Optionally, the device further comprises:

[0051] The first determination module is used to determine a storage method according to the features and scene information carried by the target text, and store the target text based on the storage method.

[0052] In a third aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cataloging method described in any one of the first aspects when executing the program.

[0053] In a fourth aspect, the present invention provides a readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the steps in the cataloging method in any one of the embodiments of the first aspect above.

[0054] In an embodiment of the present invention, a cataloging system can obtain a target text; the target text includes a first text, catalog data, and a second text; a text sample is determined according to the first text and the catalog data; based on a mapping function, the text sample and the second text are mapped to a matrix set corresponding to a third text and a mask; a parallel structure information group is constructed, and the parallel structure information group is composed of a task example and a fourth text; the parallel structure information group is input into a multimodal model for catalog prediction, and a target catalog corresponding to the fourth text is determined. In this way, through the first text and catalog data in the target text, a text sample is first determined, and then the text sample and the second text are mapped, and a matrix set after mapping can be obtained, and the matrix set is composed of a third text and a mask. At the same time, the cataloging system can construct a parallel structure information group, and the parallel structure information group includes a task example and a fourth text, that is, the task example and the fourth text are input into the multimodal model, and while obtaining a target catalog with higher accuracy, the efficiency of cataloging can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 1 ;

[0057] Figure 2 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 2 ;

[0058] Figure 3 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 3 ;

[0059] Figure 4 is a flowchart of a method for extracting directory data provided by an embodiment of the present invention;

[0060] Figure 5 It is a flowchart of the steps of constructing a directory data extraction prompt provided by an embodiment of the present invention;

[0061] Figure 6 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 4 ;

[0062] Figure 7 is a schematic structural diagram of a cataloging device provided by an embodiment of the present invention;

[0063] Figure 8 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] Figure 1 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 1 ,like Figure 1 As shown, the method may include:

[0066] Step 101, obtaining a target text; the target text includes a first text, directory data, and a second text.

[0067] The cataloging method provided in the embodiment of the present application can be applied to a cataloging system. During the cataloging process, the cataloging system can first obtain a target text; the target text includes a first text, directory data, and a second text.

[0068] In this step, the target text may be a text to be processed, and the target text includes a first text, directory data, and a second text.

[0069] Exemplarily, in this step, the first text may be an extracted text, for example, determining in a text set that a target directory area needs to be extracted, that is, determining the first text, which is not limited in the embodiments of the present application.

[0070] For example, in an article, the content of the article includes: "2. Big data era, 2.1 The development and importance of big data, XXXXXXXXX, 3. Data governance of higher education, 3.1 The importance of data governance to higher education; 3.1.1 The need to improve the quality of education; XXXXXX; 3.1.2 The need to improve the scientific nature of decision-making; XXXXXX; 3.1.3 The need to improve management efficiency; XXXXXX; It is not limited in the embodiments of the present application.

[0071] In this step, the second text may be a text to be predicted, that is, other text in the text set except the first text, which is not limited in the embodiments of the present application.

[0072] In this step, the directory data may be the extracted data, for example, the directory data extracted from the first text may be "'First-level directory': '2. Big data era' {'Second-level directory': '2.1 The development and importance of big data'}; 'First-level directory': '3. Data governance of higher education' {'Second-level directory': '3.1 The importance of data governance to higher education' {'Third-level directory': '3.1.1 The need to improve the quality of education', '3.1.2 The need to improve the scientific nature of decision-making', '3.1.3 The need to improve management efficiency' '}, {'Secondary Directory': '3.2 Analytical Framework for Data Governance in Higher Education' {'Third Level Directory': '3.2.1 Acquisition and Extraction', '3.2.2 Integration and Analysis', '3.2.3 Interpretation and Prediction'}}; 'First Level Directory': '5. The Future of Data Governance in Higher Education' {'Secondary Directory': '5.1 Application Trends of Big Data in Higher Education', 'Secondary Directory': '5.2 Challenges and Problems Faced', 'Secondary Directory': '5.3 Future Outlook and Suggestions'}" are not limited in the embodiments of the present application.

[0073] In this step, during the cataloging process, the cataloging system may first obtain the first text, and extract the catalog data and the second text from the first text, where the second text is the text to be predicted.

[0074] Furthermore, in this step, when acquiring the target text, the cataloging system can acquire multimodal data information; convert the multimodal data information into a structured data summary; and perform data cleaning and integration on the structured data summary to obtain a target text that meets preset quality requirements.

[0075] In this step, the multimodal data information may be information from multiple data sources, for example: multimodal data may include text data, image data, audio data, video data, sensor data, biometric data, etc., which is not limited in the embodiments of the present application.

[0076] Exemplarily, in this application, text data is one of the most common data modalities, including natural language text, annotated text, etc. Text data is usually used for natural language processing (NLP) tasks, such as sentiment analysis, machine translation, etc.

[0077] For example, image data is another important data modality, including photos, paintings, etc. Image data plays a key role in computer vision (CV) tasks such as image recognition, image classification, etc.

[0078] Exemplarily, the audio data includes music, speech, etc. Audio processing is widely used in the fields of speech recognition, audio classification, etc.

[0079] For example, video data combines the features of images and audio, including video clips, movies, etc. Video data is very important in tasks such as video understanding and video summarization.

[0080] Exemplarily, sensor data is data collected by various sensors, such as radar, infrared, accelerometer, etc. These data are widely used in the fields of Internet of Things (IoT), environmental monitoring, etc.

[0081] Exemplarily, biometric data includes fingerprints, irises, facial features, etc., which play an important role in the fields of biometrics and identity authentication.

[0082] In addition, multimodal data can also include other forms of information, such as user behavior data, geographic location data, etc. These multimodal data usually have different characteristics and structures, so specific technologies and methods are needed to process and analyze them.

[0083] The structured data summary can be obtained after summarizing and refining the structured data, presenting the key information in the data in a concise and clear manner.

[0084] For example, in a structured data summary, key information usually includes attributes, tuples, and the relationships between the data. For example, in a structured data such as a transcript, the attributes may include subject name, score, etc., while the tuples correspond to the specific scores of each student. The structured data summary may present this information in a concise table, chart, or text form so that users can quickly understand the overall situation and key details of the data.

[0085] It should be noted that the structured data summary should accurately reflect the key information of the original data and avoid omissions or misleading.

[0086] Exemplarily, data cleaning and integration can be a process of quality screening of structured data summaries. The reason for cleaning and integrating the data is that the standards of the acquired text data or text information are different, the descriptions are also inconsistent, and there is an overall low quality problem, so data cleaning processing is required.

[0087] For example, in the process of data cleaning, there may be data classification errors or overlaps; the data description is not standardized, the attributes are filled in incorrectly or key attributes are missing: the data description contains special characters such as "-", "—", " / ", "*", "x", etc., which are not limited in the embodiments of this application.

[0088] Exemplarily, the quality requirements for data input into the multimodal model need to meet preset quality requirements, for example: the data cannot contain certain special characters, or the data cannot have extra spaces, etc., which are not limited in the embodiments of the present application.

[0089] Exemplarily, the target text may be a text that meets the screening conditions, that is, the data format of the target text is unified, and the quality of the data meets the preset quality requirements, which is not limited in the embodiments of the present application.

[0090] In this step, in the process of obtaining the target text, it is necessary to unify the format of multimodal data with inconsistent formats, and it is also necessary to clean and integrate the data after the format unification, so as to obtain the target text that meets the preset quality requirements.

[0091] Furthermore, in an embodiment of the present application, after acquiring the target text, the cataloging system may also determine a storage method according to the features and scene information carried by the target text, and store the target text based on the storage method.

[0092] In this step, the features carried by the target text may be structured features, unstructured features, or semi-structured features, which are not limited in the embodiments of the present application.

[0093] In this step, the scene information may be any different scene information, for example, it may be data information in a traffic scene, data information in a library, etc., which is not limited in the embodiments of the present application.

[0094] In this step, the storage method may be to store different target texts according to the same storage method or according to different storage methods, which is not limited in the embodiments of the present application.

[0095] In this step, the process of storing the target text may be storing the target text in a structured manner, which is not limited in the embodiments of the present application.

[0096] In this step, during the storage process, the target text may be stored using a corresponding storage method according to the features and scene information carried by the target text.

[0097] Step 102: Determine a text sample according to the first text and the directory data.

[0098] In the cataloging method provided in the embodiment of the present application, after acquiring the target text, the cataloging system can determine the text sample according to the first text and the directory data.

[0099] In this step, the text sample may be determined based on the first text and the directory data. For example, the text sample is determined by a plurality of first texts and a plurality of directory data, which is not limited in the embodiments of the present application.

[0100] In this step, during the cataloging process, the cataloging system may first determine a text sample based on the first text and directory data in the acquired target text.

[0101] Step 103: Based on the mapping function, the text sample and the second text are mapped into a matrix set corresponding to the third text and the mask.

[0102] The cataloging method provided in the embodiment of the present application can, after determining the text sample based on the first text and the directory data, map the text sample and the second text into a matrix set corresponding to the third text and the mask based on a mapping function.

[0103] In this step, the mapping function may be a defined function that can perform mapping processing, which is not limited in the embodiments of the present application.

[0104] In this step, the third text may be a directory data extraction prompt, for example, the text in a matrix set obtained by mapping the text sample and the second text through a mapping function.

[0105] In this step, the mask can be used to indicate the part that the model should predict, which is not limited in the embodiments of the present application.

[0106] In this step, the matrix set corresponding to the third text and the mask may be obtained by mapping the text sample and the second text through a mapping function. The matrix set may include multiple rows and columns, which is not limited in the embodiments of the present application.

[0107] In this step, during the cataloging process, the cataloging system can also use a mapping function to map the text sample and the second text into a matrix set corresponding to the third text and the mask. The matrix set may include multiple rows and columns of data, and the multiple rows and columns of data are processed data.

[0108] Step 104: construct a parallel structure information group, which is composed of a task example and a fourth text.

[0109] The cataloging method provided in the embodiment of the present application can also construct a parallel structure information group after mapping the text sample and the second text into a matrix set corresponding to the third text and the mask based on a mapping function. The parallel structure information group is composed of a task example and a fourth text.

[0110] In this step, the parallel structure information group can be obtained by combining two types of data information, which can improve the data processing speed.

[0111] In this step, the task example may be an example pre-set by the cataloging system. Depending on the task example, the target directory obtained during the cataloging process may be different, which is not limited in the embodiments of the present application.

[0112] In this step, the fourth text may be the text to be predicted, that is, the fourth text and the task example may be input into the multimodal model, and then the target directory corresponding to the fourth text may be obtained.

[0113] Furthermore, in this step, when constructing the parallel structure information group, the cataloging system can add a first tag to the task example and a second tag to the fourth text; and train the parallel structure information group according to the first tag and the second tag.

[0114] In this step, the first mark may be a label added to the sample example, and the multimodal model may distinguish the sample example from the text to be predicted based on the first mark.

[0115] It should be noted that the second mark may be a label added to the fourth text, and the multimodal model may distinguish the sample example from the fourth text according to the second mark.

[0116] In this step, in the process of constructing the joint structure information group, the cataloging system can label the task example and the fourth text respectively, and then in the training process, they can be distinguished according to the labels.

[0117] Step 105: Input the parallel structure information group into the multimodal model to perform directory prediction and determine the target directory corresponding to the fourth text.

[0118] In the cataloging method provided in the embodiment of the present application, after constructing the parallel structure information group, the cataloging system can also input the parallel structure information group into the multimodal model for catalog prediction to determine the target catalog corresponding to the fourth text.

[0119] In this step, the multimodal model may include a model that uses a pre-trained model based on Transformer using Hybrid Models and a conditional random field (CRF) to implement directory data recognition, which is not limited in the embodiments of the present application.

[0120] In this step, the catalog prediction may be a process of predicting the text to be predicted.

[0121] The target directory corresponding to the fourth text may be a target directory obtained by performing directory prediction on the fourth text through a multimodal model.

[0122] In this step, during the cataloging process, the text to be predicted and the task examples are simultaneously input into the multimodal model, and a target catalog of the text to be predicted can be obtained.

[0123] Furthermore, during the cataloging process, the cataloging system can also extract the probability of each feature node corresponding to the first text through a linear discriminant analysis algorithm; calculate the similarity of each feature node; obtain a tree structure based on the similarity through a hierarchical clustering algorithm, and divide the tree structure; determine the community structure corresponding to the first text based on the divided tree structure; and determine the output text corresponding to the first text based on the community structure.

[0124] In this step, the implementation process of the linear discriminant analysis algorithm may include: calculating the mean vector of each category, calculating the global mean vector, calculating the between-class scatter matrix and the within-class scatter matrix, solving the generalized eigenvalue problem, projecting the data into a low-dimensional space, etc., which is not limited in the embodiments of the present application.

[0125] Exemplarily, the process of calculating the mean vector of each category may include, for each category, calculating the mean vector of all its samples, which represents the central position of the category in the feature space.

[0126] Exemplarily, the process of calculating the global mean vector may include calculating the global mean vector of all samples, which represents the central position of the entire data set in the feature space.

[0127] For example, the inter-class scatter matrix reflects the degree of dispersion between different classes, while the intra-class scatter matrix reflects the degree of dispersion of samples within the same class.

[0128] Exemplarily, solving the generalized eigenvalue problem may include solving the generalized eigenvalue problem of the inter-class scatter matrix and the intra-class scatter matrix to find the best projection direction, which can maximize the ratio of the inter-class distance to the intra-class distance, thereby obtaining the best classification effect.

[0129] Exemplarily, the process of projecting data into a low-dimensional space may include projecting the original data into a low-dimensional space formed by an optimal projection direction to obtain data after dimensionality reduction.

[0130] In this step, in the process of determining the probability of each characteristic node, each characteristic node may be determined first, and then the corresponding probability of each characteristic node may be determined.

[0131] Exemplarily, each characteristic node probability may be any value greater than 0, for example, the characteristic node probability may be 0.8, or 0.6, which is not limited in the embodiments of the present application.

[0132] Exemplarily, each feature node may be “data governance”, “higher education”, “improving education quality”, etc., which is not limited in the embodiments of the present application.

[0133] In this step, the similarity of each feature node may be determined to be the similarity corresponding to features from directories at different levels.

[0134] For example, in the embodiments of the present application, the similarity between the two feature nodes "Application trends of big data in higher education (from 5.1) and "Challenges and problems faced" (from 5.2)" is 0.2; or the similarity between the two feature nodes "Acquisition and extraction (from 3.2.1) and "Integration and analysis" (from 3.2.2)" is 0.6, which is not limited in the embodiments of the present application.

[0135] It should be noted that the basic idea of ​​the hierarchical clustering algorithm is to construct the data set into a hierarchical structure, in which each sample is initially represented as a separate cluster, and then gradually merge the clusters into larger clusters (agglomerative clustering) by calculating the similarity or distance between samples, or gradually split the entire data set into smaller clusters (divisive clustering). The whole process can be represented as a tree structure, called a cluster tree or dendrogram.

[0136] Exemplarily, the tree structure may be a tree structure formed according to a plurality of node pairs.

[0137] In this step, the community structure corresponding to the first text may be a community structure determined based on a plurality of node pairs.

[0138] Exemplarily, according to multiple node pairs, where the node pairs include: "Acquisition and extraction; integration and analysis"; "Improvement of education quality; Improvement of scientific decision-making"; "Improvement of education quality; Improvement of scientific decision-making"; the community structure determined according to the above node pairs may include: "About the importance of data governance (including relevant content under 3.1)"; "About data governance operations (including relevant content under 3.2)"; "About the future of higher education data governance (including relevant content under 5.1, 5.2, and 5.3)", etc., which are not limited in the embodiments of the present application.

[0139] In this step, the output text corresponding to the first text can be a determined target directory. For example, the output text can be "first-level directory: inventions on the importance of data governance; second-level directory: the need to improve the quality of education, the need to improve the scientific nature of decision-making, the need to improve management efficiency", etc., which is not limited in the embodiments of the present application.

[0140] In this step, based on the cleaned directory, the cataloging system can also perform final classification, which is also the most fine-grained classification of document directory data, for subsequent directory data management and query.

[0141] Furthermore, during the cataloging process, the cataloging system can also input the matrix set into the multimodal model for catalog prediction during the model training phase to obtain an output text corresponding to the third text; based on the output text, extract the target catalog corresponding to the mask.

[0142] Exemplarily, the cataloging system may extract the prompt text according to the target data, and determine the output text corresponding to the third text without performing any additional training.

[0143] Exemplarily, the target directory corresponding to the mask may be a target directory determined according to the output text corresponding to the third text.

[0144] In this step, the cataloging system can directly input the matrix set into the multimodal model for catalog prediction, first obtaining the output text corresponding to the third text; and then extracting the target catalog corresponding to the mask based on the output text.

[0145] In summary, in an embodiment of the present application, the cataloging system can obtain a target text; the target text includes a first text, directory data, and a second text; according to the first text and the directory data, a text sample is determined; based on a mapping function, the text sample and the second text are mapped to a matrix set corresponding to a third text and a mask; a parallel structure information group is constructed, and the parallel structure information group is composed of a task example and a fourth text; the parallel structure information group is input into a multimodal model for directory prediction, and a target directory corresponding to the fourth text is determined. In this way, through the first text and directory data in the target text, a text sample is first determined, and then the text sample and the second text are mapped, and a matrix set after mapping can be obtained, and the matrix set is composed of a third text and a mask. At the same time, the cataloging system can construct a parallel structure information group, and the parallel structure information group includes a task example and a fourth text, that is, the task example and the fourth text are input into the multimodal model, while obtaining a target directory with higher accuracy, the efficiency of cataloging can also be improved.

[0146] Figure 2 This is a step flow of a cataloging method provided by an embodiment of the present invention. Figure 2 ,like Figure 2 As shown, the method may include:

[0147] Step 201: Acquire multimodal data information.

[0148] During the cataloging process, the cataloging system can first obtain multimodal data information. The multimodal data information can be information from multiple data sources. For example, multimodal data can include text data, image data, audio data, video data, sensor data, biometric data, etc., which is not limited in the embodiments of the present application.

[0149] Step 202: Convert the multimodal data information into a structured data summary.

[0150] In the cataloging method provided in the embodiment of the present application, after acquiring the multimodal data information, the cataloging system can convert the multimodal data information into a structured data summary.

[0151] For example, in a structured data summary, key information usually includes attributes, tuples, and the relationships between the data. For example, in a structured data such as a transcript, the attributes may include subject name, score, etc., while the tuples correspond to the specific scores of each student. The structured data summary may present this information in a concise table, chart, or text form so that users can quickly understand the overall situation and key details of the data.

[0152] It should be noted that the structured data summary should accurately reflect the key information of the original data and avoid omissions or misleading.

[0153] Step 203: clean and integrate the structured data summary to obtain a target text that meets preset quality requirements; the target text includes the first text, the directory data, and the second text.

[0154] The cataloging method provided in the embodiment of the present application, after converting the multimodal data information into a structured data summary, the cataloging system can perform data cleaning and integration on the structured data summary to obtain a target text that meets the preset quality requirements; the target text includes a first text, directory data, and a second text.

[0155] Exemplarily, data cleaning and integration can be a process of quality screening of structured data summaries. The reason for cleaning and integrating the data is that the standards of the acquired text data or text information are different, the descriptions are also inconsistent, and there is an overall low quality problem, so data cleaning processing is required.

[0156] Exemplarily, the quality requirements for data input into the multimodal model need to meet preset quality requirements, for example: the data cannot contain certain special characters, or the data cannot have extra spaces, etc., which are not limited in the embodiments of the present application.

[0157] Exemplarily, the target text may be a text that meets the screening conditions, that is, the data format of the target text is unified, and the quality of the data meets the preset quality requirements, which is not limited in the embodiments of the present application.

[0158] Step 204: Determine a text sample based on the first text and the directory data.

[0159] In the cataloging method provided in the embodiment of the present application, after the structured data summary is cleaned and integrated to obtain the target text that meets the preset quality requirements, the cataloging system can determine the text sample based on the first text and the directory data.

[0160] In this step, the text sample may be determined based on the first text and the directory data. For example, the text sample is determined by a plurality of first texts and a plurality of directory data, which is not limited in the embodiments of the present application.

[0161] In this step, during the cataloging process, the cataloging system may first determine a text sample based on the first text and directory data in the acquired target text.

[0162] Step 205: Based on the mapping function, the text sample and the second text are mapped into a matrix set corresponding to the third text and the mask.

[0163] The cataloging method provided in the embodiment of the present application can, after determining the text sample based on the first text and the directory data, map the text sample and the second text into a matrix set corresponding to the third text and the mask based on a mapping function.

[0164] In this step, the mapping function may be a defined function that can perform mapping processing, which is not limited in the embodiments of the present application.

[0165] In this step, the third text may be a directory data extraction prompt, for example, the text in a matrix set obtained by mapping the text sample and the second text through a mapping function.

[0166] In this step, the mask can be used to indicate the part that the model should predict, which is not limited in the embodiments of the present application.

[0167] In this step, the matrix set corresponding to the third text and the mask may be obtained by mapping the text sample and the second text through a mapping function. The matrix set may include multiple rows and columns, which is not limited in the embodiments of the present application.

[0168] In this step, during the cataloging process, the cataloging system can also use a mapping function to map the text sample and the second text into a matrix set corresponding to the third text and the mask. The matrix set may include multiple rows and columns of data, and the multiple rows and columns of data are processed data.

[0169] Step 206: Add a first tag to the task example, and add a second tag to the fourth text.

[0170] The cataloging method provided in the embodiment of the present application maps the text sample and the second text into a matrix set corresponding to the third text and the mask based on a mapping function, and the cataloging system can add a first tag to the task example and a second tag to the fourth text.

[0171] In this step, the first mark may be a label added to the sample example, and the multimodal model may distinguish the sample example from the text to be predicted based on the first mark.

[0172] It should be noted that the second mark may be a label added to the fourth text, and the multimodal model may distinguish the sample example from the fourth text according to the second mark.

[0173] Step 207: Train the union structure information group according to the first label and the second label.

[0174] In the cataloging method provided in the embodiment of the present application, after adding a first tag to the task example and adding a second tag to the fourth text, the cataloging system can train the parallel structure information group according to the first tag and the second tag.

[0175] In this step, in the process of constructing the joint structure information group, the cataloging system can label the task example and the fourth text respectively, and then in the training process, they can be distinguished according to the labels.

[0176] Step 208: Input the parallel structure information group into the multimodal model to perform directory prediction, and determine the target directory corresponding to the fourth text.

[0177] In the cataloging method provided in the embodiment of the present application, after the parallel structure information group is trained according to the first label and the second label, the cataloging system can input the parallel structure information group into the multimodal model for catalog prediction to determine the target catalog corresponding to the fourth text.

[0178] In this step, the multimodal model may include a model that uses a pre-trained model based on Transformer using Hybrid Models and a conditional random field (CRF) to implement directory data recognition, which is not limited in the embodiments of the present application.

[0179] In this step, the catalog prediction may be a process of predicting the text to be predicted.

[0180] The target directory corresponding to the fourth text may be a target directory obtained by performing directory prediction on the fourth text through a multimodal model.

[0181] In this step, during the cataloging process, the text to be predicted and the task examples are simultaneously input into the multimodal model, and a target catalog of the text to be predicted can be obtained.

[0182] Furthermore, during the cataloging process, the cataloging system can also extract the probability of each feature node corresponding to the first text through a linear discriminant analysis algorithm; calculate the similarity of each feature node; obtain a tree structure based on the similarity through a hierarchical clustering algorithm, and divide the tree structure; determine the community structure corresponding to the first text based on the divided tree structure; and determine the output text corresponding to the first text based on the community structure.

[0183] In this step, the implementation process of the linear discriminant analysis algorithm may include: calculating the mean vector of each category, calculating the global mean vector, calculating the between-class scatter matrix and the within-class scatter matrix, solving the generalized eigenvalue problem, projecting the data into a low-dimensional space, etc., which is not limited in the embodiments of the present application.

[0184] Exemplarily, the process of calculating the mean vector of each category may include, for each category, calculating the mean vector of all its samples, which represents the central position of the category in the feature space.

[0185] Exemplarily, the process of calculating the global mean vector may include calculating the global mean vector of all samples, which represents the central position of the entire data set in the feature space.

[0186] For example, the inter-class scatter matrix reflects the degree of dispersion between different classes, while the intra-class scatter matrix reflects the degree of dispersion of samples within the same class.

[0187] Exemplarily, solving the generalized eigenvalue problem may include solving the generalized eigenvalue problem of the inter-class scatter matrix and the intra-class scatter matrix to find the best projection direction, which can maximize the ratio of the inter-class distance to the intra-class distance, thereby obtaining the best classification effect.

[0188] Exemplarily, the process of projecting data into a low-dimensional space may include projecting the original data into a low-dimensional space formed by an optimal projection direction to obtain data after dimensionality reduction.

[0189] In this step, in the process of determining the probability of each characteristic node, each characteristic node may be determined first, and then the corresponding probability of each characteristic node may be determined.

[0190] Exemplarily, each characteristic node probability may be any value greater than 0, for example, the characteristic node probability may be 0.8, or 0.6, which is not limited in the embodiments of the present application.

[0191] Exemplarily, each feature node may be “data governance”, “higher education”, “improving education quality”, etc., which is not limited in the embodiments of the present application.

[0192] In this step, the similarity of each feature node may be determined to be the similarity corresponding to features from directories at different levels.

[0193] For example, in the embodiments of the present application, the similarity between the two feature nodes "Application trends of big data in higher education (from 5.1) and "Challenges and problems faced" (from 5.2)" is 0.2; or the similarity between the two feature nodes "Acquisition and extraction (from 3.2.1) and "Integration and analysis" (from 3.2.2)" is 0.6, which is not limited in the embodiments of the present application.

[0194] It should be noted that the basic idea of ​​the hierarchical clustering algorithm is to construct the data set into a hierarchical structure, in which each sample is initially represented as a separate cluster, and then gradually merge the clusters into larger clusters (agglomerative clustering) by calculating the similarity or distance between samples, or gradually split the entire data set into smaller clusters (divisive clustering). The whole process can be represented as a tree structure, called a cluster tree or dendrogram.

[0195] Exemplarily, the tree structure may be a tree structure formed according to a plurality of node pairs.

[0196] In this step, the community structure corresponding to the first text may be a community structure determined based on a plurality of node pairs.

[0197] Exemplarily, according to multiple node pairs, where the node pairs include: "Acquisition and extraction; integration and analysis"; "Improvement of education quality; Improvement of scientific decision-making"; "Improvement of education quality; Improvement of scientific decision-making"; the community structure determined according to the above node pairs may include: "About the importance of data governance (including relevant content under 3.1)"; "About data governance operations (including relevant content under 3.2)"; "About the future of higher education data governance (including relevant content under 5.1, 5.2, and 5.3)", etc., which are not limited in the embodiments of the present application.

[0198] In this step, the output text corresponding to the first text can be a determined target directory. For example, the output text can be "first-level directory: inventions on the importance of data governance; second-level directory: the need to improve the quality of education, the need to improve the scientific nature of decision-making, the need to improve management efficiency", etc., which is not limited in the embodiments of the present application.

[0199] In this step, based on the cleaned directory, the cataloging system can also perform final classification, which is also the most fine-grained classification of document directory data, for subsequent directory data management and query.

[0200] Furthermore, during the cataloging process, the cataloging system can also input the matrix set into the multimodal model for catalog prediction during the model training phase to obtain an output text corresponding to the third text; based on the output text, extract the target catalog corresponding to the mask.

[0201] Exemplarily, the cataloging system may extract the prompt text according to the target data, and determine the output text corresponding to the third text without performing any additional training.

[0202] Exemplarily, the target directory corresponding to the mask may be a target directory determined according to the output text corresponding to the third text.

[0203] In this step, the cataloging system can directly input the matrix set into the multimodal model for catalog prediction, first obtaining the output text corresponding to the third text; and then extracting the target catalog corresponding to the mask based on the output text.

[0204] In summary, in an embodiment of the present application, the cataloging system can obtain a target text; the target text includes a first text, directory data, and a second text; according to the first text and the directory data, a text sample is determined; based on a mapping function, the text sample and the second text are mapped to a matrix set corresponding to a third text and a mask; a parallel structure information group is constructed, and the parallel structure information group is composed of a task example and a fourth text; the parallel structure information group is input into a multimodal model for directory prediction, and a target directory corresponding to the fourth text is determined. In this way, through the first text and directory data in the target text, a text sample is first determined, and then the text sample and the second text are mapped, and a matrix set after mapping can be obtained, and the matrix set is composed of a third text and a mask. At the same time, the cataloging system can construct a parallel structure information group, and the parallel structure information group includes a task example and a fourth text, that is, the task example and the fourth text are input into the multimodal model, while obtaining a target directory with higher accuracy, the efficiency of cataloging can also be improved.

[0205] Reference Figure 3 , showing the steps of a cataloging method provided in an embodiment of the present application Figure 3 ,like Figure 3 As shown, the method specifically includes steps 301 to 306:

[0206] Step 301: Multimodal data acquisition module.

[0207] The multimodal data acquisition module can collect multimodal data. Design a collection system that supports access to multiple data sources, based on a distributed architecture, using multi-threaded parallel technology, and simultaneously processing multi-source unstructured data to achieve connection with different data sources and data capture.

[0208] Step 302: Multimodal data storage module.

[0209] The multimodal data storage module focuses on the reliability and accessibility of data. For multimodal data, the appropriate storage method is selected according to its characteristics and application scenarios. A distributed storage architecture is used to store unstructured data on multiple nodes in a scalable manner to ensure data security and reliability, and facilitate the model's access to and reading of data.

[0210] Step 303: Multimodal data format unification module.

[0211] Multimodal data is formatted in a unified way through semantic understanding. This module extracts information from unstructured or semi-structured multimodal data and converts it into structured data summaries, which are then output in a unified text format.

[0212] Step 304: Multimodal data cleaning and integration module.

[0213] The multimodal data collection module collects various data and information, but the results have problems such as inconsistent data standards, irregular descriptions, and low overall quality, and data cleaning is urgently needed. The difficulties in the data cleaning process include: incorrect or overlapping data classification; irregular data description, incorrect attribute filling or missing key attributes: data description contains special characters such as "-", "—", " / ", "*", "x", etc.

[0214] The research team used natural language processing (NLP) technology to develop a set of multimodal data cleaning tools to perform intelligent cleaning on large amounts of data, specifically solving the difficulties in multimodal data cleaning. The tool can achieve the following two functions: intelligent identification and classification: through the algorithm catalog extraction function, it can automatically identify the classification attribution, and the classification that cannot be recommended can be supplemented manually; intelligent identification of key parameters: through the algorithm to identify the key parameters of multimodal data and form a structured data description.

[0215] With the help of multimodal data cleaning tools, historical data were cleaned, reclassified according to new categories, and data descriptions were identified and processed according to algorithms, solving problems such as sequence, special characters, and non-standard descriptions.

[0216] Step 305: Multimodal data directory generation module.

[0217] Taking the multimodal (Modulated Detection for End-to-End Multi-ModalUnderstanding, MDETR) model as an example, the concept of parallel structured information group is introduced as a prompt engineering, so that it can perform a variety of user-specified tasks without any weight fine-tuning. This method is applicable to all baseline models of self-supervised learning.

[0218] MDETR is an NLP-based model for multimodal object detection and segmentation, designed to parse and understand the layout structure in text. It combines traditional text classification and layout analysis techniques to jointly model and predict text and layout information.

[0219] The MDETR model learns cross-modal representations through a unified text multimodal Transformer, uses multimodal contrastive learning to train the model, realizes information conversion through multi-head self-attention and feedforward neural network layers, and combines residual connections and normalization to improve training effects.

[0220] The model preparation phase of MDETR can be divided into two main steps: pre-training. In the pre-training phase, the model is trained with large-scale unlabeled text and layout data to learn the correlation between text and layout. It adopts a multi-task learning framework to simultaneously predict text classification and layout structure to capture the complex relationship between text and layout.

[0221] Step 306: Directory data output module.

[0222] After directory data extraction and cleaning, a directory data directory is finally formed. NLP technology is used for directory extraction, to identify entities in the document directory data, remove modifying content, and obtain the final classification, which is also the base name of the document.

[0223] In the directory extraction stage, the directory information y at all levels vp1 ,y vp2 ,y vp3 , ..., the hybrid model HybridModels is used to realize directory data recognition based on the pre-trained model of Transformer + Conditional Random Fields CRF (Conditional Random Fields, CRF). Transformer and CRF are constructed by combining the advantages of multiple technologies. Transformer obtains high-quality language representation through large-scale pre-training learning, and then can automatically extract features in documents through fine-tuning. Adding a CRF layer to model the features extracted by Transformer, especially processing the transfer relationship between features, can improve the accuracy of directory data recognition. Clean the identified entities, such as personnel, channels, and methods, and remove modifying content, such as redundant descriptive text and symbols. Based on the cleaned directory, the final level classification is performed, which is also the most fine-grained classification of document directory data, which is used for subsequent directory data management and query.

[0224] Reference Figure 4 , shows a flowchart of the steps of a directory data extraction method provided by an embodiment of the present application, such as Figure 4 As shown, the method specifically includes steps 401 to 404:

[0225] Step 401: Input target text.

[0226] The input data consists of three parts: x i The extracted text, and y i Corresponding to the extracted directory data for model learning, x q Corresponding to the text to be predicted.

[0227] Step 402: Build a directory data extraction prompt.

[0228] Defining input and output examples Extract samples for input and output text, where x i is the input text, y i is the extracted directory data. Given a new text to be predicted x q , the goal is to predict the corresponding y q .

[0229] Define a function F that takes the input output sample S and the text to be predicted x q Mapped into new text and mask: [x vp ,m]=F(S,x q ). Among them, x vp is a hint for extracting catalog data, and the mask m defines the part that the model should predict.

[0230] The goal of the model is to vp , outputs y without performing any additional training vp , and take out the corresponding part to get y q .

[0231] Step 403: construct a union structure information matrix.

[0232] Construct a parallel structure information group consisting of task input and output examples and new text to be predicted. The task input and output examples and the new text to be predicted are marked to facilitate model distinction.

[0233] Step 404: Input to the model for catalog prediction.

[0234] The MDETR model is fed with the parallel structure information set. The model observes x i and x vp , will try to understand the prompt and generate the corresponding result based on the knowledge and language understanding ability acquired in the pre-training phase. Model recognition and generation of directory data extraction prompt x vp , then according to x vp Output y vp1 ,y vp2 ,y vp3 , ..., and take out the part y corresponding to the mask m q .

[0235] Reference Figure 5 , shows a flowchart of the steps of constructing a directory data extraction prompt provided by an embodiment of the present application, such as Figure 5 As shown, the method specifically includes steps 501 to 504:

[0236] Step 501: Define input and output samples S.

[0237] Defining input and output examples Extract samples for input and output text, where x i is the input text, y i is the extracted directory data. Given a new text to be predicted x q , the goal is to predict the corresponding y q .

[0238] Step 502: Given a text x to be predicted q , output the prediction result y q .

[0239] The cataloging system can use the model to obtain the output catalog prediction results based on the input text to be predicted.

[0240] Step 503: Define function F, input and output sample S and the text to be predicted x q Map into new text and mask.

[0241] Define a function F that takes the input output sample S and the text to be predicted x q Mapped into new text and mask: [x vp ,m]=F(S,x q ). Among them, x vp is a hint for extracting catalog data, and the mask m defines the part that the model should predict.

[0242] The goal of the model is to vp , outputs y without performing any additional training vp , and take out the corresponding part to get y q .

[0243] Step 504: construct a directory data extraction prompt.

[0244] Reference Figure 6 , showing the steps of a cataloging method provided in an embodiment of the present application Figure 4 ,like Figure 6 As shown, the method specifically includes steps 601 to 605:

[0245] Step 601: Input sample.

[0246] Based on the cleaned directory, the final level classification is performed, which is also the most fine-grained classification of document directory data, and is used for subsequent directory data management and query.

[0247] The cataloging system can first input samples for subsequent processing.

[0248] Step 602: extract the probability corresponding to the feature node through the LDA algorithm.

[0249] After inputting the sample, the cataloging system can also extract the probabilities corresponding to the feature nodes through the LDA algorithm.

[0250] Step 603: Calculate the similarity of feature nodes.

[0251] After extracting the probabilities corresponding to the feature nodes through the LDA algorithm, the cataloging system can also calculate the similarities of the feature nodes.

[0252] Step 604: Form a tree structure and divide it by a hierarchical clustering algorithm to obtain a community structure.

[0253] After calculating the similarity of feature nodes, the cataloging system can also form a tree structure and split it through a hierarchical clustering algorithm to obtain a community structure.

[0254] Step 605: Determine output samples based on the community structure.

[0255] After forming and dividing the tree structure through the hierarchical clustering algorithm to obtain the community structure, the cataloging system can also determine the output samples based on the community structure.

[0256] Figure 7 is a schematic diagram of the structure of a cataloging device provided by an embodiment of the present invention, such as Figure 7 As shown, the device may specifically include:

[0257] The acquisition module 701 is used to acquire a target text; the target text includes a first text, directory data, and a second text;

[0258] A determination module 702, configured to determine a text sample according to the first text and the directory data;

[0259] A mapping module 703, configured to map the text sample and the second text to a matrix set corresponding to a third text and a mask based on a mapping function;

[0260] A construction module 704 is used to construct a parallel structure information group, wherein the parallel structure information group is composed of a task example and a fourth text;

[0261] The determination module is further used to input the parallel structure information group into a multimodal model for directory prediction, and determine a target directory corresponding to the fourth text.

[0262] The embodiment of the present invention provides a cataloging device, which obtains a target text; the target text includes a first text, catalog data, and a second text; a text sample is determined according to the first text and the catalog data; based on a mapping function, the text sample and the second text are mapped to a matrix set corresponding to a third text and a mask; a parallel structure information group is constructed, and the parallel structure information group consists of a task example and a fourth text; the parallel structure information group is input into a multimodal model for catalog prediction, and a target catalog corresponding to the fourth text is determined. In this way, the text sample is first determined through the first text and the catalog data in the target text, and then the text sample and the second text are mapped, and a matrix set after mapping can be obtained, and the matrix set consists of the third text and the mask. At the same time, the cataloging system can construct a parallel structure information group, and the parallel structure information group includes a task example and a fourth text, that is, the task example and the fourth text are input into the multimodal model, and while obtaining a target catalog with higher accuracy, the efficiency of cataloging can also be improved.

[0263] Optionally, the acquisition module 701 includes:

[0264] A first acquisition submodule is used to acquire multimodal data information;

[0265] A conversion module, used for converting the multimodal data information into a structured data summary;

[0266] The module is used to clean and integrate the structured data summary to obtain a target text that meets the preset quality requirements.

[0267] Optionally, the device further comprises:

[0268] An adding module, configured to add a first mark to the task example and a second mark to the fourth text;

[0269] A training module is used to train the union structure information group according to the first label and the second label.

[0270] An extraction module, used to extract the probability of each feature node corresponding to the first text through a linear discriminant analysis algorithm;

[0271] A calculation module, used to calculate the similarity of each feature node;

[0272] A segmentation module, used to obtain a tree structure according to the similarity through a hierarchical clustering algorithm, and segment the tree structure;

[0273] A determination submodule is used to determine the community structure corresponding to the first text based on the segmented tree structure; and determine the output text corresponding to the first text according to the community structure.

[0274] Optionally, the device further comprises:

[0275] Obtaining a submodule, used for inputting the matrix set into a multimodal model to perform directory prediction in a model training phase, and obtaining an output text corresponding to the third text;

[0276] The extraction submodule is used to extract a target directory corresponding to the mask based on the output text.

[0277] Optionally, the device further comprises:

[0278] The first determination module is used to determine a storage method according to the features and scene information carried by the target text, and store the target text based on the storage method.

[0279] The present invention also provides an electronic device, see Figure 8 , including: a processor 801, a memory 802, and a computer program 8021 stored in the memory and executable on the processor, wherein the processor implements the cataloging method of the aforementioned embodiment when executing the program.

[0280] The present invention also provides a readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the cataloging method of the aforementioned embodiment.

[0281] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0282] The algorithms and catalogs provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems may also be used together with the teachings based thereon. According to the above description, it is apparent that the structure required for constructing such systems is not intended for any particular programming language. In addition, the present invention is not intended for any particular programming language either. It should be understood that various programming languages ​​may be utilized to implement the content of the present invention described herein, and the description of the particular language above is intended to disclose the best mode of the present invention.

[0283] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0284] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0285] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0286] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0287] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0288] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0289] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0290] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0291] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A cataloguing method, characterized in that: Applied to a cataloging system, the method comprises: Acquire a target text; the target text includes a first text, directory data, and a second text; Determining a text sample according to the first text and the directory data; Based on a mapping function, mapping the text sample and the second text to a matrix set corresponding to a third text and a mask; Constructing a parallel structure information group, wherein the parallel structure information group is composed of a task example and a fourth text; The parallel structure information group is input into a multimodal model for directory prediction to determine a target directory corresponding to the fourth text.

2. The method according to claim 1, characterized in that The obtaining of the target text comprises: Obtain multimodal data information; Converting the multimodal data information into a structured data summary; The structured data summary is cleaned and integrated to obtain a target text that meets preset quality requirements.

3. The method according to claim 1, characterized in that The construction and type structure information group includes: adding a first tag to the task example, and adding a second tag to the fourth text; The union structure information group is trained according to the first label and the second label.

4. The method according to claim 1, characterized in that The method further comprises: Extracting the probability of each feature node corresponding to the first text by a linear discriminant analysis algorithm; Calculating the similarity of each feature node; Obtaining a tree structure according to the similarity through a hierarchical clustering algorithm, and dividing the tree structure; Determining a community structure corresponding to the first text based on the segmented tree structure; An output text corresponding to the first text is determined according to the community structure.

5. The method according to claim 1, characterized in that The method further comprises: In the model training stage, the matrix set is input into the multimodal model to perform directory prediction, and an output text corresponding to the third text is obtained; Based on the output text, a target directory corresponding to the mask is extracted.

6. The method according to claim 1, characterized in that After acquiring the target text, the method further includes: A storage method is determined according to the features and scene information carried by the target text, and the target text is stored based on the storage method.

7. A cataloguing device, characterized in that: The device comprises: An acquisition module, used for acquiring a target text; the target text includes a first text, directory data, and a second text; A determination module, configured to determine a text sample according to the first text and the directory data; A mapping module, used for mapping the text sample and the second text to a matrix set corresponding to a third text and a mask based on a mapping function; A construction module, used for constructing a parallel structure information group, wherein the parallel structure information group consists of a task example and a fourth text; The determination module is further used to input the parallel structure information group into a multimodal model for directory prediction, and determine a target directory corresponding to the fourth text.

8. The device according to claim 7, characterized in that The acquisition module comprises: A first acquisition submodule is used to acquire multimodal data information; A conversion module, used for converting the multimodal data information into a structured data summary; The module is used to clean and integrate the structured data summary to obtain a target text that meets the preset quality requirements.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cataloging method according to any one of claims 1 to 6 when executing the program.

10. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the cataloging method described in any one of claims 1-6.

Images