Model training method, metadata extraction method and device

Abstract

The invention discloses a model training method and device and a metadata extraction method and device.The model training method comprises the steps that a first training sample set is obtained, the first training sample set comprises a plurality of first metadata entries corresponding to training documents, and the first metadata entries correspond to the training documents; each first metadata item comprises a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one piece of first derivative information corresponding to the first metadata item and a derivative information description corresponding to each piece of first derivative information; aiming at each first metadata entry, correcting the first metadata entry through the target large language model to obtain a second metadata entry corresponding to the first metadata entry; and taking the second training sample set as an external knowledge base, and training the target large language model based on the training document by using a retrieval enhancement strategy, the second training sample set comprising each second metadata entry.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to a model training method and a metadata extraction method and device. BACKGROUND

[0002] Metadata refers to data used to describe data, such as, for an electronic document, including file title, document number, issuing agency, issue date, information classification, keywords, content abstract, and data structure definition or coding rule description, mainly used for standardized storage, access management, and sharing exchange of data. In related technologies, simple template matching technology and keyword extraction technology are usually used for metadata generation, wherein the simple template matching technology extracts relevant metadata information according to a specific format template of an electronic document, thereby generating a metadata directory, and the keyword extraction technology refers to searching and extracting by matching the file title, document number, issuing agency, issue date, and other significantly identifiable metadata items. However, the simple template matching technology is only applicable to electronic documents with a specific format and is used to extract part of the metadata items in a specific position, and the keyword extraction technology can only extract part of the metadata items that can be significantly identified and generally presented in a specific position of the document. Both of the above two methods lack adaptability and have the limitations of low matching accuracy. SUMMARY

[0003] The embodiments of the present application provide a model training method, a metadata extraction method and device, which can improve the efficiency and accuracy of metadata extraction.

[0004] In a first aspect, the embodiments of the present application provide a model training method, which comprises: obtaining a first training sample set, wherein the first training sample set comprises a plurality of first metadata items corresponding to a training document, each first metadata item comprises a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information; for each first metadata item, correcting the first metadata item by a target large language model to obtain a second metadata item corresponding to the first metadata item; taking a second training sample set as an external knowledge base, and training the target large language model based on the training document by using a retrieval enhancement strategy, wherein the second training sample set comprises each second metadata item.

[0005] In a second aspect, an embodiment of the present application provides a metadata extraction method, which comprises: obtaining a target document; inputting the target document into a target large language model to obtain at least one target metadata item output by the target large language model, wherein each target metadata item comprises a document name of the training document, a target metadata item, a semantic vocabulary standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is trained based on the model training method of the first aspect; and outputting the at least one target metadata item.

[0006] In a third aspect, an embodiment of the present application provides a model training device, which comprises: an obtaining module configured to obtain a first training sample set, wherein the first training sample set comprises a plurality of first metadata items corresponding to a training document, and each first metadata item comprises a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information; a correcting module configured to correct, for each first metadata item, the first metadata item by using a target large language model to obtain a second metadata item corresponding to the first metadata item; and a training module configured to use a second training sample set as an external knowledge base, and train the target large language model based on the training document by using a retrieval enhancement strategy, wherein the second training sample set comprises each second metadata item.

[0007] In a fourth aspect, an embodiment of the present application provides a metadata extraction device, which comprises: an obtaining module configured to obtain a target document; a processing module configured to input the target document into a target large language model to obtain at least one target metadata item output by the target large language model, wherein each target metadata item comprises a document name of the training document, a target metadata item, a semantic vocabulary standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is trained based on the model training method of the first aspect; and an output module configured to output the at least one target metadata item.

[0008] In a fifth aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the method of the first aspect or the steps of the method of the second aspect.

[0009] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, wherein a program or instructions are stored on the computer readable storage medium, and the program or instructions are executed by a processor to implement steps of the method according to the first aspect or implement steps of the method according to the second aspect.

[0010] In a seventh aspect, an embodiment of the present application provides a chip, wherein the chip comprises a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement steps of the method according to the first aspect or implement steps of the method according to the second aspect.

[0011] In an eighth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to implement steps of the method according to the first aspect or implement steps of the method according to the second aspect.

[0012] In the embodiment of the present application, a first training sample set is obtained, wherein the first training sample set comprises a plurality of first metadata entries corresponding to a training document, each first metadata entry comprises a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information, then for each first metadata entry, the first metadata entry is corrected by a target large language model to obtain a second metadata entry corresponding to the first metadata entry, and then a second training sample set is used as an external knowledge base, and a retrieval enhancement strategy is used to train the target large language model based on the training document, wherein the second training sample set comprises each second metadata entry, so that the trained target large language model has more accurate semantic matching capability and generalization capability, and can automatically extract metadata items according to context understanding, and is not affected by the position of the metadata items in the document, avoiding limitations such as inaccurate word group segmentation and difficult keyword matching, and by using the external knowledge base, the metadata extraction process can be more automated, efficient and accurate. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart of a model training method provided by an embodiment of the present application; Figure 2 is a flowchart of a metadata extraction method provided by an embodiment of the present application; Figure 3is another flow diagram of a metadata extraction method provided by an embodiment of the present application; Figure 4 is a structural diagram of a model training device provided by an embodiment of the present application; Figure 5 is a structural diagram of a metadata extraction device provided by an embodiment of the present application; Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0015] The model training method, metadata extraction method and device provided by an embodiment of the present application will be described in detail below with reference to the drawings, specific embodiments and application scenarios.

[0016] Figure 1 A model training method provided by an embodiment of the present application is shown, which can be executed by an electronic device. The electronic device can include a server and / or a terminal device. In other words, the method can be executed by software or hardware installed in the electronic device, and the method includes the following steps: S110: Obtain a first training sample set.

[0017] The first training sample set includes a plurality of first metadata items corresponding to a training document, each first metadata item includes a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information.

[0018] It can be understood that the training document includes at least one first metadata item, each first metadata item corresponds to a first metadata entry, that is, the first metadata entry includes the document name of the training document, the first metadata item, the semantic vocabulary standard name corresponding to the first metadata item, the index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and the derivative information description corresponding to each first derivative information, wherein the semantic vocabulary standard name corresponding to the first metadata item refers to the unified and standardized name corresponding to the first metadata item in a specific field or application scenario, so as to ensure its uniqueness, consistency and identifiability in the field or scene. For example, the semantic vocabulary standard name corresponding to the metadata item "computer" and the metadata item "computer" can be "computer". The at least one first derivative information includes but is not limited to the synonym corresponding to the first metadata item, the other kind of language representation corresponding to the first metadata item, the abbreviation of each language representation, and the synonym of each language representation. The derivative information description is used to indicate the association between the first derivative information and the corresponding first metadata item. For example, assuming that the training document is an xx asset information document, as shown in Table 1, a plurality of first metadata entries can include: Table 1

[0019] As shown in Table 1, two first metadata entries are included, the semantic vocabulary standard name corresponding to the first metadata item "asset name" in the first first metadata entry is "asset name", the index position thereof in the xx asset information document is: start position 21, end position 24, the first derivative information corresponding to the first metadata item "asset name" is "asset name", "asset name" is the synonym of "asset name", the second first derivative information corresponding to the first metadata item "asset name" is "Asset Name", "Asset Name" is the English representation of "asset name"; the semantic vocabulary standard name corresponding to the first metadata item "asset type" in the second first metadata entry is "asset type", the index position thereof in the xx asset information document is: start position 32, end position 35, the first derivative information corresponding to the first metadata item "asset type" is "AT", "AT" is the English abbreviation of "asset type", and the second first derivative information corresponding to the first metadata item "asset type" is "Type d'actif", "Type d'actif" is the French representation of "asset type".

[0020] It should be noted that the index position can also be represented by row and column numbers, such as row 1, column 0, or can be represented by page numbers and row and column numbers, such as page 1, row 2, column 2. The description form of the index position in the embodiments of the application is not specifically limited, and can be set according to actual conditions.

[0021] S120: For each of the first metadata entries, the target large language model is used to correct the first metadata entries to obtain second metadata entries corresponding to the first metadata entries.

[0022] The target large language model is a pre-constructed large language model used to extract metadata items in a document.

[0023] It can be understood that in some cases, the initial training data, i.e., the first training sample set, may have noise, errors or missing information. By correcting the first training sample set using the target large language model, the quality of the sample can be improved to make it more consistent with the expected standard of the model. In addition, during the correction process, the target large language model can obtain feedback from the generation process. This feedback mechanism can help the target large language model continuously improve its performance and help improve training efficiency in subsequent training processes.

[0024] Each corrected first metadata entry is a second metadata entry, and each item in the second metadata entry is consistent with each item in the corresponding first metadata entry, or at least one item is inconsistent. Each second metadata entry includes a document name of a training document, a second metadata item, a semantic lexical standard name corresponding to the second metadata item, an index position of the second metadata item in the training document, second derivative information corresponding to the second metadata item, and derivative information description corresponding to each second derivative information. The second metadata item is a first metadata item in the corresponding first metadata entry.

[0025] S130: Use the second training sample set as an external knowledge base, and use a retrieval enhancement strategy to train the target large language model based on the training document.

[0026] The second training sample set includes each of the second metadata entries.

[0027] The retrieval enhancement strategy is a strategy that combines external knowledge retrieval and generation model capabilities, and can be used to enhance the answer quality and accuracy of the target large language model by retrieving relevant information.

[0028] In an example embodiment, the training of the target large language model based on the second training sample set as an external knowledge base and the retrieval enhancement strategy comprises: constructing a third instruction according to the training document and the external knowledge base, inputting the third instruction into the target large language model, and obtaining a third output result output by the target large language model based on the retrieval enhancement strategy, the third output result comprising a plurality of fourth metadata entries corresponding to the training document, each fourth metadata entry comprising a document name of the training document, a fourth metadata item, a semantic vocabulary standard name corresponding to the fourth metadata item, an index position of the fourth metadata item in the training document, at least one fourth derivative information corresponding to the fourth metadata item, and derivative information description corresponding to each fourth derivative information; and optimizing the target large language model by evaluating the third output result.

[0029] It can be understood that metadata extraction generally involves identifying and extracting structured information from a large amount of text data, and the external knowledge base can help the target large language model better understand the semantics and context of the text, thereby improving the accuracy of extraction. For example, assuming that a document contains terms or names, the external knowledge base can provide definitions, background information or synonyms related to them to help the target large language model correctly identify these terms or names. At the same time, in some cases, metadata extraction needs to follow certain standard naming or formats to ensure the consistency of metadata items in different industries or different fields or different scenarios. The second training sample set in the embodiment of the application provides the semantic vocabulary standard name corresponding to each second metadata item included in the training document, so the second training sample set can be used as an external knowledge base to provide the semantic vocabulary standard name corresponding to each second metadata item, helping the target large language model to determine the semantic vocabulary standard name corresponding to the metadata item while extracting the metadata item from the training document. In addition, during the metadata extraction process, there may be cases of semantic ambiguity or ambiguity, for example, some words or phrases have different meanings in different contexts, and the second training sample set can obtain the context information of the second metadata item in the training document based on the index position of the second metadata item, thereby helping the target large language model to eliminate ambiguity according to the specific situation.

[0030] In the embodiments of the present application, a first training sample set is obtained, wherein the first training sample set includes a plurality of first metadata entries corresponding to a training document, each first metadata entry including a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information. Then, for each first metadata entry, the target large language model is used to modify the first metadata entry to obtain a second metadata entry corresponding to the first metadata entry. Then, a second training sample set is used as an external knowledge base, and a retrieval enhancement strategy is used to train the target large language model based on the training document, wherein the second training sample set includes each second metadata entry. The trained target large language model has more accurate semantic matching ability and generalization ability, and can automatically extract metadata items according to context understanding, and is not affected by the position of the metadata items in the document, avoiding limitations such as inaccurate word group segmentation and difficult keyword matching. Moreover, by using the external knowledge base, the metadata extraction process can be more automated, efficient and accurate.

[0031] In an exemplary embodiment, before the first training sample set is obtained, the method can further include the following steps: Step 1: Convert the metadata document into a target file expressed in XML language.

[0032] The metadata document is used to standardize metadata items corresponding to at least one field, each metadata item having a corresponding semantic vocabulary standard name, a unique identifier, a type, and other description fields.

[0033] That is, for a given metadata structure in the metadata document, the XML language is used for description conversion to obtain a target file expressed in XML language. Exemplarily, the XML language of the asset information metadata document can be expressed as: <!-- Asset information metadata document --> <field> <description>Asset unique identification< / description> < / field> <field> <description>Asset name< / description> < / field> <field> <description>Asset type< / description> < / field> Step 2: Construct a first metadata semantic dictionary based on the target file.

[0034] The first metadata semantic dictionary includes a plurality of third metadata entries, each of which includes a file name, a third metadata item, a semantic vocabulary standard name of the third metadata item, at least one third derivative information corresponding to the third metadata item, and derivative information description corresponding to each of the third derivative information.

[0035] It can be understood that the first metadata semantic dictionary is constructed based on the target file, and the structure of the first metadata semantic dictionary is:

file name, third metadata item, semantic vocabulary standard name corresponding to the third metadata item, first third derivative information, derivative information description corresponding to the first third derivative information, …, n-th third derivative information, derivative information description corresponding to the n-th third derivative information

[0036] Further, in an exemplary embodiment, the obtaining of the first training sample set includes: labeling the training document according to the first metadata semantic dictionary to obtain a plurality of first metadata entries corresponding to the training document; and constructing the first training sample set according to the plurality of first metadata entries.

[0037] It can be understood that after determining the first metadata semantic dictionary, the training document can be labeled according to the first metadata semantic dictionary to obtain a plurality of first metadata entries corresponding to the training document, and then the first training sample set is constructed according to the plurality of first metadata entries. In an exemplary embodiment, the plurality of first metadata entries can be divided into a first training sample set and a test set, and the test set is used to test the trained target large language model to evaluate the extraction ability of the target large language model.

[0038] In an exemplary embodiment, the correcting the first metadata item by the target large language model to obtain a second metadata item corresponding to the first metadata item can include the following steps: Step 1: based on the first metadata item and the training document, a first instruction is constructed, and the first instruction is input into a target large language model to obtain a first output result output by the target large language model, wherein the first output result includes a first index position, and the first index position is an index position of the first metadata item determined by the target large language model from the training document.

[0039] Step 2: correcting a second index position based on the first output result, wherein the second index position is an index position corresponding to the first metadata item in the first training sample set.

[0040] Wherein, the correcting the second index position based on the first output result can include: in the case that the first index position and the second index position are inconsistent, the correctness of the first index position is judged, and the second index position is corrected based on the first correctness judgment result. Wherein, the correcting the second index position based on the first correctness judgment result can include: in the case that the first correctness judgment result is correct, the second index position is changed to the first index position; in the case that the first correctness judgment result is incorrect, the second index position remains unchanged, and feedback is given to the target large language model to further improve the retrieval and generation process, so that the target large language model can provide better answers in subsequent dialogues or tasks.

[0041] Step 3: according to the corrected first metadata item, a second instruction is constructed, and the second instruction is input into the target large language model to obtain a second output result output by the target large language model, wherein the second output result includes at least one second derivative information and derivative information description corresponding to each second derivative information, and the second derivative information is generated by the target large language model according to the first metadata item.

[0042] Step 4: correcting the at least one first derivative information based on the second output result.

[0043] Wherein, the correcting the at least one first derivative information based on the second output result can include: for each second derivative information: In a case where the derivative information description corresponding to at least one of the first derivative information matches the derivative information description corresponding to the second derivative information, the fifth derivative information corresponding to the target derivative information description is corrected according to the second derivative information, where the target derivative information description is the derivative information description corresponding to the matched first derivative information.

[0044] In another example embodiment, the correction of the fifth derivative information according to the second derivative information can include, for each fifth derivative information, in a case where the second derivative information and the fifth derivative information are inconsistent, respectively judging the correctness of the second derivative information and the fifth derivative information, and correcting the fifth derivative information based on the second correctness judgment result.

[0045] In yet another example embodiment, the correction of the fifth derivative information based on the second correctness judgment result can include at least one of the following: (1) In a case where the second correctness judgment result corresponding to the second derivative information is correct and the second correctness judgment result corresponding to the fifth derivative information is correct, the second derivative information and the derivative information description corresponding to the second derivative information are added in the first metadata entry; (2) In a case where the second correctness judgment result corresponding to the second derivative information is correct and the second correctness judgment result corresponding to the fifth derivative information is incorrect, the fifth derivative information is changed to the second derivative information; (3) In a case where the second correctness judgment result corresponding to the second derivative information is incorrect and the second correctness judgment result corresponding to the fifth derivative information is correct, the fifth derivative information remains unchanged, and feedback is provided to the target large language model to further improve the retrieval and generation process, so that the target large language model can provide better answers in subsequent dialogues or tasks.

[0046] In a case where the derivative information description corresponding to at least one of the first derivative information does not match the derivative information description corresponding to the second derivative information, the correctness of the second derivative information is judged. In a case where the third correctness judgment result is correct, the second derivative information and the derivative information description corresponding to the second derivative information are added in the first metadata entry. In a case where the third correctness judgment result is incorrect, feedback is provided to the target large language model to further improve the retrieval and generation process, so that the target large language model can provide better answers in subsequent dialogues or tasks.

[0047] In the embodiments of the present application, two types of instructions are provided, the first instruction is used to correct the index position of the first metadata item in the training document, and the second instruction is used to correct the first derivative information corresponding to the first metadata item. For example, assuming that the first instruction is: please extract the "asset information" in the training document and its position in the document? Then the first output result of the target large language model is judged and corrected, and the first metadata item in the first training sample set is corrected and adjusted. On this basis, the second instruction is designed to inquire about the derivative information of the first metadata item, such as: what is the English name and abbreviation of "computer equipment" in the document? Similarly, the second output result of the target large language model is judged and corrected, and the first metadata item in the first training sample set is corrected and adjusted.

[0048] In an exemplary embodiment, the method further comprises: constructing a second metadata semantic dictionary based on the second training sample set; and using the second metadata semantic dictionary as the external knowledge base to train the target large language model.

[0049] It can be understood that in the subsequent fine-tuning process of the target large language model, if the training document is lacking, the determined second training sample set can be used as an external knowledge base to apply to the target large language model to retrieve metadata information, improve the understanding ability of semantic matching, and solve the conversion problem of the semantic vocabulary standard name of the metadata item.

[0050] Figure 2 A flowchart of a metadata extraction method provided by an embodiment of the present application is shown, which can include the following steps: S210: obtaining a target document.

[0051] S220: inputting the target document into a target large language model to obtain at least one target metadata item output by the target large language model.

[0052] Each of the target metadata items includes a document name of the training document, a target metadata item, a semantic vocabulary standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is trained based on Figure 1 the model training method shown.

[0053] S230: outputting the at least one target metadata item.

[0054] In this embodiment, by acquiring a target document, inputting the target document into a target large language model, obtaining at least one target metadata item output by the target large language model, wherein each target metadata item includes a document name of the training document, a target metadata item, a semantic vocabulary standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, the target large language model is trained based on the model training method shown in Figure 1 The at least one target metadata item is finally output, realizing automatic, efficient and accurate extraction of metadata items in the document, and being capable of generating corresponding derivative information and converting the extracted metadata items into their corresponding semantic vocabulary standard names, thereby meeting various needs of users.

[0055] Figure 3 The embodiment of the present application is based on the above Figure 1 and Figure 2 The metadata extraction method provided by the embodiment shown in the above can include the following steps: S310: converting a metadata document into an XML file expressed in XML language.

[0056] Exemplarily, the metadata document can be a data system standard corresponding to the government field, and then the XML language is used for description conversion according to the metadata structure given in the data system standard to generate XML expression.

[0057] S320: constructing and correcting a training sample set for training of the target large language model.

[0058] In this step, the candidate electronic document sample is artificially annotated and the training sample set is constructed according to the metadata items in the XML file. According to the prompt engineering template of the target large language model, a first instruction is designed according to the semantic vocabulary standard name of each metadata item in the metadata semantic dictionary, the first instruction is used to instruct to extract the corresponding metadata item from the target document in the training sample set, such as: please extract the "asset information" in this document and its position in the document; and the answer output by the target large language model is judged and corrected, and the related information in the training sample set is corrected and adjusted; on this basis, a second instruction is designed, the second instruction is used to obtain the derivative information of the metadata item, such as: please provide the English name and abbreviation of the "computer equipment" in this document; and the answer output by the target large language model is artificially judged and corrected, and the related information in the training sample set is corrected and adjusted.

[0059] S330: training the target large language model.

[0060] The training sample set and the corresponding electronic document sample after the above correction and adjustment are used as an external knowledge base, and a retrieval enhancement strategy of a target large language model is used for training, wherein the input of the target large language model is the name and content of the electronic document, the output of the target large language model is the metadata item, the name and the position of the metadata item in the document retrieved from the above document, and the target large language model also outputs the semantic vocabulary standard name corresponding to the metadata item, that is, the target large language model automatically converts the metadata item name into the semantic vocabulary standard name according to the relationship between the semantic vocabulary standard name corresponding to the metadata item in the training sample set and its synonyms.

[0061] S340: Determine whether the test result meets the preset requirement by testing the trained target large language model.

[0062] If the test result meets the preset requirement, go to S350, if the test result does not meet the preset requirement, increase the sample quantity and go to S320.

[0063] S350: Extract the metadata item in the government document by the trained target large language model, and generate and store the metadata directory item.

[0064] In this embodiment, the target large language model is used to extract the metadata item in the government document. Compared with the traditional simple template matching and keyword extraction technology, the metadata item can be automatically extracted according to the context understanding, which is not affected by the position of the metadata item in the document, avoids the limitations of inaccurate word group segmentation and difficult keyword matching, and according to the metadata structure in the data system standard of the government field, two types of instructions are designed to correct the training sample set, which can realize the extraction of various metadata items, can identify the synonyms and convert them into standard vocabulary.

[0065] In addition, the target large language model is used to correct and adjust the training sample set, which can balance the calculation efficiency and recognition accuracy, is suitable for metadata extraction in government documents, and has strong generalization ability.

[0066] Figure 4 An embodiment of the present application shows the structure of the model training device provided by the embodiment of the present application. As shown in the figure, the model training device 400 can include an acquisition module 410, a correction module 420 and a training module 430. Figure 4

[0067] ​In this embodiment, the obtaining module 410 is configured to obtain a first training sample set, where the first training sample set includes a plurality of first metadata items corresponding to a training document, each of the first metadata items includes a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each of the first derivative information; the correcting module 420 is configured to correct, for each of the first metadata items, the first metadata item by using a target large language model to obtain a second metadata item corresponding to the first metadata item; and the training module 430 is configured to use a second training sample set as an external knowledge base, and train the target large language model based on the training document by using a retrieval enhancement strategy, where the second training sample set includes each of the second metadata items.

[0068] In an example embodiment, the method further includes constructing a module configured to: convert the metadata document into a target file expressed in an XML language; and construct a first metadata semantic dictionary based on the target file, where the first metadata semantic dictionary includes a plurality of third metadata items, each of the third metadata items includes a file name, a third metadata item, a semantic vocabulary standard name of the third metadata item, at least one third derivative information corresponding to the third metadata item, and derivative information description corresponding to each of the third derivative information.

[0069] In an example embodiment, the obtaining module 410 is specifically configured to: annotate the training document according to the first metadata semantic dictionary to obtain a plurality of the first metadata items corresponding to the training document; and construct the first training sample set according to the plurality of the first metadata items.

[0070] In an exemplary embodiment, the correction module 420 is specifically configured to: construct a first instruction based on the first metadata entry and the training document, and input the first instruction into a target large language model to obtain a first output result output by the target large language model, wherein the first output result includes a first index position, the first index position being the index position of the first metadata item determined by the target large language model from the training document; correct a second index position based on the first output result, wherein the second index position is the index position corresponding to the first metadata entry in the first training sample set; construct a second instruction according to the corrected first metadata entry, and input the second instruction into the target large language model to obtain a second output result output by the target large language model, wherein the second output result includes at least one second derived information and a description of derived information corresponding to each second derived information, the second derived information being generated by the target large language model based on the first metadata item; and correct the at least one first derived information based on the second output result.

[0071] In an exemplary embodiment, the training module 430 is specifically configured to: construct a third instruction based on the training document and the external knowledge base, and input the third instruction into the target large language model to obtain a third output result of the target large language model based on the retrieval enhancement strategy. The third output result includes multiple fourth metadata entries corresponding to the training document. Each fourth metadata entry includes the document name of the training document, a fourth metadata item, a semantic lexical standard name corresponding to the fourth metadata item, the index position of the fourth metadata item in the training document, at least one fourth derived information corresponding to the fourth metadata item, and a description of the derived information corresponding to each fourth derived information. The target large language model is optimized by evaluating the third output result.

[0072] In an exemplary embodiment, the training module 430 is further configured to: construct a second metadata semantic dictionary based on the second training sample set; and use the second metadata semantic dictionary as the external knowledge base to train the target large language model.

[0073] The training device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiments shown will not be described again here to avoid repetition.

[0074] Figure 5 This illustration shows a schematic diagram of the structure of a metadata extraction device provided in one embodiment of this application, as shown below. Figure 5 As shown, the metadata extraction device 500 may include: an acquisition module 510, a processing module 520, and an output module 530.

[0075] In this embodiment, the acquisition module 510 is configured to acquire a target document; the processing module 520 is configured to input the target document into a target large language model to obtain at least one target metadata item output by the target large language model, wherein each target metadata item includes a document name of the training document, a target metadata item, a semantic lexical standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is obtained by training the method shown in the embodiment. Figure 1 The output module 530 is configured to output the at least one target metadata item.

[0076] The metadata extraction device provided in the embodiment of the present application can realize Figure 2 The processes realized in the method embodiment shown are not repeated here to avoid repetition.

[0077] The model training device and the metadata extraction device in the embodiment of the present application can be a device, a component in an electronic device, an integrated circuit, or a chip. The embodiment of the present application is not specifically limited.

[0078] The model training device and the metadata extraction device in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems, and the embodiment of the present application is not specifically limited.

[0079] Optionally, as Figure 6 The embodiment of the present application also provides an electronic device 600, which includes a processor 610, a memory 620, and a program or instruction stored in the memory 620 and executable on the processor 610. When the processor 610 executes the program or instruction, the processes of the above model training method embodiments or the processes of the above metadata extraction method embodiments are realized, and the same technical effects are achieved. To avoid repetition, the processes are not repeated here.

[0080] The embodiment of the present application also provides a computer readable storage medium, which stores a program or instruction. When the processor executes the program or instruction, the processes of the above model training method embodiments or the processes of the above metadata extraction method embodiments are realized, and the same technical effects are achieved. To avoid repetition, the processes are not repeated here.

[0081] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0082] The chip provided in the embodiments of the present application includes a processor and a communication interface, the communication interface is coupled with the processor, the processor is used to run programs or instructions to realize the processes of the above model training method embodiments or realize the processes of the above metadata extraction method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0083] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0084] The embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer executes the processes of the above model training method embodiments or the processes of the above metadata extraction method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0085] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing functions as shown or discussed, but can also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0086] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, also can be through hardware, but many cases the former is the better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the contribution to the prior art can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), including a number of instructions to make a terminal (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0087] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative, but not limited, those skilled in the art can make many forms without departing from the purpose of the present application and the scope of the claims under the inspiration of the present application, all belong to the protection of the present application.

Claims

1. A model training method, characterized in that, The method comprises: obtaining a first training sample set, wherein the first training sample set comprises a plurality of first metadata items corresponding to a training document, each of the first metadata items comprising a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each of the first derivative information; for each of the first metadata items, correcting the first metadata item by a target large language model to obtain a second metadata item corresponding to the first metadata item; using a second training sample set as an external knowledge base and training the target large language model based on the training document by using a retrieval enhancement strategy, wherein the second training sample set comprises each of the second metadata items.

2. The method of claim 1, wherein, Before the obtaining the first training sample set, the method further comprises: converting a metadata document into a target file expressed in XML language; constructing a first metadata semantic dictionary based on the target file, wherein the first metadata semantic dictionary comprises a plurality of third metadata items, each of the third metadata items comprising a file name, a third metadata item, a semantic vocabulary standard name of the third metadata item, at least one third derivative information corresponding to the third metadata item, and derivative information description corresponding to each of the third derivative information.

3. The method of claim 2, wherein, The obtaining the first training sample set comprises: annotating the training document according to the first metadata semantic dictionary to obtain a plurality of first metadata items corresponding to the training document; constructing the first training sample set according to the plurality of first metadata items.

4. The method of claim 2, wherein, The correcting the first metadata item by the target large language model to obtain the second metadata item corresponding to the first metadata item comprises: constructing a first instruction based on the first metadata item and the training document, and inputting the first instruction into the target large language model to obtain a first output result output by the target large language model, wherein the first output result comprises a first index position, and the first index position is an index position of the first metadata item determined by the target large language model from the training document; correcting a second index position based on the first output result, wherein the second index position is an index position corresponding to the first metadata item in the first training sample set; constructing a second instruction according to the corrected first metadata item, and inputting the second instruction into the target large language model to obtain a second output result output by the target large language model, wherein the second output result comprises at least one second derivative information and derivative information description corresponding to each of the second derivative information, and the second derivative information is generated by the target large language model according to the first metadata item; correcting the at least one first derivative information based on the second output result.

5. The method of claim 1, wherein, The method comprises the following steps: According to the training document and the external knowledge base, a third instruction is constructed, and the third instruction is input into the target large language model to obtain a third output result output by the target large language model based on the retrieval enhancement strategy, wherein the third output result comprises a plurality of fourth metadata entries corresponding to the training document, and each fourth metadata entry comprises a document name of the training document, a fourth metadata item, a semantic vocabulary standard name corresponding to the fourth metadata item, an index position of the fourth metadata item in the training document, at least one fourth derivative information corresponding to the fourth metadata item, and derivative information description corresponding to each fourth derivative information. The target large language model is optimized by evaluating the third output result.

6. The method of claim 1, wherein, The method further comprises the following steps: A second metadata semantic dictionary is constructed based on the second training sample set. The second metadata semantic dictionary is used as the external knowledge base to train the target large language model.

7. A metadata extraction method characterized by, The method comprises the following steps: A target document is obtained. At least one target metadata entry output by the target large language model is obtained by inputting the target document into the target large language model, wherein each target metadata entry comprises a document name of the training document, a target metadata item, a semantic vocabulary standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is trained based on the model training method of any one of claims 1-6. The at least one target metadata entry is output.

8. A model training apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a first training sample set, wherein the first training sample set comprises a plurality of first metadata entries corresponding to a training document, and each first metadata entry comprises a document name of the training document, a first metadata item, a semantic vocabulary standard name corresponding to the first metadata item, an index position of the first metadata item in the training document, at least one first derivative information corresponding to the first metadata item, and derivative information description corresponding to each first derivative information. A correction module is configured to correct, for each first metadata entry, the first metadata entry by using a target large language model to obtain a second metadata entry corresponding to the first metadata entry. A training module is configured to train the target large language model by using a second training sample set as an external knowledge base and based on the training document and a retrieval enhancement strategy, wherein the second training sample set comprises each second metadata entry.

9. A metadata extraction apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a target document. The processing module is configured to input the target document into a target large language model to obtain at least one target metadata item output by the target large language model, wherein each target metadata item comprises a document name of the training document, a target metadata item, a semantic lexical standard name of the target metadata item, an index position of the target metadata item in the training document, and derivative information corresponding to the target metadata item, and the target large language model is trained based on the model training method in any one of claims 1-6. The output module is configured to output the at least one target metadata item.

10. An electronic device, comprising: The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to perform the method in any one of claims 1-7. The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to perform the method in any one of claims 1-7.

11. A computer readable storage medium, characterized in that, The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to perform the method in any one of claims 1-7.

12. A computer program product, characterised in that, ​

Images