Technical parameter labeling system and method based on large language model and reflection mechanism

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a technical parameter labeling system and method based on a large language model and a reflection mechanism, and the method comprises the steps: obtaining a to-be-labeled technical parameter document, analyzing a document needing auxiliary verification in the technical parameter document, and obtaining a to-be-assisted document; external knowledge related to the to-be-assisted document is retrieved, an auxiliary knowledge point set is obtained, the semantic similarity between the auxiliary knowledge point set and the to-be-assisted document is calculated, and a target knowledge point set related to the to-be-assisted document is screened out; performing reflection processing on the target knowledge point set and a document parameter corresponding to the to-be-assisted document to obtain an reflection value, and adjusting the parameter confidence of the document parameter based on the reflection value to obtain a confidence adjustment parameter; and performing parameter annotation on the technical parameter document by using the confidence adjustment parameter to obtain a target annotation parameter. The technical parameter labeling precision can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a technical parameter labeling system and method based on a large language model and a reflection mechanism. BACKGROUND

[0002] Under the background of the rapid development of current artificial intelligence technology, accurate labeling of technical parameters has become a crucial basic link in the process of model training and optimization. Technical parameters cover a wide range, including model structure setting, hyperparameter configuration, training strategy, and evaluation indicators, and the labeling quality directly affects the final performance and reliability of the model.

[0003] Currently, for technical parameter labeling, a traditional rule engine is usually used, which first sets parameter identification and matching rules, and then completes parameter labeling through fixed logic. Such a method is relatively fixed and cannot identify and correct problems such as parameter confusion and numerical deviation that occur during labeling, making it lack a deviation correction link for parameter labeling in high professional scenarios, resulting in labeling errors and low technical parameter labeling accuracy. SUMMARY

[0004] The present application provides a technical parameter labeling system and method based on a large language model and a reflection mechanism, which aims to improve the accuracy of technical parameter labeling.

[0005] To achieve the above purpose, the technical parameter labeling system based on a large language model and a reflection mechanism provided by the present application comprises an auxiliary document identification module, an auxiliary knowledge screening module, a parameter confidence adjustment module, and a parameter labeling module. The auxiliary document identification module is used to obtain a technical parameter document to be labeled, identify document parameters of the technical parameter document using a large language model, analyze documents that need to be assisted in verification in the technical parameter document based on the document parameters, and obtain a document to be assisted. The auxiliary knowledge screening module is used to retrieve external knowledge related to the document to be assisted, obtain a set of auxiliary knowledge points, calculate the semantic similarity between each knowledge point in the set of auxiliary knowledge points and the corresponding text content of the document to be assisted, and screen a set of target knowledge points about the document to be assisted from the set of auxiliary knowledge points based on the semantic similarity. The parameter confidence adjustment module is used to perform reflection processing on the set of target knowledge points and the document to be assisted, obtain a reflection value, adjust the parameter confidence of the document to be assisted based on the reflection value, and obtain a confidence adjustment parameter. The parameter labeling module is used to label the technical parameter document using the confidence adjustment parameter to obtain target labeling parameters.

[0006] Optionally, the target knowledge point set is subjected to reflection processing with the to-be-assisted document to obtain a reflection value, including: identifying semantic matching degree and parameter expression matching degree of the target knowledge point set and the to-be-assisted document; analyzing relevance of the target knowledge point set and the to-be-assisted document based on the semantic matching degree and the parameter expression matching degree; evaluating support of the target knowledge point set to the to-be-assisted document by using the relevance; analyzing authority level, timeliness and industry recognition of the target knowledge point set; analyzing availability of the target knowledge point set in document revision of the to-be-assisted document according to the authority level, the timeliness and the industry recognition; calculating processing score of the target knowledge point set in reflection processing with the to-be-assisted document based on the relevance, the support and the availability, to obtain a reflection value.

[0007] Optionally, a three-dimensional scoring space of the target knowledge point set and the to-be-assisted document is constructed by using the relevance, the support and the availability; calculating scores corresponding to the relevance, the support and the availability by using the three-dimensional scoring space, to obtain a relevance score, a support score and an availability score; performing nonlinear transformation on the relevance score, the support score and the availability score to obtain a relevance enhanced score, a support enhanced score and an availability enhanced score; calculating a synergistic effect factor of the relevance enhanced score, the support enhanced score and the availability enhanced score; calculating processing score of the target knowledge point set in reflection processing with the to-be-assisted document based on the synergistic effect factor, to obtain a reflection value.

[0008] Optionally, external knowledge related to the to-be-assisted document is retrieved to obtain an auxiliary knowledge point set, including: identifying business category and technical features of the to-be-assisted document; determining retrieval range and retrieval priority of external knowledge retrieval corresponding to the to-be-assisted document based on the business category and the technical features; constructing a retrieval link based on the retrieval range and the retrieval priority; retrieving external knowledge related to the to-be-assisted document according to the retrieval link to obtain an auxiliary knowledge point set.

[0009] Optionally, the semantic similarity of each knowledge point in the auxiliary knowledge point set and the text content corresponding to the document to be assisted is calculated, comprising: The auxiliary knowledge point set and the document to be assisted are vector converted to obtain a first numerical vector and a second numerical vector; The cosine similarity of the first numerical vector and the second numerical vector is calculated. The Jaccard similarity between the knowledge elements in the auxiliary knowledge point set and the document segments in the document to be assisted is calculated. The cosine similarity and the Jaccard similarity are linearly weighted and fused to obtain the semantic similarity.

[0010] Optionally, the target labeled parameter is obtained by using the confidence adjustment parameter to perform parameter labeling on the technical parameter document, comprising: Based on the parameter confidence in the confidence adjustment parameter, the technical parameter document is prioritized to generate a parameter labeling sequence. Based on the parameter labeling sequence, each parameter is sequentially labeled to obtain an initial labeled parameter set, and the labeling operation includes parameter name labeling, parameter value labeling, and parameter confidence labeling. The initial labeling result is logically consistent, and the initial labeled parameter set is corrected based on the verification result of the logical consistency verification to obtain the target labeled parameter.

[0011] Optionally, the document parameters of the technical parameter document are identified using a large language model, comprising: The technical parameter document is semantically analyzed using a large language model to identify parameter description segments in the technical parameter document. The parameter name and parameter value in the parameter description segment are identified. The parameter name and the parameter value are mapped to a predefined parameter category to obtain a document parameter.

[0012] Optionally, based on the document parameter, the document in the technical parameter document that needs to be assisted is analyzed to obtain a document to be assisted, comprising: The initial confidence of each parameter in the document parameter is calculated. Based on the initial confidence, the verified parameters are selected from the document parameters. The text segment corresponding to the verified parameter in the technical parameter document is located to obtain the document to be assisted.

[0013] Optionally, based on the reflection value, the parameter confidence of the document to be assisted is adjusted to obtain a confidence adjustment parameter, comprising: constructing a segmented mapping function of the reflection value and a parameter confidence degree corresponding to the document to be assisted; mapping the reflection value to a confidence degree adjustment value by using the segmented mapping function; adjusting an initial confidence degree of the document to be assisted by using the confidence degree adjustment value to obtain a confidence adjustment parameter.

[0014] The technical parameter labeling method based on a large language model and a reflection mechanism is characterized in that the method comprises: obtaining a technical parameter document to be labeled, identifying a document parameter of the technical parameter document by using a large language model, analyzing a document needing to be assisted for verification in the technical parameter document based on the document parameter, and obtaining a document to be assisted; retrieving external knowledge related to the document to be assisted to obtain an auxiliary knowledge point set, calculating a semantic similarity between each knowledge point in the auxiliary knowledge point set and text content corresponding to the document to be assisted, and screening a target knowledge point set about the document to be assisted from the auxiliary knowledge point set based on the semantic similarity; performing reflection processing on the target knowledge point set and the document to be assisted to obtain a reflection value, adjusting a parameter confidence degree of the document to be assisted based on the reflection value, and obtaining a confidence adjustment parameter; labeling a parameter of the technical parameter document by using the confidence adjustment parameter to obtain a target labeling parameter.

[0015] Compared with the prior art, the present application first uses a large language model to identify document parameters in the document to ensure that all subsequent operations have a clear processing object, avoiding process interruption due to missing data sources, then uses a large language model to accurately capture hidden professional parameters in the document, then analyzes the to-be-assisted document to pinpoint parts of the document where parameter expression is ambiguous, the source is unknown, or there is a difference from regular cognition, avoiding resource waste caused by indiscriminate processing of all document content, focusing on core verification objects for subsequent external knowledge retrieval, laying the foundation for accurate processing; then, the present application effectively remedies the problems of incomplete parameter description and missing technical basis in the to-be-assisted document by retrieving external knowledge, then calculates semantic similarity to quantitatively analyze the close degree of knowledge points and to-be-assisted document text, eliminates knowledge points unrelated to verification requirements or with weak relevance, avoids invalid knowledge interference in the subsequent verification process, and ensures the adaptability of the content in the auxiliary knowledge point set to the to-be-assisted document; then, the present application adjusts the parameter confidence of the document parameters through a reflection mechanism, so that the reliability of the document parameters has a clear quantitative basis, avoiding the problem that in traditional labeling, parameter reliability only depends on subjective judgment, and the adjusted confidence adjustment parameter is accompanied by a clear reliability identifier, providing a clear parameter reliability reference for the subsequent labeling link, and ensuring that the labeling result can reflect the true reliability of the parameters; finally, the technical parameter document is labeled using the confidence adjustment parameter, and the target labeling parameter can convert the parameter confidence verified through reflection into a final usable labeling result, thereby outputting structured parameter data with high precision. Therefore, the present application can improve the technical parameter labeling precision. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The functional module diagram of the technical parameter labeling system based on a large language model and a reflection mechanism provided by an embodiment of the present application is shown in the figure. Figure 2 The flowchart of the technical parameter labeling method based on a large language model and a reflection mechanism provided by an embodiment of the present application is shown in the figure. The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0017] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0018] In addition, the step timing in each of the following method embodiments is only an example and is not strictly limited.

[0019] In fact, the server device deployed by the technical parameter labeling system based on a large language model and a reflection mechanism can be composed of one or more devices. The technical parameter labeling system based on a large language model and a reflection mechanism can be implemented as a business instance, a virtual machine, or a hardware device. For example, the technical parameter labeling system based on a large language model and a reflection mechanism can be implemented as a business instance deployed on one or more devices in a cloud node. In short, the technical parameter labeling system based on a large language model and a reflection mechanism can be understood as a software deployed on a cloud node, which provides a technical parameter labeling service based on a large language model and a reflection mechanism for each user terminal. Alternatively, the technical parameter labeling system based on a large language model and a reflection mechanism can also be implemented as a virtual machine deployed on one or more devices in a cloud node. The virtual machine has application software for managing each user terminal. Alternatively, the technical parameter labeling system based on a large language model and a reflection mechanism can also be implemented as a server composed of a plurality of hardware devices of the same or different types, and one or more hardware devices are provided to provide a technical parameter labeling service based on a large language model and a reflection mechanism for each user terminal.

[0020] In terms of implementation, the technical parameter labeling system based on a large language model and a reflection mechanism and the user terminal adapt to each other. That is, the technical parameter labeling system based on a large language model and a reflection mechanism is installed as an application on a cloud service platform, and the user terminal is a client that establishes a communication connection with the application; or the technical parameter labeling system based on a large language model and a reflection mechanism is implemented as a website, and the user terminal is implemented as a webpage; or the technical parameter labeling system based on a large language model and a reflection mechanism is implemented as a cloud service platform, and the user terminal is implemented as an applet in an instant messaging application.

[0021] Referring to Figure 1 FIG. 1 shows a functional module diagram of a technical parameter labeling system based on a large language model and a reflection mechanism according to an embodiment of the present application.

[0022] The technical parameter labeling system based on a large language model and a reflection mechanism 100 can be set in a cloud server, and in terms of implementation, it can be one or more service devices, or it can be installed as an application on a cloud (such as a server for monitoring the enteral nutrition of critically ill patients, a server cluster, etc.), or it can be developed as a website. According to the implemented functions, the technical parameter labeling system based on a large language model and a reflection mechanism 100 includes an auxiliary document recognition module 101, an auxiliary knowledge screening module 102, a parameter confidence adjustment module 103, and a parameter labeling module 104.

[0023] In the embodiment of the present application, based on the technical parameter labeling of the large language model and the reflection mechanism, each of the above modules can be independently implemented and called with other modules. Here, the calling can be understood as that a module can connect multiple modules of another type and provide corresponding services for the connected multiple modules. In the technical parameter labeling system based on the large language model and the reflection mechanism provided by the embodiment of the present application, without modifying the program code, the application range of the technical parameter labeling architecture based on the large language model and the reflection mechanism can be adjusted by increasing the modules and directly calling, realizing cluster horizontal expansion, so as to achieve the purpose of quickly and flexibly expanding the technical parameter labeling system based on the large language model and the reflection mechanism. In practical application, the above modules can be arranged in the same device or different devices, or can be arranged in a virtual device, such as a service instance in a cloud server.

[0024] Next, the various components of the technical parameter labeling system based on the large language model and the reflection mechanism and the specific work flow will be described in conjunction with specific embodiments.

[0025] The auxiliary document identification module 101 is configured to obtain a technical parameter document to be labeled, identify document parameters of the technical parameter document by using a large language model, analyze documents that need to be assisted in verification in the technical parameter document based on the document parameters, and obtain a document to be assisted.

[0026] In the embodiment of the present application, the technical parameter document to be labeled can be obtained, and the processing object required for subsequent labeling operations can be obtained, ensuring that the labeling work has a clear data source.

[0027] The technical parameter document refers to a document containing professional technical parameters in various fields and corresponding explanation information, which can be obtained by calling an enterprise internal production management system, such as calling a device debugging report from a device operation and maintenance management platform of an intelligent manufacturing enterprise.

[0028] In the embodiment of the present application, the document parameters of the technical parameter document can be identified by using a large language model, and the parameter name and value can be identified from the unstructured technical document. For example, in the PCB quotation scene, the material model, size specification, process requirement, and order quantity of the PCB board and other document parameters can be identified, providing a core data basis for technical parameter labeling.

[0029] The large language model refers to a pre-trained language model optimized for technical parameter document processing scenarios and having professional semantic analysis and technical entity recognition capabilities, which can be deployed by using a BERT-Tech model. The document parameters refer to a parameter set of parameter names and parameter value data classes for formal documents.

[0030] As an embodiment of the present application, the document parameters of the technical parameter document are identified by using a large language model, including: The technical parameter document is semantically analyzed by using a large language model to identify parameter description segments in the technical parameter document. The parameter name and parameter value in the parameter description segment are identified. The parameter name and the parameter value are mapped into a predefined parameter category to obtain a document parameter.

[0031] In detail, the technical parameter document is first split into short sentences according to paragraphs, and a large language model optimized for a technical scenario is input. With the technical term understanding ability of the model, the probability that each short sentence belongs to a parameter description segment is output. When the probability reaches a preset threshold, the short sentence is determined to be a parameter description segment. A built-in named entity recognition (NER) module of the large language model is called to identify the parameter name and parameter value in the parameter description segment. A pre-constructed domain-specific parameter category library is called to calculate the similarity of the parameter name and the category core keyword. When the similarity meets a preset threshold, the "parameter name-parameter value" pair is mapped to the corresponding category to form a document parameter.

[0032] The embodiment of the present application can filter out technical parameters with lower credibility or ambiguity by analyzing the documents that need to be assisted in verification in the technical parameter document based on the document parameters to obtain a document to be assisted.

[0033] As an embodiment of the present application, based on the document parameters, the documents that need to be assisted in verification in the technical parameter document are analyzed to obtain a document to be assisted, including: The initial confidence of each parameter in the document parameter is calculated. Based on the initial confidence, the parameters to be verified are filtered out from the document parameters. The text segment corresponding to the parameter to be verified in the technical parameter document is located to obtain a document to be assisted.

[0034] In detail, the reliability of each parameter can be preliminarily quantified by using rule matching and basic semantic analysis technology in combination with the completeness of the source of the document parameter (such as whether the parameter basis is marked) and the clarity of the expression (such as whether there is ambiguous expression) to obtain an initial confidence. A preset initial confidence threshold (the threshold is set according to the reliability requirement of parameter marking in the field, and the specific setting needs to be combined with actual application) is used to compare the initial confidence of each document parameter with the threshold. When the initial confidence of the parameter is lower than the threshold, the parameter is determined to be a parameter to be verified. The text segment corresponding to the parameter to be verified can be determined according to the parameter name, associated expression and other information of the parameter to be verified.

[0035] The auxiliary knowledge screening module 102 is used for retrieving external knowledge related to the document to be assisted, obtaining an auxiliary knowledge point set, calculating semantic similarity of each knowledge point in the auxiliary knowledge point set and corresponding text content of the document to be assisted, and screening a target knowledge point set about the document to be assisted from the auxiliary knowledge point set based on the semantic similarity.

[0036] The embodiment of the application can supplement external reference basis for the document to be assisted by retrieving external knowledge related to the document to be assisted to obtain an auxiliary knowledge point set, and make up for problems such as incomplete parameter description and missing basis of the document to be assisted.

[0037] The auxiliary knowledge point set refers to a knowledge set directly related to verification requirements of the document to be assisted, which is collected by retrieving external knowledge related to the document to be assisted, and contains information such as knowledge text content, knowledge source and knowledge application scenario.

[0038] As an embodiment of the application, retrieving external knowledge related to the document to be assisted to obtain an auxiliary knowledge point set comprises: identifying a business category and a technical feature of the document to be assisted; determining a retrieval range and a retrieval priority of external knowledge retrieval corresponding to the document to be assisted based on the business category and the technical feature; constructing a retrieval link based on the retrieval range and the retrieval priority; retrieving external knowledge related to the document to be assisted according to the retrieval link to obtain an auxiliary knowledge point set.

[0039] The business category refers to a specific business field classification to which the document to be assisted belongs, and the technical feature refers to key information reflecting technical attributes of the document to be assisted, including types of parameters to be verified, technical scenarios of parameter application and types of technical standards associated with parameters.

[0040] In detail, the to-be-assisted document can be input into a text classification model and a technical feature extraction model (convolutional neural network), then the text classification model is used to determine the business category (such as PCB quotation, device debugging) to which the document belongs according to the document content, and then the technical feature extraction model is used to extract the core technical terms in the document as technical features, such as the specific quotation value of the PCB quotation; based on the identified business category, the corresponding external knowledge base (such as a PCB industry standard library, a device operation and maintenance knowledge base) is locked, and then the technical feature correlation degree is set to determine the retrieval priority; the standardized retrieval interface (such as an API interface) of the external knowledge base is called, and the retrieval range, priority and technical features are converted into retrieval parameters recognizable by the interface to generate a retrieval link that can accurately point to the target content; the retrieval request is sent to the external knowledge base through the retrieval link, after receiving the returned knowledge data, the data deduplication algorithm is used to eliminate duplicate content, and the results containing complete knowledge content and source information are screened out to obtain the set of auxiliary knowledge points.

[0041] Further, by calculating the semantic similarity of each knowledge point in the set of auxiliary knowledge points and the text content corresponding to the to-be-assisted document, the content closely related to the to-be-assisted document can be accurately distinguished from the set of auxiliary knowledge points, and the knowledge points with weak or irrelevant association can be eliminated, thereby avoiding invalid knowledge interference in subsequent verification and ensuring the pertinence of the knowledge points used for verification.

[0042] As an embodiment of the present application, the semantic similarity of each knowledge point in the set of auxiliary knowledge points and the text content corresponding to the to-be-assisted document is calculated, comprising: vector conversion is performed on the set of auxiliary knowledge points and the to-be-assisted document to obtain a first numerical vector and a second numerical vector; cosine similarity of the first numerical vector and the second numerical vector is calculated; Jaccard similarity between the knowledge elements in the set of auxiliary knowledge points and the document segments of the to-be-assisted document is calculated; the cosine similarity and the Jaccard similarity are linearly weighted and fused to obtain the semantic similarity.

[0043] In detail, the semantic similarity of each knowledge point in the auxiliary knowledge point set can be compared with a preset threshold, and the knowledge points with a semantic similarity not less than the preset threshold are retained, that is, the target knowledge point set is obtained, and the preset threshold can be set to 0.8, or can be set according to actual application.

[0044] Further, the embodiment of the present application can accurately filter out the core knowledge points directly related to the to-be-verified parameter from a large amount of knowledge by screening the target knowledge point set about the to-be-assisted document from the auxiliary knowledge point set based on the semantic similarity, and then construct a high-quality knowledge subset.

[0045] In detail, the semantic similarity of each knowledge point in the auxiliary knowledge point set can be compared with a preset threshold, and the knowledge points with a semantic similarity not less than the preset threshold are retained, that is, the target knowledge point set is obtained, and the preset threshold can be set to 0.8, or can be set according to actual application.

[0046] The parameter confidence adjustment module 103 is configured to perform reflection processing on the target knowledge point set and the to-be-assisted document to obtain a reflection value, adjust the parameter confidence of the to-be-assisted document based on the reflection value, and obtain a confidence adjustment parameter.

[0047] The embodiment of the present application can compare the content consistency of the to-be-verified parameter and the authoritative knowledge point by performing reflection processing on the target knowledge point set and the to-be-assisted document to obtain a reflection value, and convert subjective technical judgment into objective numerical basis, thereby providing a measurable standard for parameter reliability.

[0048] The reflection value refers to a comprehensive matching degree of the target knowledge point set and the to-be-assisted document in terms of semantic association, actual support capability and applicability reliability.

[0049] As an embodiment of the present application, the reflection processing of the target knowledge point set and the to-be-assisted document to obtain a reflection value includes: identifying a semantic matching degree and a parameter expression matching degree between the target knowledge point set and the document to be assisted; analyzing a relevance between the target knowledge point set and the document to be assisted based on the semantic matching degree and the parameter expression matching degree; evaluating a support of the target knowledge point set to the document to be assisted using the relevance; analyzing an authoritative level, a timeliness, and an industry recognition of the target knowledge point set; analyzing an availability of the target knowledge point set in document revision of the document to be assisted according to the authoritative level, the timeliness, and the industry recognition; calculating a processing score of the target knowledge point set in reflection processing of the document to be assisted based on the relevance, the support, and the availability, and obtaining a reflection value.

[0050] The relevance refers to a content level fit degree between the target knowledge point set and the document to be assisted, which is a basis for judging whether the knowledge point can be matched with the document to be assisted in a targeted manner. The support refers to an actual support capability of the target knowledge point set in verifying parameter rationality of the document to be assisted and answering parameter questions in the document. The availability refers to a degree of applicability and reliability of the knowledge point in revising the document to be assisted and verifying parameters.

[0051] In detail, in analyzing the relevance, a joint decision rule can be used to comprehensively analyze results of both. If the semantic matching degree and the parameter expression matching degree are both higher than respective preset thresholds, the relevance is determined to be “high”. If only one of them meets the requirement, the relevance is “medium”. If neither of them meets the requirement, the relevance is “low”. A corresponding rule of the relevance level and the support can be set. If the relevance is “high” and a preset high-relevance duration threshold (for example, a core parameter matching ratio is more than 80%) is met, the support is evaluated to be “strong support”. If the relevance is “medium”, the support is “medium support”. If the relevance is “low”, the support is “weak support”. The authoritative level can be determined according to a knowledge point source type (for example, an industry standard library source is “high authority” and a common manufacturer document is “low authority”). The timeliness can be determined by comparing a knowledge point publishing time and a current time. If the time is within a preset time threshold (for example, one year), the timeliness is “high”. The industry recognition is determined by searching a frequency of knowledge point citation in field literature. If the frequency is higher than a citation threshold, the industry recognition is “high recognition”.

[0052] Preferably, the calculating the processing score of the target knowledge point set in reflection processing of the document to be assisted based on the relevance, the support, and the availability, and obtaining the reflection value includes: constructing a three-dimensional score space of the target knowledge point set and the document to be assisted using the relevance, the support, and the availability; Utilize the three-dimensional score space, the corresponding score of the correlation, the support and the availability is calculated, the correlation score, the support score and the availability score are obtained; The correlation score, the support score and the availability score are nonlinearly transformed, and the correlation enhancement score, the support enhancement score and the availability enhancement score are obtained; The synergistic effect factor of the correlation enhancement score, the support enhancement score and the availability enhancement score is calculated; Based on the synergistic effect factor, the processing score of the target knowledge point set and the to-be-assisted document when reflecting processing is calculated, and the reflection value is obtained.

[0053] The three-dimensional score space refers to a three-dimensional coordinate system constructed by correlation, support and availability as three mutually perpendicular dimensions, each dimension corresponds to a preset score interval, is used for positioning the specific coordinate point of the correlation score, the support score and the availability score in the space, and directly reflects the comprehensive level of the three indexes. The synergistic effect factor refers to a parameter for quantifying the mutual influence and synergistic effect degree of the correlation enhancement score, the support enhancement score and the availability enhancement score. In detail, the three indexes can be mapped into the value interval of the corresponding coordinate axis by using coordinate mapping technology, so that the matching condition of each target knowledge point and the to-be-assisted document can correspond to a coordinate point in the space, forming a three-dimensional score space; the domain index quantization rule is called, the correlation score is calculated by extracting the correlation feature through a semantic analysis tool (such as technical term coincidence degree), the support degree of the knowledge point to the document is analyzed by means of a support strength evaluation model (such as parameter value matching degree) to obtain the support score, and the availability score is obtained by using an availability rating tool in combination with the knowledge attribute; a Sigmoid nonlinear transformation function is used to transform the three scores, and a transformation threshold can be set to filter extreme abnormal values, so that the enhanced scores after transformation are in a reasonable and effective interval.

[0054] Further, as another embodiment of the application, the synergistic effect factor calculation formula is as follows:

[0055] Among them, The synergistic effect factor is represented by, The correlation enhancement score is represented by, The support enhancement score is represented by, The availability enhancement score is represented by, The minimum value function is represented by, The minimum value function is represented by; Further, as another embodiment of the application, the reflection value calculation formula is as follows;

[0056] wherein, represents a reflection value, represents a correlation enhancement score, represents a support enhancement score, represents an availability enhancement score, represents a sensitivity parameter of the reflection value, represents a sensitivity parameter of the reflection value, represents a sensitivity parameter of the reflection value, represents a synergistic effect weight coefficient, represents a synergistic effect factor.

[0057] Further, the embodiment of the present application can convert the quantitative evaluation value generated by the reflection process into the actual adjustment amount of the parameter reliability by adjusting the parameter reliability of the document to be assisted based on the reflection value to obtain the confidence adjustment parameter, so that the parameter reliability can be adjusted in real time according to the pros and cons of the verification result.

[0058] As an embodiment of the present application, adjusting the parameter reliability of the document to be assisted based on the reflection value to obtain the confidence adjustment parameter comprises: constructing a segmented mapping function of the reflection value and the parameter reliability corresponding to the document to be assisted; mapping the reflection value into a reliability adjustment value by using the segmented mapping function; adjusting the initial reliability of the document to be assisted by using the reliability adjustment value to obtain the confidence adjustment parameter.

[0059] The segmented mapping function refers to a function constructed for establishing the correlation between the reflection value and the parameter reliability corresponding to the document to be assisted, and the core feature thereof is to divide the reflection value into multiple continuous intervals and preset the corresponding reliability adjustment rule for each interval.

[0060] In detail, the segmented intervals of the reflection value can be divided in combination with the historical annotation data of the field to which the document to be assisted belongs (for example, the intervals are divided according to the matching rationality reflected by the reflection value), and then the reliability adjustment rule corresponding to each interval is defined (for example, the high reflection value interval corresponds to positive adjustment, and the low reflection value interval corresponds to negative adjustment), and the function modeling tool is used to solidify the “reflection value interval-reliability adjustment rule” into a segmented mapping function; the reflection value is substituted into the constructed segmented mapping function, the function first judges the segmented interval to which the reflection value belongs, and then calculates and outputs the corresponding reliability adjustment value according to the adjustment rule corresponding to the interval.

[0061] The parameter annotation module 104 is used to annotate the technical parameter document using the confidence adjustment parameters to obtain the target annotation parameters.

[0062] This invention utilizes the confidence adjustment parameters to annotate the technical parameter document, obtaining target annotation parameters. This allows the reflected and verified parameter confidence to be transformed into a final usable annotation result, thereby outputting highly accurate structured parameter data.

[0063] As an embodiment of the present invention, the confidence adjustment parameters are used to annotate the technical parameter document to obtain target annotation parameters, including: Based on the confidence level of the parameters in the confidence adjustment parameters, the technical parameter documents are prioritized and sorted to generate a parameter annotation sequence; Based on the parameter annotation sequence, annotation operations are performed on each parameter sequentially to obtain an initial set of annotated parameters. The annotation operations include parameter name annotation, parameter value annotation, and parameter confidence level annotation. The initial annotation results are subjected to logical consistency verification, and the parameter annotation content of the initial annotation parameter set is corrected based on the verification results to obtain the target annotation parameters.

[0064] In detail, the annotation order can be determined from high to low confidence level. Based on the generated parameter annotation sequence, the document annotation tool is called to locate the corresponding parameters in the technical parameter document in turn, and the parameter name, parameter value and corresponding parameter confidence level are annotated respectively. After associating and matching the three, they are integrated into structured data to obtain the initial annotation parameter set. The initial annotation parameter set is verified by a logical verification algorithm to check whether there are logical contradictions between the parameters (such as mismatch between specification parameters and performance parameters). If contradictions exist, the generation process of confidence adjustment parameters is backtracked to check and the contradictory annotation content is corrected. After the verification is passed or the correction is completed, the target annotation parameters that meet the logical consistency requirements are obtained.

[0065] like Figure 2 The diagram shown is a flowchart illustrating a technical parameter annotation method based on a large language model and a reflection mechanism according to an embodiment of the present invention. In this embodiment, the technical parameter annotation method based on a large language model and a reflection mechanism includes: Obtain the technical parameter document to be labeled, use a large language model to identify the document parameters of the technical parameter document, and based on the document parameters, analyze the documents in the technical parameter document that need to be assisted in verification to obtain the document to be assisted. search external knowledge related to the document to be assisted, obtain an auxiliary knowledge point set, calculate semantic similarity between each knowledge point in the auxiliary knowledge point set and text content corresponding to the document to be assisted, and select a target knowledge point set about the document to be assisted from the auxiliary knowledge point set based on the semantic similarity; perform reflection processing on the target knowledge point set and the document to be assisted, obtain a reflection value, adjust a parameter confidence of the document to be assisted based on the reflection value, and obtain a confidence adjustment parameter; perform parameter labeling on the technical parameter document by using the confidence adjustment parameter, and obtain a target labeled parameter.

[0066] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0067] Finally, it should be noted that in the above embodiments, each embodiment can be combined or independent, and deleting any individual does not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present application and are not limited. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A technical parameter labeling system based on a large language model and a reflection mechanism, characterized in that, The system comprises an auxiliary document identification module, an auxiliary knowledge screening module, a parameter confidence adjustment module, and a parameter labeling module; The auxiliary document identification module is configured to obtain a technical parameter document to be labeled, identify document parameters of the technical parameter document by using a large language model, analyze a document that needs to be verified in the technical parameter document based on the document parameters, and obtain a document to be assisted; The auxiliary knowledge screening module is configured to search external knowledge related to the document to be assisted, obtain an auxiliary knowledge point set, calculate semantic similarity between each knowledge point in the auxiliary knowledge point set and corresponding text content of the document to be assisted, and screen a target knowledge point set about the document to be assisted from the auxiliary knowledge point set based on the semantic similarity; The parameter confidence adjustment module is configured to perform reflection processing on the target knowledge point set and the document to be assisted, obtain a reflection value, adjust a parameter confidence of the document to be assisted based on the reflection value, and obtain a confidence adjustment parameter; The parameter labeling module is configured to perform parameter labeling on the technical parameter document by using the confidence adjustment parameter, and obtain target labeling parameters.

2. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, The reflection value is obtained by performing reflection processing on the target knowledge point set and the document to be assisted, and comprises: identifying semantic matching degree and parameter expression matching degree of the target knowledge point set and the document to be assisted; analyzing relevance of the target knowledge point set and the document to be assisted based on the semantic matching degree and the parameter expression matching degree; evaluating support of the target knowledge point set for the document to be assisted by using the relevance; analyzing authority level, timeliness, and industry recognition of the target knowledge point set; analyzing availability of the target knowledge point set in document correction of the document to be assisted according to the authority level, the timeliness, and the industry recognition; calculating a processing score of the reflection processing on the target knowledge point set and the document to be assisted based on the relevance, the support, and the availability, and obtaining a reflection value.

3. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 2, wherein, The reflection value is obtained by performing reflection processing on the target knowledge point set and the document to be assisted based on the relevance, the support, and the availability, and comprises: constructing a three-dimensional scoring space of the target knowledge point set and the document to be assisted by using the relevance, the support, and the availability; calculating scores corresponding to the relevance, the support, and the availability by using the three-dimensional scoring space, and obtaining a relevance score, a support score, and an availability score; performing nonlinear transformation on the relevance score, the support score, and the availability score, and obtaining a relevance enhanced score, a support enhanced score, and an availability enhanced score; calculating a synergistic effect factor of the relevance enhanced score, the support enhanced score, and the availability enhanced score; calculating a processing score of the reflection processing on the target knowledge point set and the document to be assisted based on the synergistic effect factor, and obtaining a reflection value.

4. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, The external knowledge related to the document to be assisted is searched, and the auxiliary knowledge point set is obtained, which comprises: identifying a business category and a technical feature of the to-be-assisted document; determining a retrieval range and a retrieval priority of external knowledge retrieval corresponding to the to-be-assisted document based on the business category and the technical feature; constructing a retrieval link based on the retrieval range and the retrieval priority; retrieving external knowledge related to the to-be-assisted document according to the retrieval link to obtain an auxiliary knowledge point set.

5. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, calculating semantic similarity between each knowledge point in the auxiliary knowledge point set and corresponding text content of the to-be-assisted document, including: performing vector conversion on the auxiliary knowledge point set and the to-be-assisted document to obtain a first numerical vector and a second numerical vector; calculating cosine similarity of the first numerical vector and the second numerical vector; calculating Jaccard similarity between knowledge elements in the auxiliary knowledge point set and document segments of the to-be-assisted document; linearly weighting and fusing the cosine similarity and the Jaccard similarity to obtain semantic similarity.

6. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, annotating the technical parameter document using the confidence adjustment parameter to obtain a target annotated parameter, including: performing priority sorting on the technical parameter document based on the parameter confidence in the confidence adjustment parameter to generate a parameter annotation sequence; based on the parameter annotation sequence, sequentially performing annotation operations on each parameter to obtain an initial annotated parameter set, the annotation operations including parameter name annotation, parameter value annotation, and parameter confidence annotation; performing logical consistency verification on the initial annotation result, and based on the verification result of the logical consistency verification, performing parameter annotation content correction on the initial annotated parameter set to obtain a target annotated parameter.

7. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, identifying document parameters of the technical parameter document using a large language model, including: performing semantic analysis on the technical parameter document using a large language model to identify parameter description segments in the technical parameter document; identifying parameter names and parameter values in the parameter description segments; mapping the parameter names and the parameter values to predefined parameter categories to obtain document parameters.

8. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, based on the document parameters, analyzing documents in the technical parameter document that need to be assisted in verification to obtain a to-be-assisted document, including: calculating an initial confidence of each parameter in the document parameters; based on the initial confidence, filtering out to-be-verified parameters from the document parameters; locating text segments corresponding to the to-be-verified parameters in the technical parameter document to obtain a to-be-assisted document.

9. The technical parameter labeling system based on a large language model and a reflection mechanism according to claim 1, wherein, based on the reflection value, adjusting the parameter confidence of the to-be-assisted document to obtain a confidence adjustment parameter, including: constructing a segmented mapping function of the reflection value and the parameter confidence corresponding to the to-be-assisted document; mapping the reflection value to a confidence adjustment value using the segmented mapping function; adjusting the initial confidence of the to-be-assisted document using the confidence adjustment value to obtain a confidence adjustment parameter.

10. A technical parameter labeling method based on a large language model and a reflection mechanism, characterized in that, the method comprises: obtaining a to-be-annotated technical parameter document, identifying document parameters of the technical parameter document using a large language model, and based on the document parameters, analyzing documents in the technical parameter document that need to be assisted in verification to obtain a to-be-assisted document; search external knowledge related to the document to be assisted, obtain an auxiliary knowledge point set, calculate semantic similarity between each knowledge point in the auxiliary knowledge point set and text content corresponding to the document to be assisted, and filter a target knowledge point set about the document to be assisted from the auxiliary knowledge point set based on the semantic similarity; perform reflection processing on the target knowledge point set and the document to be assisted to obtain a reflection value, adjust a parameter confidence of the document to be assisted based on the reflection value, and obtain a confidence adjustment parameter; perform parameter labeling on the technical parameter document by using the confidence adjustment parameter to obtain a target labeled parameter.

Images