Corpus quality evaluation system

Abstract

The embodiment of the invention relates to the technical field of computers, and discloses a corpus quality evaluation system. The system comprises a feature extraction module used for determining an explicit rule feature and an implicit depth feature of a neural symbol according to an input corpus; the knowledge graph construction module is used for constructing a knowledge graph of the corpus according to the explicit rule features and the implicit depth features; the multi-head self-attention fusion module is used for determining fusion features according to the knowledge graph; the logical reasoning module is used for performing rule reasoning according to the fusion features to obtain a reasoning result; and the multi-task quality scoring module is used for generating a multi-dimensional quality score according to the reasoning result. The technical problem that a traditional method is difficult to meet the current efficiency requirement can be solved at least.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a corpus quality assessment system. Background Art

[0002] Corpus quality assessment refers to the systematic evaluation and inspection of all aspects of a corpus to determine whether it meets specific standards and requirements and whether it can meet specific usage purposes. It can be seen that corpus quality assessment is a multi-dimensional, multi-method comprehensive evaluation process aimed at ensuring the quality and applicability of the corpus to meet the needs of different users.

[0003] In the related technology, the methods of corpus quality assessment mainly include rule matching method, statistical feature method, manual evaluation method and deep learning method.

[0004] However, the inventors have at least found that the traditional method relies too much on manual work in corpus quality assessment, which can lead to serious problems. With the acceleration of digital transformation and the rapid expansion of corpus size, traditional methods can no longer meet current efficiency requirements, and an innovative solution is urgently needed to overcome these challenges. Summary of the invention

[0005] One purpose of the present application is to provide a corpus quality assessment system, at least to solve the technical problem that traditional methods can no longer meet current efficiency requirements.

[0006] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0007] Some embodiments of the present application also provide a corpus quality assessment system, the system comprising: a feature extraction module, used to determine the explicit rule features and implicit deep features of neural symbols based on the input corpus; a knowledge graph construction module, used to construct a knowledge graph of the corpus based on the explicit rule features and implicit deep features; a multi-head self-attention fusion module, used to determine the fusion features based on the knowledge graph; a logical reasoning module, used to perform rule reasoning based on the fusion features to obtain reasoning results; and a multi-task quality scoring module, used to generate multi-dimensional quality scores based on the reasoning results.

[0008] Compared with the related art, the scheme provided in the embodiment of the present application provides a neural symbol fusion adaptive multi-dimensional corpus quality assessment system based on artificial intelligence. The system may at least include: a feature extraction module for determining the explicit rule features and implicit depth features of the neural symbol according to the input corpus; a knowledge graph construction module for constructing the knowledge graph of the corpus according to the explicit rule features and implicit depth features; a multi-head self-attention fusion module for determining the fusion features according to the knowledge graph; a logical reasoning module for performing rule reasoning according to the fusion features to obtain the reasoning results; a multi-task quality scoring module for generating multi-dimensional quality scores according to the reasoning results. The system can achieve deep semantic understanding and structural analysis of the corpus through the fusion of neural networks and symbolic logic. The ability to grasp complex contexts is enhanced by using the knowledge graph construction module, and an explainable evaluation basis is provided through the logical reasoning module. In addition, multi-dimensional adaptive evaluation can be achieved through the multi-task quality scoring module. Therefore, this application has significant advantages in improving the accuracy, comprehensiveness and interpretability of corpus quality assessment, and has strong adaptive capabilities and can adapt to various complex corpus assessment scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0010] Figure 1 It is an overall exemplary schematic diagram of a corpus quality assessment system provided according to some embodiments of the present application;

[0011] Figure 2 This is an exemplary schematic diagram of the knowledge graph construction module in a corpus quality assessment system provided according to some embodiments of the present application;

[0012] Figure 3 This is an exemplary schematic diagram of the logic reasoning module in a corpus quality assessment system provided according to some embodiments of the present application;

[0013] Figure 4 This is an exemplary schematic diagram of the multi-task quality scoring module in a corpus quality assessment system provided according to some embodiments of the present application;

[0014] Figure 5 This is an exemplary schematic diagram of the causal inference module in a corpus quality assessment system provided according to some embodiments of the present application;

[0015] Figure 6This is an exemplary schematic diagram of the lifelong learning module in a corpus quality assessment system provided according to some embodiments of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] The following terms are used in this article.

[0018] Neural-symbolic fusion is a technical paradigm that combines the learning ability of neural networks with the logical reasoning ability of symbolic systems.

[0019] Meta-learning, also known as Meta-learning, is a technique for learning how to learn that enables models to quickly adapt to new tasks.

[0020] Lifelong learning, also known as Lifelong Learning, refers to the ability of a system to continuously learn and adapt to new knowledge.

[0021] Knowledge distillation, also known as Knowledge Distillation, refers to a technique for transferring the knowledge of a complex model to a simpler model.

[0022] Multi-head self-attention, also known as Multi-head Self-attention, refers to an attention mechanism that can focus on multiple feature dimensions at the same time.

[0023] Knowledge graph is a structured data storage method that can store the relationship between entities, which helps the system understand the entities in the corpus and the relationship between them, and enhances the ability to grasp complex contexts. The knowledge graph contains rich entity and relationship information, which helps the system understand the content of the corpus more comprehensively.

[0024] Graph neural network is a neural network that specializes in processing graph structure data. It can capture the relationship and dependency between entities. In this application, the graph neural network is used to help the system understand and analyze the structural information in complex corpus.

[0025] First embodiment

[0026] The first embodiment of the present application relates to a corpus quality assessment system. Figure 1 As shown, the system may include:

[0027] A feature extraction module is used to determine the explicit rule features and implicit deep features of the neural symbol based on the input corpus;

[0028] The knowledge graph construction module is connected to the feature extraction module and is used to construct the knowledge graph of the corpus according to the explicit rule features and implicit deep features;

[0029] The multi-head self-attention fusion module is connected to the knowledge graph construction module to determine the fusion features according to the knowledge graph;

[0030] The logic reasoning module is connected to the multi-head self-attention fusion module and is used to perform rule reasoning according to the fusion features to obtain reasoning results;

[0031] The multi-task quality scoring module is connected to the logic reasoning module and is used to generate a multi-dimensional quality score according to the reasoning result.

[0032] Specifically, in some examples, the input corpus refers to the corpus to be evaluated; the corpus is the original data of the system. Exemplarily, the corpus can be any form of language data such as text, dialogue, article, etc., which is not specifically limited in this embodiment.

[0033] In some examples, the explicit rule features may be grammatical rules, collocation rules, style guidelines, professional terminology usage rules, etc., which are used to define the norms and standards that should be followed in the corpus, and help the system to understand and process the corpus more accurately. This implementation does not make specific limitations on this.

[0034] Specifically, in some examples, the feature extraction module may be a neural symbolic feature extraction module, which may utilize the fusion technology of neural networks and symbolic logic to perform deep semantic understanding and structural analysis on the corpus.

[0035] Specifically, in some examples, the knowledge graph construction module may be a multimodal knowledge graph construction module, which may construct a multimodal knowledge graph to enhance the system's ability to grasp complex contexts.

[0036] Specifically, in some examples, the multi-head self-attention fusion module can be a graph-enhanced multi-head self-attention fusion module, which can improve the effect of the multi-head self-attention mechanism in processing multimodal data through graph enhancement technology.

[0037] Specifically, in some examples, the logic reasoning module can be a differentiable neural logic reasoning module, which can provide an explainable evaluation basis to make the logic reasoning process differentiable for optimization.

[0038] Specifically, in some examples, the multi-task quality scoring module can be a meta-learning driven multi-task quality scoring module, and the multi-task quality scoring module can adopt meta-learning and multi-task learning techniques to achieve multi-dimensional adaptive evaluation.

[0039] See also Figure 2 As shown in the figure, the collaboration process between the modules includes:

[0040] The neural symbol feature extraction module is the starting point of the entire system processing flow. It is responsible for extracting key features from the input corpus, including explicit rule features and implicit deep features.

[0041] Multimodal knowledge graph construction module: used to receive the output from the neural symbolic feature extraction module, and dynamically build and update the knowledge graph related to the input corpus based on explicit rule features and implicit deep features.

[0042] Graph-enhanced multi-head self-attention fusion module: combines the output of the multimodal knowledge graph construction module (knowledge graph) and the output of the neural symbolic feature extraction module (explicit rule features and implicit deep features) to obtain fused features. Through the graph-enhanced multi-head self-attention mechanism, the system can integrate and process information from different sources and types to extract richer and more comprehensive feature representations.

[0043] Differentiable neural logic reasoning module: It is used to receive the output (the fused features) of the multi-head self-attention fusion module of the graph enhancement, and use the differentiable neural logic reasoning technology to perform rule reasoning on the fused features to obtain the reasoning results. This reasoning method combines the learning ability of neural networks and the interpretability of symbolic logic, making the reasoning process both accurate and interpretable.

[0044] Meta-learning driven multi-task quality scoring module: used to receive the output (inference result) of the differentiable neural logic reasoning module, and generate a multi-dimensional quality scoring result based on the reasoning result.

[0045] It can be seen that the system realizes the complete process from corpus input to quality scoring and improvement suggestion output through close cooperation between modules. Each module undertakes specific tasks, and the output and input relationship between modules forms a coherent processing chain, ensuring the effectiveness and adaptability of the entire system.

[0046] The corpus quality assessment system proposed in this application is applicable to a variety of scenarios, including but not limited to: education and training content assessment, technical document management, content creation platform, multilingual localization, enterprise knowledge base management, academic research publishing, and legal document processing. The system can automatically evaluate the quality, standardization, and professionalism of text content. Through neural symbolic fusion technology, fully automatic processing is achieved, with multi-dimensional analysis, continuous learning, and intelligent features. This can not only significantly improve the evaluation efficiency, but also maintain the consistency of the evaluation standards, and is particularly suitable for application scenarios that require large-scale, standardized, and continuous quality management. For example, the system can help educational institutions evaluate the quality of teaching materials, assist enterprises in managing technical documents, support content platforms for content review, or assist publishers in evaluating academic papers, thereby greatly improving work efficiency and ensuring the stability of evaluation quality.

[0047] It can be understood that in the relevant technology, the traditional corpus quality assessment system is usually composed of a simple combination of multiple functional modules, which lack effective collaboration between these modules and are difficult to play the overall advantages. In addition, the system lacks adaptive capabilities and is difficult to cope with emerging corpus types and changing quality standards. The opacity of the assessment process limits the ability to provide clear and explainable assessment basis. The high degree of manual participation and low evaluation efficiency make it difficult to meet the needs of large-scale applications. First, the system has poor adaptability and it is difficult to quickly respond to new corpus types and changes in quality standards; second, the manual evaluation method basically relies on expert experience, the evaluation standards vary from person to person, and there is a lack of objective and unified evaluation basis; third, excessive reliance on manual labor leads to slow processing speed and high cost, and it is impossible to effectively cope with the processing needs of massive corpora; fourth, the scalability of these methods is limited, and it is difficult to achieve continuous accumulation and optimization of evaluation capabilities. These problems seriously restrict the large-scale development of corpus quality assessment work.

[0048] In summary, the automation technology breakthrough of this application is of great significance. In this application, the corpus quality assessment is transformed from manual assessment to intelligent automation, which significantly improves the evaluation efficiency and achieves an improvement of several orders of magnitude. This transformation greatly reduces the investment in labor costs and enables the evaluation work to be carried out on a large scale. At the same time, this application overcomes the subjectivity of manual evaluation and ensures the consistency of evaluation standards. The system can continuously optimize itself and continuously improve the evaluation capabilities, truly realizing the leap from manual dependence to intelligent automation.

[0049] Second embodiment

[0050] The second embodiment of the present application relates to a corpus quality assessment system. The second embodiment is an improvement on the first embodiment, and the specific improvement is that: in the second embodiment of the present application, the corpus quality assessment system can also include a causal inference module and / or a lifelong learning module.

[0051] Specifically, in some embodiments, in combination Figure 1 As shown, the causal inference module is connected to the multi-task quality scoring module to analyze influencing factors and generate improvement suggestions based on the quality score; the lifelong learning module is respectively connected to the feature extraction module, the knowledge graph construction module, the multi-head self-attention fusion module, the logical reasoning module, the multi-task quality scoring module and the causal inference module to continuously optimize the corpus quality assessment system.

[0052] Specifically, in some embodiments, the causal inference module can be used to analyze influencing factors and generate improvement suggestions based on the quality score.

[0053] Specifically, in some embodiments, the lifelong learning module is used to continuously optimize the corpus quality assessment system, which can ensure that the system can be continuously optimized and adapted to new assessment standards.

[0054] That is, the causal inference module is used to analyze the various factors that affect the quality of the corpus based on the output (quality scoring results) of the meta-learning-driven multi-task quality scoring module, and to generate improvement suggestions based on these analyses. The lifelong learning module plays a role in the entire evaluation process and is used to continuously optimize the parameters and structures of each module.

[0055] It is not difficult to find that in the embodiments of the present application, the corpus quality assessment system may further include a causal inference module and / or a lifelong learning module, which can analyze influencing factors and generate improvement suggestions based on the quality score, and continuously optimize the corpus quality assessment system.

[0056] Third embodiment

[0057] The third embodiment of the present application relates to a corpus quality assessment system. The third embodiment is an improvement on the first embodiment, and the specific improvement is: in the third embodiment of the present application, a specific implementation of the feature extraction module is provided.

[0058] Specifically, in some embodiments, the first extraction module is used to determine implicit deep features based on the corpus and the pre-trained language model;

[0059] The second extraction module is used to determine explicit rule features based on the corpus, background knowledge and neural symbolic program induction;

[0060] The processing module is used to determine structured features based on the function used to convert text into a graph structure, the graph neural network and the corpus.

[0061] Specifically, the feature extraction module can extract and process the input corpus through three main steps, and extract the neural symbol features of the corpus to obtain explicit rule features and implicit deep features.

[0062] Specifically, in some examples, the feature extraction module may determine implicit deep features based on the corpus and the pre-trained language model through the first extraction module. For details, please refer to the following:

[0063] E = Transformer (C);

[0064] Wherein, C represents the corpus, and E represents the implicit deep feature, that is, the context representation.

[0065] In this example, the pre-trained language model is specifically a Transformer model. However, it is worth emphasizing that this is only an example. In some other examples, the pre-trained language model can also be other types of models. The embodiments of the present application do not specifically limit this. Among them, the pre-trained language model is usually trained on a large-scale corpus and can capture rich language patterns and semantic information. Through the first extraction module, the system can obtain an understanding of the deep semantic features of the corpus.

[0066] Specifically, in some embodiments, the feature extraction module can determine the explicit rule feature through the second extraction module according to the corpus, background knowledge and neural symbolic program induction. The specific formula can be referred to as follows:

[0067] R = NSPI(C,B);

[0068] Wherein, C represents the corpus, B represents background knowledge, and R represents the explicit rule feature. Exemplarily, the explicit rule feature may be expressed in the form of a rule set.

[0069] In this example, the NSPI represents the neural symbolic program induction technology. The neural symbolic program induction technology NSPI is a technology that combines neural networks and symbolic reasoning, which can learn and summarize rules from data. That is to say, this embodiment can use the neural symbolic program induction technology NSPI to extract the explicit rule features from the corpus C and the background knowledge B. Among them, the background knowledge refers to the background knowledge related to the subject of the corpus retrieved from a predefined knowledge base, which is used to help understand and interpret the corpus, and can assist the system to better extract the explicit rule features. The background knowledge may include but is not limited to: domain dictionaries, language specifications, writing rules, etc. The explicit rule features may include but are not limited to: grammar rules, format rules, domain rules and other formalized corpus quality assessment rules.

[0070] Specifically, in some embodiments, the feature extraction module may determine the structured features through the processing module according to the function for converting text into a graph structure, the graph neural network and the corpus. The specific formula may be referred to as follows:

[0071] Graph_s = TextToGraph(C);

[0072] S = GNN(Graph_s);

[0073] Wherein, the C represents the corpus, the Graph_s represents the heterogeneous graph representation of the corpus, and the S represents a structured feature.

[0074] In this example, the TextToGraph represents a function that converts text into a graph structure, and GNN represents a graph neural network. That is, the corpus is first converted into a corpus graph structure by the TextToGraph function, and the corpus graph structure Graph_s represents the entities and the relationships between entities in the corpus. Then, the heterogeneous graph representation Graph_s is processed by the graph neural network GNN, and the system can extract a structured feature S, which is used to reflect the structured information such as the organizational structure and entity relationships in the corpus.

[0075] Optionally, in some embodiments, the processing module includes a construction unit, a learning unit, a level representation unit, and a structure quality score calculation unit. The construction unit is used to implement the function of the above-mentioned TextToGraph function, construct a heterogeneous graph representation Graph_s of the corpus, the learning unit, as a part of the graph neural network GNN, uses a relationship-aware graph attention network to perform node representation learning on the heterogeneous graph representation Graph_s; the level representation unit, as a part of the graph neural network GNN, applies a dynamic gating mechanism to fuse representations of different levels; the structure quality score calculation unit, as the last part of the graph neural network GNN, calculates the final structured feature S.

[0076] Specifically, in some embodiments, the processing module may include a construction unit, and the construction unit is used to construct a heterogeneous graph representation of the corpus. The specific formula can be referred to as follows:

[0077] Graph_s=TextToGraph(V_s,E_s,R_type);

[0078] Among them, the V_s represents a node set containing words, sentences, paragraphs, etc., the E_s represents the edge set of the connection relationship between nodes in the node set, the R_type represents the relationship type set that defines the relationship types between different types of nodes, and the Graph_s represents the constructed heterogeneous graph representation. Then, the Graph_s is processed by the graph neural network, including the steps of node representation learning, dynamic gating mechanism fusion and structured feature calculation, and finally the structured feature S mentioned above is obtained. The structured feature S is used to reflect the structured information such as the organizational structure and entity relationship in the corpus. The heterogeneous graph representation Graph_s can be used as the input of the learning unit for node representation learning.

[0079] Further, in some embodiments, the processing module may include a learning unit, the learning unit being configured to perform node representation learning on the heterogeneous graph representation Graph_s using a relation-aware graph attention network. Given a heterogeneous graph representation Graph_s = (V_s, E_s, R_type), for each node in the graph, the specific formula may be referred to as follows:

[0080] α_ij^r=softmax(LeakyReLU(a^T[W_node v_i||W_node v_j||We_r]));

[0081] v_i'=f_act(Σr∈R_typeΣj∈N_i^rα_ij^r W^r v_j);

[0082] Among them, the α_ij^r represents the attention weight of node i to node j under the relationship type r (r∈R_type), the a represents the parameter vector of the attention network, the W_node represents the node feature transformation matrix, which is different from the meaning of the weight matrix W^O in this article, the We_r represents the feature matrix of the relationship r, and the v_i and v_j represent the feature representation of nodes i and j.

[0083] Wherein, N_i^r represents the neighbor set of node i under relationship r,

[0084] Further, in some embodiments, the processing module may include a level representation unit, and the level representation unit is used to apply a dynamic gating mechanism to fuse the representations of nodes at different levels in the heterogeneous graph representation Graph_s. For the representations of nodes at different levels in the heterogeneous graph representation Graph_s, the specific formulas can be referred to as follows:

[0085] g_t=f_act(W_g[v_word; v_sent; v_para]);

[0086] v_final=g_t×v_word+(1-g_t)×(W_s v_sent+W_p v_para);

[0087] Among them, the g_t represents the gating vector that controls the fusion ratio of features at different levels, the v_word, v_sent, and v_para respectively represent the word-level, sentence-level, and paragraph-level representations extracted from the heterogeneous graph representation Graph_s, the W_g represents the learnable weight matrix of the gating mechanism (different from the other weight matrices in the multi-head self-attention fusion module later), the W_s and W_p respectively represent the learnable weight matrices of sentence-level and paragraph-level feature fusion, and the f_act represents the activation function used for the graph neural network.

[0088] Herein, v_final represents the final representation obtained by fusion through the dynamic gating mechanism.

[0089] Further, in some embodiments, the processing module may include a structure quality score calculation unit, and the structure quality score calculation unit is used to calculate the structure quality score according to the final representation. The specific formula can be referred to as follows:

[0090] Q_structure = MLP(v_final);

[0091] Wherein, the Q_structure represents the structural quality score of the corpus.

[0092] It can be understood that the structural quality score calculation unit can provide important structural information for the overall corpus quality assessment by analyzing the structural features of the corpus.

[0093] It can be understood that the processing module can work closely with the knowledge graph construction module to provide complementary structured information, thereby significantly improving the accuracy of the overall corpus quality assessment. By utilizing the graph attention network and the dynamic gating mechanism, it is possible to effectively construct a heterogeneous graph, learn the representation of each node in the graph, and fuse representation information from different levels. This series of operations aims to deeply explore the structural features of the corpus and ensure that the evaluation process can fully consider multiple aspects of the corpus, including its implicit deep features, explicit rule features, and structured features. Key factors. Through this comprehensive evaluation, the system can more accurately understand and evaluate the quality of the corpus.

[0094] In summary, the feature extraction module of this system is responsible for extracting multi-dimensional features from the input corpus to comprehensively evaluate the quality of the corpus. This module includes three key parts: The first extraction module: focuses on determining deep semantic features, which provide an understanding of the deep meaning of the corpus. This usually involves using deep learning models to mine complex patterns and associations such as semantic similarity and sentiment tendency. The second extraction module: is responsible for identifying explicit rule features, which include aspects that can be directly observed and defined, such as grammatical rules and vocabulary usage, to facilitate the assessment of whether the corpus conforms to specific specifications. The processing module: is used to determine structured features, which provide insights into the organizational structure and entity relationships of the corpus, and help understand the organization and structure of the corpus. Through this multi-dimensional feature extraction, this system is able to comprehensively evaluate the quality of the corpus, not only considering the surface form of the corpus, but also deeply analyzing its deep meaning, norm compliance, and key aspects such as structural organization. This comprehensiveness provides a solid foundation for building a corpus quality assessment system that is both accurate and reliable, and highly interpretable.

[0095] It should be noted that this embodiment may also be an improvement based on the second embodiment.

[0096] It is not difficult to find that in the embodiment of the present application, the feature extraction module extracts and processes the features of the corpus from different angles through the first extraction module, the second extraction module and the processing module, including implicit deep features, explicit rules and structural information. These features can provide rich information for the subsequent modules of the system, so that the system can understand and evaluate the quality of the corpus more comprehensively and accurately. Due to the combination of pre-trained language models, neural symbolic program induction and graph neural network technologies, the system has higher flexibility and accuracy in processing complex corpus, thereby improving the reliability of corpus quality assessment.

[0097] Fourth embodiment

[0098] The fourth embodiment of the present application relates to a corpus quality assessment system. The fourth embodiment is an improvement on the third embodiment. The specific improvement is: in the fourth embodiment of the present application, a specific implementation method of the knowledge graph construction module is provided, such as Figure 2 shown.

[0099] Specifically, in some embodiments, the knowledge graph construction module can be used to receive the implicit deep features E, explicit rule features R and structured features S output by the feature extraction module. That is, the implicit deep features E, explicit rule features R and structured features S are used as inputs of the knowledge graph construction module. After that, the knowledge graph construction module can determine the entity set and the relationship set through the set extraction module according to the implicit deep features, named entity recognition function and relationship extraction function. That is to say, the knowledge graph construction module is used to dynamically construct and update the knowledge graph related to the input corpus.

[0100] Among them, according to the implicit deep features, named entity recognition function and relationship extraction function, the entity set and the relationship set are determined, and the following formula can be referred to:

[0101] Entities,Relations=NER(E)+RE(E);

[0102] Wherein, the E represents the implicit deep feature, the Entities represents the extracted entity set, and the Relations represents the extracted relationship set.

[0103] Wherein, the NER represents the named entity recognition function, and the RE represents the relation extraction function. The named entity recognition function is used to identify specific entities in the text, such as names, places, organizations, etc.; the relation extraction function is used to identify the relationship between entities. The entity set and the relationship set are the basis for constructing the knowledge graph.

[0104] Furthermore, in some embodiments, the knowledge graph construction module may include a link establishment module, and the link establishment module is used to link an entity to an existing knowledge base, and may refer to the following formula:

[0105] Entities_linked=EntityLinking(Entities,KB,R);

[0106] Among them, the EntityLinking represents the entity linking function, the Entities represents the entity set, the KB represents the external knowledge base, the R represents the explicit rule feature, and the Entities_linked is used to represent the target entity set after the link is established.

[0107] That is, the link establishment module can obtain the target entity set Entities_linked by combining the entity set Entities, the existing external knowledge base KB and the explicit rule feature R according to the entity linking function EntityLinking. The entity linking function EntityLinking is a technology that matches entities in text with entities in a knowledge base, which helps to connect the information in the text with a wider knowledge network and enhance the semantic information of the entity.

[0108] In some embodiments, the knowledge graph construction module may further include a fusion construction module, which is used to fuse multimodal information to construct a graph. For example, the following may be referred to:

[0109] G_km=FuseGraph(Entities_linked,Relations,V,A,S);

[0110] Among them, the FuseGraph represents the graph fusion function, the Entities_linked represents the target entity set, the Relations represents the relationship set, the V represents the visual feature, the A represents the audio feature, the S represents the structured feature, and the G_km represents the constructed multimodal knowledge graph.

[0111] That is to say, the multimodal knowledge graph G_km can be constructed by fusing the target entity set Entities_linked, the relationship set Relations, the visual feature V, the audio feature A and the structured feature S. Since the fused content includes not only text information but also visual information and audio information, the knowledge graph can more comprehensively represent the multimodal characteristics of the corpus.

[0112] Furthermore, in some embodiments, the knowledge graph construction module may also include an updating module, and the updating module is used to update entity and relationship representations according to the graph neural network. For example, the following formula may be referred to:

[0113] H = GraphTransformer(G_km)

[0114] Among them, the G_km represents the knowledge graph, the GraphTransformer represents the graph transformation network model, and the H represents the node representation in the knowledge graph, that is, the updated entity and relationship representation.

[0115] That is, the updating module can process the knowledge graph G_km using the graph neural network model Graph Transformer to obtain updated entity and relationship representations H. Doing so helps capture the complex interactions between entities and relationships and provides richer semantic and structural information.

[0116] In summary, the knowledge graph construction module can provide rich semantic and structural information for corpus quality assessment. These semantic and structural information can not only include entity and relationship representations in the text, but also include link information with external knowledge bases and fusion of multimodal information, so that the knowledge graph can more comprehensively represent the content of the corpus.

[0117] It should be noted that this embodiment may also be an improvement based on the first embodiment and / or the second embodiment.

[0118] It is not difficult to find that in the embodiment of the present application, the knowledge graph construction module collaborates with the feature extraction module, and the knowledge graph construction module uses the implicit deep features, explicit rule features and structured features extracted by the feature extraction module to enhance the construction process of the knowledge graph. For example, the extracted implicit deep features can be used to improve the recognition of entities and relationships, the explicit rule features can be used to guide entity linking and relationship extraction, and the structured features can be used to build the structure of the graph. This collaboration enables the entire system to more accurately understand and evaluate the quality of the corpus. In other words, the knowledge graph construction module relies on three types of features output by the feature extraction module: implicit deep features, explicit rule features and structured features. These features are used for entity relationship extraction, entity linking and multimodal information fusion, respectively. Through the collaborative work of these three types of features, the construction process of a complete knowledge graph from entity relationship extraction, entity linking to multimodal information fusion can be realized. It can be seen that this embodiment can realize the automatic construction from the corpus to the knowledge graph through the close collaboration of the feature extraction module and the knowledge graph construction module.

[0119] Fifth embodiment

[0120] The fifth embodiment of the present application relates to a corpus quality assessment system. The fifth embodiment is an improvement on the first embodiment, and the specific improvement is: in the fifth embodiment of the present application, a specific implementation method of the multi-head self-attention fusion module is provided.

[0121] Specifically, in some embodiments, the multi-head self-attention fusion module can be used to receive two groups of inputs: the first group is the output from the feature extraction module, including implicit deep features E, explicit rule features R and structured features S; the second group is the output from the knowledge graph construction module, that is, the node representation H in the knowledge graph. By integrating these two groups of inputs, the multi-head self-attention fusion module can effectively fuse feature information from different sources to enhance the representation ability of the knowledge graph. In other words, the multi-head self-attention fusion module is used to integrate multi-source heterogeneous features, that is, features from different sources and different types (heterogeneous). Through graph enhancement, the system can better understand the relationship between features, and the multi-head self-attention mechanism allows the model to focus on multiple different aspects of the corpus at the same time, improving the effect of feature integration.

[0122] Optionally, in some embodiments, the multi-head self-attention fusion module may include a preprocessing module, which is used to preprocess the implicit deep features, explicit rule features and structured features output by the feature extraction module to obtain a splicing result of the neural symbolic features. Exemplarily, the preprocessing process can refer to the following formula:

[0123] X=Concat(E,R,S)

[0124] Q=Linear_q(X)

[0125] K=Linear_k(X)

[0126] V1=Linear_v(X)

[0127] Among them, the Concat represents a feature concatenation function, which is used to connect different features by dimension, the E represents the implicit deep feature, the R represents the explicit rule feature, the S represents the structured feature, the X represents the concatenation result of the neural symbolic feature, the Linear_q represents the first linear transformation function, the Q represents the query matrix, the Linear_k represents the second linear transformation function, the K represents the key matrix, the Linear_v represents the second linear transformation function, and the V1 represents the value matrix.

[0128] Furthermore, in some embodiments, the multi-head self-attention fusion module may include a calculation module, and the calculation module is used to calculate the multi-head self-attention according to the splicing result. Exemplarily, the following formula may be referred to:

[0129] headi=Attention(QWi^Q,KWi^K,V1Wi^V1)

[0130] Wherein, Q represents the query matrix, K represents the key matrix, V1 represents the value matrix, Wi^Q, Wi^K, Wi^V1 represent the learnable weight matrices, Attention represents the attention calculation function, and headi represents the output of the i-th attention head. Each attention head will output an attention-weighted result.

[0131] That is to say, the calculation module can calculate the output of each attention head in the multi-head self-attention mechanism. It can be understood that the self-attention mechanism allows the model to focus on information at different positions in the sequence when processing sequence data. Here, QWi^Q, KWi^K, V1Wi^V1 respectively represent the results of multiplying the query matrix Q, the key matrix K and the value matrix V1 with the learnable weight matrices Wi^Q, Wi^K, and Wi^V1.

[0132] Furthermore, in some embodiments, the multi-head self-attention fusion module may include a multi-head attention output module. The multi-head attention output module is used to perform splicing according to the number of attention heads and the linear transformation matrix to determine the output result. For example, the following formula can be referred to:

[0133] MultiHead(Q,K,V1)=Concat(head1,...,headh)W^O;

[0134] Among them, h represents the number of attention heads, W^O represents the output linear transformation matrix, and MultiHead(Q,K,V1) represents the output result of the multi-head attention output module.

[0135] That is, the multi-head attention output module can be used to concatenate the outputs of all attention heads and process them through a linear transformation matrix W^O to obtain a fused feature representation. The concatenation operation allows the model to capture information in different representation subspaces and merge them at the end.

[0136] Furthermore, in some embodiments, the multi-head self-attention fusion module may include a fusion feature output module. The fusion feature output module is used to determine the final fusion feature according to the output result, the node representation in the knowledge graph, the feedforward neural network and the layer normalization processing function. Exemplarily, the following formula can be referred to:

[0137] F=LayerNorm(MultiHead(Q,K,V)+FFN(H));

[0138] Among them, the MultiHead (Q, K, V1) represents the output of the multi-head attention output module, the FFN represents the feedforward neural network, the H represents the node representation of the knowledge graph, the LayerNorm represents the layer normalization function, and the F represents the final fusion feature. It can be understood that the layer normalization processing of the layer normalization function helps to stabilize the training process, and the feedforward neural network FFN can achieve further conversion and integration of features.

[0139] It should be noted that this embodiment may also be an improvement based on any one or more of the first to third embodiments.

[0140] It is not difficult to find that in the embodiment of the present application, the multi-head self-attention fusion module can provide a more comprehensive and in-depth information representation for corpus quality assessment by fusing the implicit deep features E, explicit rule features R and structured features S from the feature extraction module and the node representation H of the knowledge graph output by the knowledge graph construction module. Such feature fusion can not only enrich the dimension of information, but also provide a comprehensive feature basis for the subsequent logical reasoning module, making the entire evaluation process more accurate and reliable.

[0141] Sixth embodiment

[0142] The sixth embodiment of the present application relates to a corpus quality assessment system. The sixth embodiment is an improvement on the fifth embodiment. The specific improvement is: in the sixth embodiment of the present application, a specific implementation method of the logic reasoning module is provided, such as Figure 3 shown.

[0143] Specifically, in some embodiments, the logic reasoning module can receive two inputs: one is the fusion feature F from the multi-head self-attention fusion module, and the other is the explicit rule feature set R from the neural symbolic feature extraction module, and perform the following operations to achieve rule-based soft reasoning and enhance the interpretability of corpus quality assessment. In other words, the logic reasoning module is used to perform rule-based soft reasoning, that is, embedding logical rules in the neural network so that the reasoning process can be differentiated, which can be optimized by the back-propagation algorithm. This method that combines rule reasoning and neural networks can provide more accurate and interpretable reasoning results.

[0144] Specifically, in some embodiments, the logic reasoning module may include a conversion module, the conversion module, the conversion module, is used to convert the symbolic rule in the explicit rule feature into a tensor rule. For example, see the following formula:

[0145] T(r)=NeuralTensor(r)for r in R

[0146] Among them, the R represents the explicit rule feature, r represents the symbolic rule in the explicit rule feature R, the NeuralTensor represents the conversion function of the tensor rule, and T(r) represents the neural tensor representation of the symbolic rule r.

[0147] The conversion module enables traditional symbolic rules to be executed in the form of tensor operations in the neural network. r can be expressed as a tensor operation.

[0148] Further, in some embodiments, the logic reasoning module may include a soft logic reasoning module, and the soft logic reasoning module is used to perform soft logic reasoning according to the tensor rule, the fusion feature and the weight information used to characterize the importance of the symbol rule, and determine the soft logic reasoning result. For example, the following formula can be referred to:

[0149] y = σ(Σ(w_k×T(r_k)(F)));

[0150] Wherein, k=1,2,...,n, wherein n represents the total number of the symbolic rules, the w_k represents the weight information of the importance of the kth symbolic rule in the explicit rule feature R, and the weight information is used to represent the importance of different rules in quality assessment, the r_k represents the kth symbolic rule in the explicit rule feature R, the T(r_k) represents the neural tensor representation of the kth rule in the explicit rule feature R, the F represents the fusion feature, the σ represents the activation function, and the y represents the soft inference result.

[0151] In this step, the neural tensor representation T(r_k) of each symbolic rule is multiplied by the corresponding rule weight w_k, and then all the results are summed up and activated by an activation function σ (such as a sigmoid function). The soft logic reasoning module allows the activation degree of the symbolic rule to vary between 0 and 1, providing a fuzzy and differentiable reasoning method.

[0152] Furthermore, in some embodiments, the logic reasoning module may include a weight updating module, and the weight updating module is used to update the weight information according to the weight information, the learning rate and the loss function to obtain the target weight information. For example, the following formula may be referred to:

[0153]

[0154] Wherein, the w_k represents the weight information, the η represents the learning rate, the L represents the loss function, and the w_k' represents the target weight information.

[0155] The target weight information is used to optimize the soft logic reasoning result.

[0156] It should be noted that this embodiment may also be an improvement based on any one or more of the first to fourth embodiments.

[0157] It is not difficult to find that in the embodiment of the present application, the logical reasoning module realizes soft logical reasoning by innovatively converting symbolic rules into tensor rules, which allows the weight information to be updated during the training process. This soft reasoning method based on symbolic rules can not only improve the interpretability of corpus quality assessment, but also provide an interpretable intermediate result, i.e., soft reasoning result y, for multi-task quality scoring by utilizing the fusion feature F provided by the multi-head self-attention fusion module and the explicit rule feature R provided by the feature extraction module. The reasoning process of the logical reasoning module can enhance the transparency of the evaluation, so that the evaluation results are not only accurate, but also easy to understand. Through this reasoning method, the system can provide clear evaluation basis and improvement suggestions in multi-task quality scoring, thereby bringing new technological breakthroughs to the field of corpus quality assessment.

[0158] Seventh embodiment

[0159] The seventh embodiment of the present application relates to a corpus quality assessment system. The seventh embodiment is an improvement on the sixth embodiment. The specific improvement is: in the seventh embodiment of the present application, a specific implementation method of the multi-task quality scoring module is provided, such as Figure 4 shown.

[0160] In some embodiments, after obtaining the soft reasoning result y, the multi-task quality scoring module can implement adaptive multi-dimensional quality assessment. That is, the multi-task quality scoring module uses the soft reasoning result y as input to adaptively evaluate multiple quality dimensions so as to quickly switch and learn between different quality assessment tasks. For example, meta-learning technology can be used to adaptively evaluate multiple quality dimensions. Meta-learning enables the model to quickly adapt to new tasks. Meta-learning technology enables the system to adaptively evaluate different quality dimensions and improve the accuracy and flexibility of scoring.

[0161] Specifically, in some embodiments, the multi-task quality scoring module may include a task distribution module, and the task distribution module is used to determine the task distribution according to the quality assessment subtasks and the total number of the quality assessment subtasks. For example, the following formula can be referred to:

[0162] p(T)over{T1,T2,...,Tn1};

[0163] Wherein, T represents the quality assessment subtask, n1 represents the total number of quality assessment subtasks, and p(T) represents the task distribution;

[0164] Specifically, in some examples, the quality assessment subtasks may be, but are not limited to, specific assessment tasks such as grammatical correctness, content coherence, terminology usage assessment, language fluency assessment, and format standardization assessment.

[0165] That is, the task distribution module specifically defines a task distribution p(T), which can cover all possible quality assessment subtasks {T1, T2, ..., Tn'}. The task distribution can be used to reflect the importance or frequency of occurrence of different subtasks in the overall assessment.

[0166] In some embodiments, the multi-task quality scoring module may include a meta-learning optimization module, and the meta-learning optimization module is used to optimize the target. For example, see the following formula:

[0167] θ1=argmin_θE_T~p(T)[L_T(fθ(y))]

[0168] Among them, the argmin_θ represents the parameter θ corresponding to the minimum value of the target expression, the θ represents the parameter of the quality scoring model fθ, the fθ(y) represents the parameterized model with the soft logic reasoning result y as input, the p(T) represents the probability distribution on the task set {T1, T2, ..., Tn1}, the L_T represents the loss function of task T, the E_T~p(T) represents the expectation on multiple quality assessment subtask distributions p(T), and the θ1 represents the optimal model parameters obtained by minimizing the expected loss on multiple evaluation dimensions.

[0169] That is, the learning optimization module finds the optimal model parameter θ1 by optimizing a meta-learning objective. The learning objective is to minimize the p(T) The expected loss on ET is E_T~p(T)[L_T(fθ)]. This means that the model is trained to perform best on average over all possible tasks, rather than just optimizing on a single task.

[0170] In some embodiments, the multi-task quality scoring module may include a task adaptation module, and the task adaptation module is used to determine the task target parameters. For example, see the following formula:

[0171]

[0172] Wherein, θ1 represents the optimal model parameter, α represents the task adaptation learning rate, Represents the gradient with respect to the parameter θ, and θ_T represents the task target parameter.

[0173] That is, once the task adaptation module obtains the optimal model parameters θ1, it will fine-tune each specific task T to obtain the task target parameters θ_T. This is achieved by updating the parameters with the task adaptation learning rate α in the gradient direction of the loss function L_T of task T. This process allows the model to quickly adapt to new tasks without the need to train from scratch.

[0174] In some embodiments, the multi-task quality scoring module may include a scoring module, and the scoring module is used to score the task quality. For example, the following formula can be referred to:

[0175] Q_m=f_θ_T(x)for m in{1,2,...,n2}

[0176] Among them, x represents input data, f_θ_T represents a parameterized model using task target parameter θ_T, m represents the index of the quality dimension, n2 represents the total number of the quality dimensions, and Q_m represents the scoring result of the mth quality dimension.

[0177] That is, the scoring module scores the input data x (including the soft logic reasoning result y, the fusion feature F, the explicit rule feature R and the knowledge graph node representation H) by using the task target parameter θ_T to obtain the quality dimension scoring result Q_m of each quality dimension m. In this way, the model can provide customized scoring for each quality assessment subtask, thereby realizing multi-dimensional quality assessment.

[0178] It should be noted that this embodiment may also be an improvement based on any one or more of the first to fifth embodiments.

[0179] It is not difficult to find that in the embodiment of the present application, the multi-task quality scoring module determines the distribution of different quality assessment tasks by defining the task distribution; by optimizing the meta-learning goal, to set and optimize the objective function of meta-learning to improve the generalization ability of the model; by performing task-specific adaptation: adjusting the model parameters for each specific task to achieve optimal performance. Through multi-task quality scoring, quality scoring is performed on multiple tasks to evaluate multiple quality dimensions of the corpus. This evaluation method not only improves the generalization ability of the model when facing new tasks, but also provides support for in-depth analysis and improvement suggestions for the causal inference module by providing detailed quality scores. This module can use the output of the differentiable neural logic reasoning module, and combined with the meta-learning strategy, to adaptively evaluate multiple quality dimensions, thereby providing comprehensive quality scoring information for the causal inference module.

[0180] Eighth embodiment

[0181] The eighth embodiment of the present application relates to a corpus quality assessment system. The eighth embodiment is an improvement on the seventh embodiment. The specific improvement is that: in the eighth embodiment of the present application, a specific implementation method of the causal inference module is provided, such as Figure 5 shown.

[0182] In some embodiments, the causal inference module can be used to analyze the quality influencing factors and generate improvement suggestions after obtaining the scoring result Q_m output by the multi-task quality scoring module. That is, the causal inference module is used to analyze the quality influencing factors and generate improvement suggestions. Specifically, it is used to analyze various factors affecting the quality of the corpus and generate improvement suggestions based on these analyses. Causal inference technology reveals the causal relationship between variables and provides deeper insights for quality improvement.

[0183] Specifically, in some embodiments, the causal inference module may include a model building module, and the model building module is used to determine a structured causal model according to the fusion feature, the explicit rule feature and the scoring result. For example, the following formula can be referred to:

[0184] SCM=BuildCausalGraph(F,R,Scores);

[0185] Wherein, the F represents the fusion feature, the R represents the explicit rule feature, and the Scores represents the scoring result set including all Q_m, that is, the scoring result. In other words, the complete quality scoring set Scores includes the scoring results of all dimensions {Score_1, Score_2, ..., Score_n3}, where Score_1 represents the score of the first quality dimension (such as grammatical correctness), Score_2, ..., Score_n3 and so on, the n3 represents the number of scores in the complete quality score, and the SCM represents the structured causal model.

[0186] That is, the model building module can construct the structured causal model SCM by using the fusion feature F, the explicit rule feature R and the set of quality scores Scores. The structured causal model SCM is a mathematical model that can be used to describe the causal relationship between variables, including potential causal graphs and corresponding structural equations.

[0187] Further, in some embodiments, the causal inference module may include a counterfactual reasoning module, the counterfactual reasoning module, the counterfactual reasoning module, the counterfactual reasoning module, is used to determine the reasoning result of the counterfactual reasoning under a given intervention according to the causal model. For example, the following formula can be referred to:

[0188] CF=Counterfactual(SCM,Features,Scores_actual,I);

[0189] Wherein, the SCM represents the structured causal model, the Features represents the feature input of the current corpus, the Scores_actual represents the actual quality score output of the current corpus, the I represents the specific intervention operation on the features of the corpus, and the CF represents the reasoning result of counterfactual reasoning under the given intervention I. It can be understood that the reasoning result CF is a hypothetical reasoning used to explore how the current result would be different if a certain event in the past had not occurred. In this step, the counterfactual reasoning module uses the structured causal model, the feature input Features of the current corpus, the actual quality score output Scores_actual of the current corpus, and the specific intervention operation I on the features of the corpus to perform counterfactual reasoning. In this way, it helps to understand how the output may change under different intervention conditions.

[0190] In some embodiments, the causal effect calculation module is used to determine the average treatment effect based on the intervened corpus feature variables and quality score variables. For example, the following formula can be used:

[0191] ATE=E1[Score_i|do(Feature_j=f1)]-E1[Score_i|do(Feature_j=f0)];

[0192] Among them, E1 represents expectation (average value), Feature_j represents the j-th intervened feature variable in the above Features, Score_i represents the i-th quality score variable in Scores_actual, f0 represents the original value of Feature_j, f1 represents the new value of Feature_j, do(Feature_j=f0) and do(Feature_j=f1) represent the intervention operation on the feature variable Feature_j so that its values ​​are f0 and f1, and ATE represents the average treatment effect.

[0193] Specifically, the calculation process of the average processing effect can be understood as: E1[Score_i|do(Feature_j=f1)], which represents the expected value of the quality score Score_i when the feature Feature_j is forced to be set to the new value f1, and E1[Score_i|do(Feature_j=f0)] represents the expected value of the quality score Score_i when the feature Feature_j is forced to be set to the original value f0. The difference ATE between the two reflects the average impact of the feature on the quality score when it changes from f0 to f1. For example, if Feature_j represents the average length of the sentence and Score_i represents the readability score, the positive and negative values ​​of ATE can reflect whether increasing the sentence length improves or reduces the readability of the text, thereby providing a specific quantitative basis for improving the quality of the corpus.

[0194] Furthermore, in some embodiments, the suggestion generation module is used to generate improvement suggestions based on the inference result and the average processing effect. For example, the following formula can be referred to:

[0195] Suggestions=GenerateAction(CF,ATE)

[0196] Wherein, the CF represents the inference result, the ATE represents the average treatment effect, and the Suggestions represents the generation of improvement suggestions.

[0197] That is, the suggestion generation module can generate improvement suggestions based on the inference result CF and the average processing effect ATE. The improvement suggestions can be used to provide specific action plans, aiming to improve the quality of the corpus by changing certain factors.

[0198] It should be noted that this embodiment may also be an improvement based on any one or more of the first to sixth embodiments.

[0199] Ninth embodiment

[0200] The ninth embodiment of the present application relates to a corpus quality assessment system. The ninth embodiment is an improvement on the eighth embodiment. The specific improvement is that: in the ninth embodiment of the present application, a specific implementation method of the lifelong learning module is provided, such as Figure 6 shown.

[0201] Specifically, in some embodiments, the lifelong learning module can continuously optimize the evaluation model during the operation of the corpus quality evaluation system. The evaluation model is the whole of all modules in the system that work together to complete the corpus quality evaluation task. In other words, the lifelong learning module is used to continuously optimize each module and adapt to new corpus types and quality standards. In this way, it can be ensured that the system can continue to learn and optimize and adapt to new corpus types and quality standards. Lifelong learning allows the system to update and improve its model as it is constantly exposed to new data, maintaining the accuracy and relevance of the quality scoring results.

[0202] Exemplarily, the lifelong learning module may adopt a preset strategy to adapt to new corpus types and quality standards. In some embodiments, the lifelong learning module may include an experience playback module, which is used to determine a data subset based on a historical data set and a current data set. Exemplarily, the following formula can be used:

[0203] D'=SampleExperience(D,Dcurrent)

[0204] Wherein, D represents the historical data set of all corpora historically processed by the system, Dcurrent represents the current data set of the new corpus currently input, and D' represents the data subset sampled from the historical data set and the current data set;

[0205] That is, the experience replay module can sample historical data from the historical data set D and the current data set Dcurrent to form a data subset D'. In this way, the experience replay module can enable the system to revisit and learn past experience to improve the adaptability and generalization ability of the model to new tasks.

[0206] Furthermore, in some embodiments, the lifelong learning module may include a knowledge distillation module, which is used to receive the data subset D' sampled by the experience playback module, use the teacher model (trained evaluation model) to guide the student model (new evaluation model) to learn, and obtain the loss function of knowledge distillation. For example, the following formula can be used:

[0207] L_distill=E_{x∈D'}[KL(p_teacher(x)||p_student(x))];

[0208] Wherein, the E_{x∈D'} represents the expectation on the data subset D', p_teacher represents the output distribution of the teacher model (old evaluation model) on the input x, the p_student(x) represents the output distribution of the student model (new evaluation model) on the same input x, the KL represents the KL divergence that calculates the difference between the two distributions, and the L_distill represents the loss function of knowledge distillation. It can be understood that knowledge distillation is a compression technique, in which the teacher model is usually a pre-trained, high-performance model, and the knowledge of the teacher model can be transferred to the student model. Here, the loss function L_distill of knowledge distillation measures the difference between the output distributions of the teacher model p_teacher and the student model p_student through the KL divergence, prompting the student model to learn the behavior of the teacher model.

[0209] Further, in some embodiments, the lifelong learning module may include an elastic weight integration module, which is used to determine the elastic weight integration loss function L_ewc based on the parameter θi of the current evaluation model and the previous parameter θi_prev, so as to maintain the performance of the learned task when learning a new task. For example, the following formula can be referred to:

[0210] L_ewc=Σ(λ / 2×Fi×(θi-θi_prev)^2)

[0211] Among them, λ represents a trade-off factor for balancing the importance of new and old tasks, Fi represents the Fisher information matrix (indicating the importance of parameters), θi represents the current value of the i-th parameter in the evaluation model, which is a component of the aforementioned overall parameter θ, θi_prev represents the previous value of the parameter θi; Σ represents a summation symbol, and L_ewc represents the loss function of elastic weight integration.

[0212] That is, the elastic weight integration module can dynamically adjust the importance of model parameters through the Fisher information matrix Fi and the trade-off factor λ. The elastic weight integration loss function L_ewc penalizes the difference between the current model parameter θi and its previous value θi_prev, which helps to retain previously learned knowledge while learning new tasks.

[0213] Furthermore, in some embodiments, the lifelong learning module may include a dynamic expansion module; the dynamic expansion module is used to receive the output of each layer, and determine the hidden representation h_l of the current layer by combining the current layer output h_l-1 and the historical network layer output hl-1(prev), so as to realize the dynamic expansion of the network capacity. For example, the following formula can be referred to:

[0214] h_l=[h_l-1,ReLU(Wl[h_l-1;hl-1(prev)])]

[0215] Wherein, the h_l represents the hidden representation of the current layer, the h_l-1 represents the output of the previous layer, the hl-1(prev) represents the output of the corresponding layer in the previously trained network, Wl represents the learnable weight matrix of the current layer, the ReLU represents the activation function, and [;] represents the vector concatenation operation. It can be understood that the progressive neural network is a technology that can dynamically expand the capacity of the network. In this embodiment, the hidden representation h_l of the current layer is updated by adding new neurons and the learnable weight matrix Wl of the current layer, which allows the model to increase complexity when necessary to adapt to more complex tasks.

[0216] It can be seen that the lifelong learning module continuously optimizes the feature extraction module, knowledge graph construction module, multi-head self-attention fusion module, logical reasoning module, multi-task quality scoring module and causal inference module in the corpus quality assessment system through the collaborative work of the experience replay module, knowledge distillation module, elastic weight integration module and dynamic expansion module, so that the system can continuously learn and adapt to new evaluation needs.

[0217] It should be noted that this embodiment may also be an improvement based on any one or more of the first to seventh embodiments.

[0218] In related technologies, traditional model update methods mainly rely on regular full retraining, which is not only computationally expensive but also inefficient. Although incremental learning methods can integrate new data, they are prone to losing previously acquired knowledge while absorbing new knowledge, which is catastrophic forgetting. Although model integration methods can improve performance by combining multiple dedicated models, this increases the complexity of the system and makes maintenance more difficult. These methods are difficult to achieve continuous accumulation and optimization of model capabilities, and therefore difficult to adapt to rapidly changing application scenarios.

[0219] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0220] This system has achieved significant innovations, mainly in the following aspects:

[0221] The important breakthrough from relying on static rules to dynamic adaptation has enhanced the collaborative working ability between modules.

[0222] An end-to-end automated evaluation framework has been established, which can significantly reduce manual involvement and improve the level of automation in evaluation.

[0223] The system has the ability of autonomous learning and continuous optimization, and can adapt to changing evaluation needs.

[0224] Through multi-level explainability design, clear evaluation basis and improvement suggestions are provided, which can enhance the transparency of the evaluation.

[0225] These innovations have significantly improved the accuracy, efficiency, and scalability of the system, bringing new technological advances to the field of corpus quality assessment.

[0226] This application has at least the following technical effects:

[0227] Qualitative leap in quality assessment efficiency: This application implements a fully automated processing mechanism based on neural symbolic fusion technology, which makes a qualitative leap in quality assessment efficiency. The system can work 24 hours a day without interruption, while processing massive corpus, can meet the needs of fast iteration scenarios, and generate assessment results in real time. This automation innovation significantly reduces labor costs and time costs, and fundamentally solves the problem of low efficiency of traditional manual evaluation.

[0228] The accuracy and consistency of evaluation have been significantly improved: Through the synergy of multimodal knowledge graphs and differentiable neural logic reasoning, this application can successfully overcome the subjective bias of manual evaluation and ensure the objectivity and uniformity of evaluation standards. The evaluation results generated by the system are reproducible, which can significantly improve the reliability of the evaluation and solve the problem of inconsistent evaluation standards in traditional methods.

[0229] The interpretability of the evaluation results has been fundamentally improved: This application uses an innovatively designed causal inference module to provide clear evaluation basis and reasoning process, and automatically generates targeted improvement suggestions. This interpretable design not only helps users understand the root cause of quality problems, but also effectively guides the continuous optimization of corpus quality, thereby overcoming the black box problem of traditional deep learning methods.

[0230] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0231] The scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used to distinguish the description, and do not indicate any particular order, nor can they be understood as indicating or implying relative importance.

[0232] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A corpus quality assessment system, characterized in that: The system comprises: A feature extraction module is used to determine the explicit rule features and implicit deep features of the neural symbol based on the input corpus; A knowledge graph construction module, used to construct a knowledge graph of the corpus based on the explicit rule features and implicit deep features; A multi-head self-attention fusion module, used to determine fusion features according to the knowledge graph; A logic reasoning module, used for performing rule reasoning according to the fusion features to obtain a reasoning result; The multi-task quality scoring module is used to generate a multi-dimensional quality score according to the reasoning result.

2. The system according to claim 1, characterized in that The system further comprises: a causal inference module and / or a lifelong learning module; The causal inference module is used to analyze influencing factors and generate improvement suggestions based on the quality score; The lifelong learning module is used to continuously optimize the corpus quality assessment system.

3. The system according to claim 2, characterized in that The feature extraction module includes a first extraction module, a second extraction module and a processing module; The first extraction module is used to determine implicit deep features based on the corpus and the pre-trained language model; The second extraction module is used to determine explicit rule features based on the corpus, background knowledge and neural symbolic program induction; The processing module is used to determine structured features based on the function used to convert text into a graph structure, the graph neural network and the corpus.

4. The system according to claim 3, characterized in that The processing module includes a construction unit, a learning unit, a level representation unit and a structure quality score calculation unit; The construction unit is used to construct a heterogeneous graph representation of the corpus; The learning unit is used to perform node representation learning based on the heterogeneous graph representation using a relation-aware graph attention network; The level representation unit is used to fuse representations of different levels based on the heterogeneous graph representation using a dynamic gating mechanism to obtain a final representation; The structure quality score calculation unit is used to calculate the structure quality score according to the final representation, and the structure quality score is used to characterize the structured feature.

5. The system according to claim 3, characterized in that The knowledge graph construction module includes a collection extraction module, a link establishment module, a fusion construction module and an update module; The set extraction module is used to determine the entity set and the relationship set according to the implicit deep features, the named entity recognition function and the relationship extraction function; The link establishment module is used to determine the target entity set after the link is established according to the entity set, the external knowledge base, the explicit rule features and the entity link function; The fusion construction module is used to construct a multimodal knowledge graph according to the target entity set, the relationship set, the visual features, the audio features, the structural features and the graph fusion function; The updating module is used to determine the node representation in the knowledge graph according to the knowledge graph and the graph conversion network model.

6. The system according to claim 5, characterized in that The multi-head self-attention fusion module includes a preprocessing module, a calculation module, a multi-head attention output module and a fusion feature output module; The preprocessing module is used to preprocess the implicit deep features, explicit rule features and structured features output by the feature extraction module to obtain a splicing result of neural symbol features; The calculation module is used to calculate multi-head self-attention according to the splicing result; The multi-head attention output module is used to perform splicing according to the number of attention heads and the linear transformation matrix to determine the output result; The fusion feature output module is used to determine the final fusion feature based on the output result, the node representation in the knowledge graph, the feedforward neural network and the layer normalization processing function.

7. The system according to claim 6, characterized in that The logic reasoning module includes a conversion module, a soft logic reasoning module and a weight updating module: The conversion module is used to convert the symbolic rule in the explicit rule feature into a tensor rule; The soft logic reasoning module is used to perform soft logic reasoning according to the tensor rule, the fusion feature and the weight information used to characterize the importance of the symbol rule, and determine the soft logic reasoning result; The weight updating module is used to update the weight information according to the weight information, learning rate and loss function to obtain target weight information.

8. The system according to claim 7, characterized in that The multi-task quality scoring module includes a task distribution module, a meta-learning optimization module, a task adaptation module and a scoring module: The task distribution module is used to determine the task distribution according to the quality assessment subtasks and the total number of the quality assessment subtasks; The meta-learning optimization module is used to determine the optimal model parameters obtained by minimizing the expected loss in multiple evaluation dimensions according to the task distribution and the soft logic reasoning results; The task adaptation module is used to determine the task target parameters according to the optimal model parameters; The scoring module is used to determine a scoring result based on the task target parameters and the total number of quality dimensions.

9. The system according to claim 8, characterized in that The causal inference module includes a model building module, a counterfactual reasoning module, a causal effect calculation module and a suggestion generation module; The model building module is used to determine a structured causal model according to the fusion feature, the explicit rule feature and the scoring result; The counterfactual reasoning module is used to determine the reasoning result of counterfactual reasoning under a given intervention according to the causal model; The causal effect calculation module is used to determine the average treatment effect based on the intervened corpus feature variables and quality score variables; The suggestion generating module is used to generate improvement suggestions according to the inference result and the average processing effect.

10. The system according to any one of claims 2 to 9, characterized in that The lifelong learning module includes an experience replay module, a knowledge distillation module, an elastic weight integration module and a dynamic expansion module; The experience playback module is used to sample and obtain a data subset based on the historical data set and the current data set; The knowledge distillation module uses the teacher model to guide the student model to learn according to the data subset, and obtains the loss function of knowledge distillation; The elastic weight integration module determines a loss function of elastic weight integration based on the parameters of the current evaluation model and the previous parameters, so as to maintain the performance of the learned task when learning a new task; The dynamic expansion module receives the output of each layer, determines the hidden representation of the current layer by combining the current layer output and the historical network layer output, and realizes the dynamic expansion of the network capacity.

Images