A knowledge graph construction method and system based on a large language model

Abstract

The application relates to a kind of knowledge graph construction method and system based on large language model, through multimodal data fusion, reinforcement learning and contrast learning joint optimization, teacher-student model knowledge migration, time sequence dynamic analysis and incremental updating mechanism, realize the automatic construction and dynamic maintenance of knowledge graph.Construct reinforcement learning framework, with accuracy, integrity as reward index training large model extraction entity relationship;Large-scale knowledge graph is used as teacher model, through graph attention network, realize the conflict resolution and semantic alignment of new knowledge and existing graph;Based on historical data and field knowledge, dynamically generate verification rules, combine fourier transform and CUSUM algorithm to detect relationship periodicity and mutation point;Finally, through incremental updating and version control, generate traceable knowledge graph.Improve the precision of multimodal data processing, entity relationship extraction dynamic adaptability and knowledge graph maintenance efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a method and system for constructing a knowledge graph based on a large language model. Background Art

[0002] Knowledge graphs, a core technology in the field of artificial intelligence, aim to represent real-world entities and their relationships in a structured manner. They are widely used in scenarios such as intelligent question answering, recommendation systems, and decision analysis. However, traditional knowledge graph construction methods face significant challenges in multimodal data fusion, dynamic relationship extraction, and knowledge verification and updating.

[0003] Existing knowledge graph construction mostly relies on single text data, making it difficult to effectively integrate semantic information from unstructured data such as images and audio. Traditional early feature splicing or simple weighted multimodal fusion methods fail to fully capture the semantic associations between different modalities, resulting in strong heterogeneity in the feature space and low fusion accuracy. For example, there is a lack of hierarchical extraction of local and global features of images, the acoustic features of audio are difficult to accurately align with the semantic content, and there is a semantic gap between the contextual semantics of text and the cross-modal mapping of non-text modalities. In addition, the means of filtering abnormal data and eliminating redundant features are single, making it difficult to deal with noise interference and information redundancy in multimodal data.

[0004] Traditional entity relationship extraction methods based on rule-based template matching or statistical learning models rely on manually defined features or annotated data, making them difficult to adapt to the dynamic changes in entity relationships in open domain scenarios. For example, models lack the ability to distinguish between pairs of semantically similar entities or ambiguous relationships in a fine-grained manner. In time series data, traditional methods are unable to capture the cyclical patterns of relationships. While reinforcement learning and contrastive learning have been gradually introduced and applied to natural language processing, they have yet to form a systematic framework for knowledge graph construction. In particular, there is a lack of reward mechanisms designed to comprehensively target accuracy, completeness, and extraction efficiency.

[0005] Existing knowledge graph fusion technologies often use static mapping strategies, making them difficult to address semantic conflicts and structural heterogeneity during large-scale graph updates. For example, there is a lack of dynamic weight adjustment mechanisms for entity ambiguity and relationship inconsistencies between newly added entity relationships and existing graphs. Traditional graph neural networks ignore the impact of temporal dynamics on entity relationship weights during the fusion process, resulting in insufficient semantic coherence in the fused graph. While the teacher-student model architecture can be used for knowledge transfer, effectively transferring the distributional characteristics of large-scale pre-trained knowledge graphs to newly added knowledge still requires the design of more optimized alignment loss functions and attention mechanisms.

[0006] Therefore, there is an urgent need to improve the existing knowledge graph construction process to enhance semantic understanding capabilities, the adaptive optimization mechanism of reinforcement learning, the dynamic modeling capabilities of graph neural networks, and the anomaly detection capabilities of time series analysis algorithms, so as to form a fully automated and dynamic knowledge graph construction framework. Summary of the Invention

[0007] The purpose of the present invention is to provide a knowledge graph construction method and system based on a large language model, so as to enhance the semantic understanding ability, the adaptive optimization mechanism of reinforcement learning, the dynamic modeling ability of graph neural networks, and the anomaly detection ability of time series analysis algorithms, and form a fully automated and dynamic knowledge graph construction framework.

[0008] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0009] In a first aspect, a method for constructing a knowledge graph based on a large language model is provided, comprising the following steps:

[0010] S1: Acquire multimodal data of text, images, and audio, convert the image and audio data into semantic vectors, fuse them with the text data, and process the fused data;

[0011] S2: Build a reinforcement learning framework, using the large model as an intelligent agent and entity relationship extraction accuracy and completeness as reward indicators. By interacting with the data environment, we learn the optimal extraction strategy. We use contrastive learning methods to enhance the large language model's ability to distinguish similar entities and relationships. We extract entities and relationships from preprocessed data to generate entity relationship candidate sets.

[0012] S3: A large-scale knowledge graph is pre-set as the teacher model, and the new entity relationship knowledge to be integrated is used as the student model. The student model learns the knowledge distribution of the teacher model, integrates the newly extracted entity relationships with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights during the integration process to achieve conflict resolution and semantic alignment.

[0013] S4: Leveraging a large language model based on the historical update data and domain knowledge of the knowledge graph, dynamically generates knowledge verification rules, analyzes the evolution trend of entity relationships in the knowledge graph, identifies potential knowledge errors and omissions, and enhances the optimized knowledge graph to supplement missing relationships and correct erroneous relationships.

[0014] S5: Update the optimized knowledge representation to the knowledge graph storage system in an incremental manner, and combine it with the version control mechanism to generate the final knowledge graph with historical traceability capabilities.

[0015] Preferably, the multi-modal data of text, image and audio obtained in step S1 is converted into semantic vectors in the following specific process:

[0016] S11: A neural network model is constructed to extract local features and global features of the obtained image data, and a high-dimensional image feature vector is obtained;

[0017] S12: The audio data is converted to a specified sampling rate, the amplitude range thereof is scaled to a preset interval, the audio frame length and frame shift are set to form an overlapping frame structure, a windowing function is set to reduce spectral leakage, and the fast Fourier transform is performed on each windowed signal to convert it to the frequency domain to obtain an audio amplitude spectrum. A triangular filter is constructed in a preset frequency range to perform frequency conversion, extract acoustic features of the audio, and construct an audio feature vector;

[0018] S13: A shared semantic space is constructed, and a mapping layer is constructed by training a fully connected neural network to map the image feature vector and the audio feature vector into the shared semantic space, respectively. The vectors of the image data and the audio data in the shared semantic space are the semantic vectors.

[0019] Before mapping, the network parameters are optimized by training the paired data of image-text and audio-text, so that the vector distance of images, audios and texts with similar semantics in the semantic space is as small as possible.

[0020] Preferably, the text data is fused in step S1, and the fused data is processed in the following specific process:

[0021] S14: The text vector is obtained by word embedding from the text data, the dimensions of the semantic vector and the text vector of the image data and the audio data are obtained, and the semantic vector and the text vector are mapped to a preset dimension by a mapping layer of a fully connected neural network;

[0022] S15: A weight prediction model of a multi-layer perceptron is trained, the weights are set according to the importance of different modal data, the accuracy of entity relation extraction is taken as the optimization target, the weights of the image, audio and text vectors are learned, and the fusion feature vector is obtained based on the weights;

[0023] S16: The density of the fusion feature vector in the feature space is calculated, the vector with a density lower than a threshold value is regarded as an abnormal value, the neighborhood radius and the minimum sample number are set, and if the sample number in the neighborhood with the vector as the center and the neighborhood radius as the radius is less than the minimum sample number, the vector is determined as an abnormal value and removed;

[0024] S17: The cosine similarity between the fusion feature vectors is calculated, a similarity threshold is set, and if the cosine similarity between two fusion feature vectors is greater than the threshold, the vector with richer information is retained and the redundant vector is deleted.

[0025] S18: Use the large language model to perform semantic analysis on the fused feature vector, input the multimodal data description corresponding to the fused feature vector, and determine whether the semantics are consistent. If the semantic consistency score output by the large language model is lower than the set threshold, the fused feature vector is corrected.

[0026] Preferably, the specific process of step S2 is as follows:

[0027] S21: converting the fused feature vector into an input format that can be processed by the large model, and concatenating the converted fused feature vector with the corresponding text description as the current state of the environment;

[0028] S22: Define the action space as all possible entity relationship extraction results. An action is an entity pair and its relationship identified from the data. The large model represents the action by generating a corresponding text sequence. The large language model is used as an intelligent agent. The intelligent agent receives the current state of the environment, generates an action based on internal parameters, and executes the action in the environment.

[0029] S23: Calculate the degree of match between the entity relationships extracted by the agent and the correct entity relationships in the annotated data, and give corresponding rewards or penalties based on the degree of match; if the extracted entity relationships are completely correct, a higher reward is given; if there are partial errors or complete errors, a lower reward or penalty is given based on the degree of error;

[0030] Measure the extent to which the extraction results cover all entity relationships in the labeled data, and give corresponding rewards or penalties based on the coverage;

[0031] If all correct entity relationships are extracted, rewards are given; if there are relationships that are not extracted, penalties are given according to the number of missing relationships.

[0032] S24: Dynamically adjust the weights of accuracy and completeness rewards based on task requirements and data characteristics, while also taking extraction speed into consideration. Speed ​​rewards are awarded if extraction is completed in a short time while ensuring accuracy and completeness.

[0033] S25: Based on the current strategy, the agent generates and executes actions according to the environment state. The environment calculates rewards based on the labeled data and reward function, and feeds back the next environment state to the agent.

[0034] S26: Use the policy gradient algorithm to update the large model parameters based on the reward and state transition information, with the goal of maximizing the long-term cumulative reward;

[0035] S27: Select sample pairs containing similar entities or relations from the preprocessed data, divide them into positive sample pairs with entity relations of the same or similar semantics and negative sample pairs with entity relations of different semantics, input the sample pairs into the large language model, obtain the feature representation of its output, and optimize it using the contrastive learning loss function;

[0036] S28: Input the new preprocessed data into the large language model that has been trained by reinforcement learning and optimized by contrastive learning, extract entity relationships according to the learned optimal strategy, generate multiple possible entity relationship results, filter the extraction results according to a certain confidence threshold, retain entity relationships with higher confidence, and form the final entity relationship candidate set.

[0037] Preferably, the specific process of step S3 is as follows:

[0038] S31: Map entities and relationships to a low-dimensional vector space by presetting a large-scale knowledge graph. Use the student model to represent newly extracted entity relationships using a vector space of the same dimension. Initialize the representation using random embeddings or a pre-trained language model.

[0039] S32: Construct a mapping matrix and learn alignment parameters by optimizing a preset objective function. The teacher model generates a soft label distribution for each triple in the student model and incorporates hard label loss and soft label loss.

[0040] S33: Construct a graph attention network, concatenate entity vectors and relationship vectors as node initial features, calculate multi-head attention scores, and adjust knowledge fusion weights through a gating mechanism;

[0041] S34: Calculate the entity similarity matrix. When the similarity exceeds the threshold and the relationship is inconsistent, mark it as a conflict. Construct an alignment loss function, adjust the fusion weight of the conflicting entities based on the attention score, evaluate the knowledge graph completion performance, evaluate the entity alignment accuracy, and use the semantic similarity distribution to evaluate the fusion quality.

[0042] Preferably, the specific process of step S4 is as follows:

[0043] S41: Sort the historical update data of the knowledge graph by timestamp, construct the entity relationship evolution sequence, calculate the difference between adjacent snapshots, extract the added / deleted entity relationships, and form a change pattern library;

[0044] S42: Encode domain knowledge into ontology constraints, generate templates based on preset design rules, input a small number of rule examples into the large model to guide it to generate new rules, quickly adapt to the rule generation needs of different domains through meta-learning, and adjust model parameters to minimize rule generation errors;

[0045] S43: Capture the temporal dynamics of the knowledge graph, map entity relationships into time-varying embedding vectors, use Fourier transform to detect the periodicity of relationship occurrences, and identify relationship mutation points using a preset CUSUM algorithm;

[0046] S44: Determine the causal dependencies between entity relationships, simulate the potential impact of relationship changes, convert the generated verification rules into Datalog queries, determine the semantic rationality of entity relationships, combine the TransE knowledge graph embedding model with the text features generated by the large model, predict missing relationships, use the reasoning ability of the large model to complete implicit knowledge, select the optimal correction solution based on confidence, update the knowledge graph, and combine the link prediction results, knowledge generated by the large model, and feedback from domain experts to construct a supplementary candidate set.

[0047] Preferably, in step S43, the temporal dynamics of the knowledge graph are captured, entity relationships are mapped into time-varying embedding vectors, Fourier transform is used to detect the periodicity of relationship appearance, and the specific process of detecting relationship mutation points by using a preset CUSUM algorithm is as follows:

[0048] S431: Represent the knowledge graph as a set of time series triples, divide the time window, and discretize the continuous time;

[0049] S432: For entities and relationships Introducing time-dependent embedding vectors;

[0050] S433: For each relationship, extract its embedding vector at each time point, or calculate the interaction strength of the associated entity pair;

[0051] S434: De-meaning and normalizing the time series, calculating the discrete Fourier transform, and taking the modulus to obtain the amplitude spectrum and frequency resolution;

[0052] S435: Filter low-frequency noise, retain high-frequency components, set amplitude thresholds, extract significant frequencies, draw amplitude spectrograms, and mark the periods corresponding to significant frequencies;

[0053] S436: Setting the normalized timing value, presetting the reference value and offset, and calculating the forward cumulative sum and the backward cumulative sum;

[0054] S437: Preset a control threshold. When the forward cumulative sum and the backward cumulative sum exceed the control threshold, a sudden change alarm is triggered and parameters are initialized.

[0055] S438: When the cumulative sum exceeds the control threshold, it is determined to be an upward mutation;

[0056] When the backward cumulative sum exceeds the control threshold, it is determined to be a downward mutation, the mutation time is recorded, and the cumulative sum is reset to 0 to detect subsequent mutations.

[0057] Preferably, the specific process of step S5 is as follows:

[0058] S51: Define the change operation set {CREATE, UPDATE, DELETE, MERGE}, where CREATE, UPDATE, DELETE, and MERGE correspond to entity / relationship creation, update, deletion, and merge operations respectively;

[0059] S52: Use graph edit distance-based difference detection to calculate the difference between the optimized knowledge representation and the existing knowledge graph;

[0060] S53: Serialize change operations into JSON-LD objects with timestamps, use Protobuf to serialize large-scale change data, and use gzip compression to reduce transmission overhead;

[0061] S54: Automatically generates a new version after an incremental update is committed, records the change summary and diff hash, implements rollback by reverse-applying the historical diff, uses Cypher statements to query the graph state at a certain point in time, and visualizes the differences between the new version and the historical version;

[0062] Record key operations including entity creation, relationship modification, and version merging, store them in an independent audit log library, and implement fine-grained permission management for access to historical versions;

[0063] Set an incremental storage strategy, bucket and store incremental data by time window, and create a tree index for entity ID and timestamp.

[0064] In a second aspect, a knowledge graph construction system based on a large language model is provided, which is used to implement the knowledge graph construction method based on a large language model, including a data acquisition module, a data preprocessing module, a learning framework construction module, a dynamic adjustment module, an update enhancement module, and a knowledge graph generation module;

[0065] The data acquisition module is used to acquire multimodal data such as text, image, and audio;

[0066] The data preprocessing module is used to convert image and audio data into semantic vectors, fuse them with text data, and process the fused data;

[0067] The learning framework construction module is used to build a reinforcement learning framework, using the large model as an intelligent agent and the accuracy and completeness of entity relationship extraction as reward indicators. The optimal extraction strategy is learned by interacting with the data environment, and the contrastive learning method is used to enhance the large language model's ability to distinguish similar entities and relationships. Entities and relationships are extracted from preprocessed data to generate entity relationship candidate sets;

[0068] The dynamic adjustment module is used to preset a large-scale knowledge graph as a teacher model and use the new entity relationship knowledge to be integrated as a student model. The student model learns the knowledge distribution of the teacher model, integrates the newly extracted entity relationships with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights during the integration process to achieve conflict resolution and semantic alignment.

[0069] The update enhancement module is used to dynamically generate knowledge verification rules based on the historical update data and domain knowledge of the knowledge graph using the large language model, analyze the evolution trend of entity relationships in the knowledge graph, discover potential knowledge errors and omissions, enhance the optimized knowledge graph, supplement missing relationships, and correct erroneous relationships;

[0070] The knowledge graph generation module is used to update the optimized knowledge representation to the knowledge graph storage system in an incremental manner, and combine with the version control mechanism to generate a final knowledge graph with historical traceability capabilities.

[0071] The beneficial effects of the present invention include:

[0072] The present invention provides a knowledge graph construction method and system based on a large language model, which realizes the automatic construction and dynamic maintenance of the knowledge graph through multimodal data fusion, joint optimization of reinforcement learning and contrastive learning, teacher-student model knowledge transfer, time series dynamic analysis and incremental update mechanism. A reinforcement learning framework is constructed to train a large model to extract entity relationships with accuracy and completeness as reward indicators; a large-scale knowledge graph is used as a teacher model to achieve conflict resolution and semantic alignment between new knowledge and existing graphs through a graph attention network; verification rules are dynamically generated based on historical data and domain knowledge, and Fourier transform and CUSUM algorithm are combined to detect relationship periodicity and mutation points; finally, a traceable knowledge graph is generated through incremental update and version control. This improves the accuracy of multimodal data processing, the dynamic adaptability of entity relationship extraction, and the efficiency of knowledge graph maintenance.

[0073] First, a neural network layer extracts local and global image features and audio acoustic features, constructing a shared semantic space to achieve cross-modal vector mapping between images, audio, and text, addressing the multimodal semantic gap in traditional methods. Furthermore, a multi-layer perceptron dynamically learns modal weights, adaptively adjusting the contributions of text, images, and audio based on task requirements, avoiding feature redundancy or omissions caused by simple weighting.

[0074] Second, we combined DBSCAN with cosine similarity to filter outliers and redundant vectors, reducing noise in multimodal data. We also eliminated low-density vectors corresponding to blurred images and merged audio features from similar scenes, improving the input quality for subsequent extraction tasks. We also used a large language model to perform semantic consistency checks, correcting semantic conflicts in cross-modal data and ensuring the logical consistency of the fused features.

[0075] Third, by using the large language model as an intelligent agent and taking accuracy, completeness, and extraction speed as multi-objective reward functions, the extraction strategy is optimized through the policy gradient algorithm. This can balance the performance requirements in different business scenarios without relying on large-scale labeled data. Through the environmental interaction mechanism of reinforcement learning, the model can learn extraction patterns from a small number of samples, making it suitable for vertical fields with high labeling costs such as medicine and law.

[0076] Fourth, the prior knowledge of the preset large-scale knowledge graph (teacher model) is used to guide the embedding learning of the new knowledge (student model), and the smooth transfer of knowledge is achieved through soft label distribution (such as the probabilistic mapping of entity relationships) to avoid the overfitting problem caused by hard label matching. The graph attention network dynamically adjusts the entity relationship weights and prioritizes the integration of high-frequency or domain-critical knowledge.

[0077] Fifth, using the entity similarity matrix and alignment loss function, we automatically detect and resolve conflicts such as homonyms and relationship contradictions. This significantly improves conflict resolution accuracy compared to traditional rule-based alignment methods. Fourier transforms capture the temporal periodicity of relationships, providing a basis for predictive analysis. The CUSUM algorithm detects relationship mutation points in real time, significantly reducing latency compared to traditional sliding window detection. Verification rules are automatically generated based on historical change data and domain knowledge, reducing the cost of manual rule writing. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 Schematic diagram of the process of constructing a knowledge graph based on a large language model of the present invention.

[0079] Figure 2 The figure is a flow chart of converting image and audio data into semantic vectors according to the present invention.

[0080] Figure 3 Schematic diagram of the reinforcement learning framework of the present invention. DETAILED DESCRIPTION

[0081] The following is combined with Figure 1-Figure 3 The present invention is described in further detail:

[0082] Example 1

[0083] See attached Figure 1 As shown, a method for constructing a knowledge graph based on a large language model includes the following steps:

[0084] S1: Acquire multimodal data of text, images, and audio, convert the image and audio data into semantic vectors, fuse them with the text data, and process the fused data;

[0085] S2: Build a reinforcement learning framework. Figure 3 As shown in the figure, the large model is used as an intelligent agent, and the accuracy and completeness of entity relationship extraction are used as reward indicators. The optimal extraction strategy is learned by interacting with the data environment. The contrastive learning method is used to enhance the large language model's ability to distinguish similar entities and relationships. Entities and relationships are extracted from preprocessed data to generate entity relationship candidate sets.

[0086] S3: A large-scale knowledge graph is pre-set as the teacher model, and the new entity relationship knowledge to be integrated is used as the student model. The student model learns the knowledge distribution of the teacher model, integrates the newly extracted entity relationships with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights during the integration process to achieve conflict resolution and semantic alignment.

[0087] S4: Leveraging a large language model based on the historical update data and domain knowledge of the knowledge graph, dynamically generates knowledge verification rules, analyzes the evolution trend of entity relationships in the knowledge graph, identifies potential knowledge errors and omissions, and enhances the optimized knowledge graph to supplement missing relationships and correct erroneous relationships.

[0088] S5: Update the optimized knowledge representation to the knowledge graph storage system in an incremental manner, and combine it with the version control mechanism to generate the final knowledge graph with historical traceability capabilities.

[0089] This example uses a commercial bank's need to build a knowledge graph for corporate credit risk management. This graph integrates multimodal information, including basic corporate information, transaction records, and guarantee relationships, to dynamically monitor the company's associated risks. Multimodal data includes corporate financial report text, business license images, relevant meeting audio, and transaction flow tables.

[0090] ResNet50 was used to extract global features from the business license image, combined with YOLOv8 to extract local text region features. This generated a specified-dimensional image feature vector. The meeting recordings were sampled at a uniform rate, divided by frame length, frame shift, and Hanning window. After Fourier transformation to the frequency domain, MFCC features were extracted to generate a specified-dimensional audio vector. Sentence vectors for the financial report text were generated with a specified dimension. Keywords such as "guarantee" and "default" were mapped to 300-dimensional vectors using Word2Vec. A fully connected layer was used to unify the image, audio, and text vectors into 512 dimensions, and modal weights were learned using a multi-layer perceptron. The feature vectors were fused and concatenated with the text description to create the environment state. Accuracy rewards: completely correctly extracting the guarantee relationship will receive a specific reward, while partial errors will be penalized accordingly. Completeness rewards: covering all associated companies will receive a specific reward, while missing the main guarantee relationship will be penalized accordingly. Speed ​​rewards: completing the extraction within a specified time will receive a specific reward, while exceeding the specified time will result in no reward. Weights were dynamically adjusted based on the reward function.

[0091] The newly extracted "Company A - Guarantee - Company B" relationship is embedded using BERT-KE. The teacher model provides soft labels and node features: the company vector and the relationship vector are concatenated, such as the company A vector + the "guarantee" relationship vector, and attention weights are calculated.

[0092] The business registration address of Enterprise A differs from the address recorded in the teacher model, triggering the entity alignment process. The entity change records are supplemented to confirm whether they are the same entity. The guarantee amount differs significantly from the industry average, generating a conflict label and submitting it for manual verification. The fused graph includes a circular guarantee subgraph, noting that the guarantee chain "Enterprise A-Enterprise B-Enterprise C" has a "high" risk rating.

[0093] A time series is generated for the monthly occurrences of the "Enterprise A - Guarantee" relationship. After normalization and Fourier transformation, the frequency and periodicity are detected, corresponding to a pattern of concentrated guarantee processing at the end of each quarter. If the number of guarantees suddenly increases at a certain point in time, and the forward cumulative sum of the CUSUM exceeds the control threshold, an alert is triggered. Based on domain knowledge, a rule is generated: "If the number of guarantees issued by a single enterprise in a single month exceeds the historical average by a specified multiple without a reasonable explanation, it is considered an anomaly," automatically marking the enterprise as a risk alert target.

[0094] The TransE model predicts missing relationships: Enterprises A and D have multiple transaction records, suggesting a potential implicit guarantee relationship. This generates candidate triples (Enterprise A, Guarantee, Enterprise D, ?). Large language model reasoning analyzes the text of Enterprise A's meeting recording, "Reaching a Credit Enhancement Agreement with Company D," to complete the relationship timeline. A new circular guarantee relationship is added, and Enterprise A's risk rating attribute is updated (from "Medium" to "High"). Serialization is changed to timestamped JSON-LD. Bucketed storage: Buckets are bucketed by specified time points. Incremental data is compressed and stored as Parquet files, resulting in a smaller size than a full update. Querying the difference between the graph before and after the specified time point visualizes the newly added guarantee chain relationship. RBAC restricts permissions, limiting access to risk level change records to only the risk control department, while the audit department can trace all change logs from historical versions.

[0095] Example 2

[0096] Based on Example 1, see Figure 2 As shown, in step S1, multimodal data of text, image, and audio are obtained, and the specific process of converting image and audio data into semantic vectors is as follows:

[0097] S11: Construct a neural network model to extract local features and global features of the acquired image data to obtain a high-dimensional image feature vector;

[0098] S12: Convert the audio data to a specified sampling rate, scale its amplitude range to a preset range, set the audio frame length and frame shift to form an overlapping frame structure, set a windowing function to reduce spectrum leakage, perform a fast Fourier transform on the windowed signal per frame to convert it into the frequency domain to obtain an audio amplitude spectrum, construct a triangular filter to perform frequency conversion within a preset frequency range, extract the acoustic features of the audio, and construct an audio feature vector;

[0099] S13: Constructing a shared semantic space, mapping the image feature vector and the audio feature vector to the shared semantic space by training a fully connected neural network to construct a mapping layer, where the vectors of the image data and the audio data in the shared semantic space are semantic vectors;

[0100] Before mapping, the network is trained using image-text and audio-text paired data to optimize network parameters so that the vector distance between semantically similar images, audio, and text is as small as possible in the semantic space.

[0101] The specific process of fusing the data with the text data in step S1 and processing the fused data is as follows:

[0102] S14: Obtain text vectors for the text data by word embedding, obtain the dimensions of the semantic vectors and text vectors of the image data and audio data, and map the semantic vectors and text vectors to preset dimensions through a mapping layer of a fully connected neural network;

[0103] S15: Train a multi-layer perceptron weight prediction model, setting weights based on the importance of different modal data. Optimize the task goal of entity relationship extraction accuracy, learn the weights of image, audio, and text vectors, and perform feature fusion based on the weights to obtain a fused feature vector.

[0104] S16: Calculate the density of the fused feature vector in the feature space, consider vectors with a density lower than the threshold as outliers, set the neighborhood radius ϵ and the minimum number of samples MinPts, and if the number of samples of a vector in the neighborhood with a radius of ϵ, which is centered on it, is less than MinPts, it is considered an outlier and removed;

[0105] S17: Calculate the similarity between the fused feature vectors using cosine similarity, set a similarity threshold, and if the cosine similarity of the two fused feature vectors is greater than the threshold, retain the vector with richer information and delete the redundant vector;

[0106] S18: Use the large language model to perform semantic analysis on the fused feature vector, input the multimodal data description corresponding to the fused feature vector, and determine whether the semantics are consistent. If the semantic consistency score output by the large language model is lower than the set threshold, the fused feature vector is corrected.

[0107] Faster R-CNN was used as a local feature extractor to identify key areas in business licenses, including the company name, registered capital, and registration number, and output a 2048-dimensional local feature vector. ResNet101 was used as a global feature extractor to capture the overall image layout, including the document's border structure and field arrangement, and output a 2048-dimensional global feature vector. Local and global features were concatenated to generate a 4096-dimensional high-dimensional image feature vector.

[0108] Taking a business license image as input, Faster R-CNN locates the "Registered Capital" field and extracts the visual features of the text "10 million yuan." ResNet101 identifies the global structural features of the document type as "business license." Sampling rate conversion: Downsampling the conference recording from 44.1kHz to 16kHz reduces computational complexity. Amplitude scaling: Normalizes the audio waveform amplitude to the range [-1, 1] to prevent overflow. Framing: Sets the frame length to 512ms (8192 samples) and the frame shift to 256ms with 50% overlap to form a continuous audio frame sequence. Windowing: Applying a Hanning window to each frame reduces spectral leakage.

[0109] Feature extraction: Each frame of the windowed signal is subjected to a 512-point Fourier transform to obtain a 257-dimensional frequency domain amplitude spectrum, retaining the positive frequency portion. 40 triangular filters are set in the 0-8kHz frequency range to convert the linear frequency domain to the Mel-frequency domain and extract 40-dimensional Mel-frequency cepstral coefficients. The first-order and second-order differences of the Mel-frequency cepstral coefficients are calculated to form a 120-dimensional audio feature vector.

[0110] We collected image-text and audio-text pairs, constructing 100,000 annotated pairs. For example, image-text pairs were mapped using a fully connected neural network (4096-dimensional input, 2048-dimensional hidden layers, and 512-dimensional output). The loss function was the cosine distance between the image-text pairs. For audio mapping, we used a fully connected neural network (120-dimensional input, 512-dimensional hidden layers, and 512-dimensional output). The loss function was the cosine distance between the audio-text pairs. We simultaneously trained the semantic alignment of images, audio, and text using a cross-entropy loss function, ensuring that the cross-modal vector distance for similar semantics was less than 0.2. For example, the vector distances between the image seal, audio, and text for the word "guarantee" were all less than 0.2. For example, the business license image was mapped to a semantic vector [0.6, -0.3, 0.4, ...] (512-dimensional), which is close to the text vectors for words like "enterprise" and "registration." For audio clips of "guarantee," the semantic vector was mapped to a semantic vector [-0.2, 0.5, 0.3, ...], which is close to the text vectors for words like "guarantee" and "agreement."

[0111] The financial report text is encoded to generate a 768-dimensional sentence vector. The image / audio semantic vector (512-dimensional) and text vector (768-dimensional) are uniformly mapped to 300 dimensions through a fully connected layer. In the multi-layer perceptron structure, the input is 300×3 dimensions (image + audio + text vector concatenation), the hidden layer is 128 dimensions, and the output is 3-dimensional weight ( w img , w audio , w text ), guided by the accuracy of entity relationship extraction, the weights are learned through the cross entropy loss function. In the guarantee relationship extraction, the weight of text data w text =0.6, image data w img =0.3, audio data w audio =0.1. Fusion formula: v fusion = w img · v img + w audio · v audio + wtext · v text .

[0112] Parameter settings: Neighborhood radius ϵ = 0.5, minimum number of samples MinPts = 3, threshold setting: Cosine similarity > 0.9 is considered redundant. For multiple records of the same enterprise, vectors containing complete information such as "guarantee amount" and "guarantee period" are retained, and simplified vectors containing only the enterprise name are deleted. The multimodal description corresponding to the fused vector is "The business license image shows that Enterprise A has a registered capital of 10 million yuan, the audio recording mentions guaranteeing Enterprise B, and the text describes Enterprise A investing in Enterprise C." Semantic consistency check: The large language model analysis found a semantic conflict between "guarantee" and "investment" and output a consistency score of 0.4 (threshold 0.7), triggering a correction process: MFCC features of the audio clip were re-extracted to confirm that the keyword is "guarantee" rather than "investment"; the relationship type in the text vector was corrected, and the fused feature vector was updated.

[0113] Example 3

[0114] The specific process of step S2 based on Example 1 or Example 2 is as follows:

[0115] S21: Convert the fused feature vector into an input format that can be processed by the large model, and concatenate the converted fused feature vector with the corresponding text description as the current state of the environment s t ,in t represents the time step;

[0116] S22: Define the action space A as all possible entity relationship extraction results, action a t To identify an entity pair and its relationship from the data, the large model generates a corresponding text sequence to represent the action, and the large language model is used as an intelligent agent, which receives the current state of the environment. s t , generating actions based on internal parameters a t , and perform the action in the environment;

[0117] S23: Calculate the degree of match between the entity relationship extracted by the agent and the correct entity relationship in the labeled data, and give corresponding rewards or penalties based on the degree of match; if the extracted entity relationship is completely correct, a higher reward will be given r + acc If there are partial or complete errors, a lower reward or penalty will be given according to the degree of error. r − acc ;

[0118] Measure the extent to which the extraction results cover all entity relationships in the labeled data, and give corresponding rewards or penalties based on the coverage;

[0119] If all correct entity relationships are extracted, rewards are given r + comp ; If there are any relations that are not extracted, a penalty will be imposed based on the number of missing r comp ;

[0120] S24: Dynamically adjust the weights of accuracy and completeness rewards based on task requirements and data characteristics oh acc 、 oh comp , while considering the extraction speed factor, if the extraction is completed in a short time and the accuracy and completeness are guaranteed, a speed reward will be given r + speed , the final reward function:

[0121] R=ω acc ·r + acc +oh comp ·r + comp +oh speed ·r + speed ;

[0122] S25: Agent based on current strategy π θ ( a t ∣ s t ), i is a large model parameter, according to the environmental state s t Generate Action a t And execute, the environment calculates the reward based on the labeled data and reward function R t , and the next environment state s t+1 Feedback to the agent;

[0123] S26: Using the policy gradient algorithm, update the large model parameters θ according to the reward Rt and state transition information, with the goal of maximizing the long-term cumulative reward G t =∑k=0 gamma k R t+k where gamma is a discount factor to balance short-term and long-term rewards, k is the offset of the time step, indicating the number of steps from the time step t to the start, R t+k represents the immediate reward obtained by the agent at t+k time step;

[0124] S27: Select sample pairs containing similar entities or relationships from preprocessed data, divide them into positive sample pairs with the same or similar semantic entity relationships and negative sample pairs with different semantic entity relationships, input the sample pairs into the large language model, obtain the feature representation of the output, and optimize it through the contrastive learning loss function. The formula of the contrastive learning loss function is as follows:

[0125] ;

[0126] where h i , h j + are the feature representations of the positive sample pairs, h j k is the feature representation of the negative sample pair, sim is a similarity calculation function, t is a temperature parameter for adjusting the difficulty of contrast;

[0127] S28: Input the new preprocessed data into the large language model trained by reinforcement learning and optimized by contrastive learning, and perform entity relationship extraction according to the learned optimal strategy to generate multiple possible entity relationship results. According to a certain confidence threshold, the extraction results are screened, and the entity relationships with higher confidence are retained to form the final entity relationship candidate set.

[0128] The specific process of step S3 is as follows:

[0129] S31: Map entities and relationships to low-dimensional vector space through pre-set large-scale knowledge graph, use the same dimensional vector space to represent newly extracted entity relationships through student model, and initialize using random embedding or representation based on pre-trained language model;

[0130] S32: Construct a mapping matrix W, learn alignment parameters by optimizing a pre-set objective function, and generate soft label distribution for each triple (h, r, t) in the student model by the teacher model. And integrate hard label loss and soft label loss;

[0131] S33: Build a graph attention network, concatenate entity vectors and relationship vectors as node initial features, calculate multi-head attention scores, and adjust knowledge fusion weights through a gating mechanism;

[0132] S34: Calculate the entity similarity matrix. When the similarity exceeds the threshold and the relationship is inconsistent, mark it as a conflict. Construct an alignment loss function, adjust the fusion weight of the conflicting entities based on the attention score, evaluate the knowledge graph completion performance, evaluate the entity alignment accuracy through F1-score, and use the semantic similarity distribution to evaluate the fusion quality.

[0133] In this embodiment, the teacher model uses the TransE embedding model, with an entity / relationship vector dimension of 200. The training data contains more than 5 million guarantee relationship triplets; for example, the triplet: (Enterprise X, Guarantee, Enterprise Y, probability 0.92). The student model randomly embeds: for the new enterprises "Enterprise A" and "Enterprise B", a uniformly distributed random initialization vector in the range of [-0.1, 0.1] is used; for the enterprise "Enterprise C" with a known name, BERT-Embedding is used to generate a 768-dimensional text vector, which is mapped to 200 dimensions through a fully connected layer: estudentinit=FC(BERT(Enterprise C name)). Relationship initialization: The new relationship "Counter-guarantee" uses a random perturbation of the "Guarantee" relationship vector in the industry graph.

[0134] Entity similarity is calculated using cosine similarity, with a threshold of 0.85. Conflict scenario: The similarity between the vector of "Enterprise F" in the student model and the vector of "Enterprise F Co., Ltd." in the teacher model is 0.92, but their industries are "technology" and "manufacturing" respectively, which are marked as attribute conflicts.

[0135] Loss weight of conflicting triples: L align =∑ c∈C β c || e h mapped + e r mapped − e t mapped ||2 2 ,in β c is the attention score. The higher the attention of the conflicting node, the greater the correction weight. align Alignment loss is used to measure whether the conflicting triples satisfy the semantic alignment relationship in the mapped vector space. The goal is to correct the conflicting triples by minimizing the loss and improve the consistency of the knowledge graph. C represents the set of conflicting triples. c is a single conflicting triple in the set,e h mapped is the embedded vector of the mapped head entity, referring to the head entity h The vector representation obtained after mapping is e r mapped is the relation embedding vector after mapping, which refers to the relation r The vector representation obtained after mapping is e t mapped is the tail entity embedding vector after mapping, referring to the tail entity t The vector representation obtained after mapping is,|| · ||2 2 The square of the L2 norm is used to calculate the square of the Euclidean distance between vectors, which measures e h mapped + e r mapped and e t mapped The formula assigns different attention weights to conflicting triples. β c , focusing on correcting high-priority conflicts so that the mapped entity and relationship vectors meet the semantic alignment constraints, thereby resolving conflicts in the knowledge graph.

[0136] Example 4

[0137] On the basis of Example 1 or Example 2 or Example 3, the specific process of step S4 is as follows:

[0138] S41: Sort the historical update data of the knowledge graph by timestamp and construct the entity relationship evolution sequence {(G1, t1), (G2, t2), ..., (Gn, tn)}, where Gi represents the knowledge graph snapshot at the i-th time point. Calculate the differences between adjacent snapshots, extract the added / deleted entity relationships, and form a change pattern library;

[0139] S42: Encode domain knowledge as ontology constraints, generate templates based on preset design rules, input a small number of rule examples into the large model to guide it to generate new rules, quickly adapt to the rule generation needs of different domains through meta-learning, and adjust model parameters to minimize rule generation errors:

[0140] ;

[0141] in, I are model parameters, n is the sample size, i‌is the sample index, from 1 to n , used to identify each specific data point, r i pred‌ ‌ is the predicted value‌, the output value generated by the model based on the input data, r i true‌‌ is the true value, indicating the data point i The corresponding actual rules or results, L ‌ is the loss function‌, which is used to quantify the predicted value r i pred‌ ‌ and the true value r i true‌‌ the differences between;

[0142] Meta-learning, also known as learning to learn, is a subcategory of machine learning that trains artificial intelligence models to understand and adapt to new tasks on their own. The main goal of meta-learning is to give machines skills about how to learn.

[0143] S43: Capture the temporal dynamics of the knowledge graph, map entity relationships into time-varying embedding vectors, use Fourier transform to detect the periodicity of relationship occurrences, and identify relationship mutation points using a preset CUSUM algorithm;

[0144] S44: Determine the causal dependencies between entity relationships, simulate the potential impact of relationship changes, convert the generated verification rules into Datalog queries, determine the semantic rationality of entity relationships, combine the TransE knowledge graph embedding model with the text features generated by the large model, predict missing relationships, use the reasoning ability of the large model to complete implicit knowledge, select the optimal correction solution based on confidence, update the knowledge graph, and combine the link prediction results, knowledge generated by the large model, and feedback from domain experts to construct a supplementary candidate set.

[0145] In step S43, the temporal dynamics of the knowledge graph are captured, entity relationships are mapped into time-varying embedding vectors, Fourier transform is used to detect the periodicity of relationship occurrences, and the specific process of detecting relationship mutation points is identified by the preset CUSUM algorithm as follows:

[0146] S431: Represent the knowledge graph as a set of time-series triples: G ={( h , r , t , t )},in t is the timestamp, divided into time windows Δ t , discretize the continuous time into T ={ t1, t 2,…, t n};

[0147] S432: For entities h and relationships r Introducing time-dependent embedding vectors:

[0148] e h ( t ) =e h + Δ h ( t );

[0149] e r ( t ) =e r + Δ r ( t );

[0150] Among them, Δ( t ) is the time t The corresponding dynamic offset vector, e h ( t ) is an entity h In time t The embedding vector at time , represents the time-dependent entity embedding, e r ( t ) is the relationship r In time t The embedding vector at time represents the time-dependent relationship embedding, e h For Entity h The basic embedding vector of is the static part that does not change over time, e r For the relationship r The basic embedding vector is the static part that does not change over time, Δ h ( t ) is an entity h In time t The embedding increment at Δ is the dynamic part that changes with time and is used to capture the state changes of the entity at different time points. r ( t ) is the relationship r The embedding increment at time τ is the dynamic part that changes over time and is used to capture the semantic changes of the relationship at different time points;

[0151] S433: For each relationshipr , extract its embedding vector at each time point e r ( t i ), or calculate the interaction strength of related entity pairs, s ( h,t,t ) =e h ( t ) T e r ( t ) e t ( t ) ), forming a time series S r ={ s (τ1), s (τ2),…,s(τ n )};

[0152] S434: Time Series S r Perform de-averaging and normalization processing, calculate the discrete Fourier transform, and take the modulus to obtain the amplitude spectrum and frequency resolution;

[0153] S435: Filter low-frequency noise, retain high-frequency components, set amplitude thresholds, extract significant frequencies, draw amplitude spectrograms, and mark the periods corresponding to significant frequencies;

[0154] S436: Order x t =S r ( t ) is the normalized timing value, the preset reference value m 0 and offset d (minimum change to detect), calculate the forward cumulative sum C t + and backward cumulative sum C t − :

[0155] C t + = max {0, C t-1 + +( x t −( m 0+ d / 2))};

[0156] Ct − = max {0, C t-1 − +( m 0− d / 2− x t )};

[0157] in, C t-1 + is the forward cumulative sum calculated at time step t-1, C t-1 − is the backward cumulative sum calculated at time step t-1;

[0158] S437: Preset control threshold h (Usually 4-6 s , s is the time series standard deviation), when C t + > h or C t − > h When a mutation alert is triggered, the parameters are initialized: C 0 + = C 0 − =0;

[0159] S438: When C t + Exceed h , it is determined to be an upward mutation (relationship enhancement);

[0160] when C t − Exceed h , judged as a downward mutation (relationship weakened), record the mutation time t , and reset the cumulative sum to 0 to detect subsequent mutations.

[0161] The specific process of step S5 is as follows:

[0162] S51: Define the change operation set {CREATE, UPDATE, DELETE, MERGE}, where CREATE, UPDATE, DELETE, and MERGE correspond to entity / relationship creation, update, deletion, and merge operations respectively;

[0163] S52: Calculate the difference between the optimized knowledge representation and the existing knowledge graph using difference detection based on graph edit distance;

[0164] S53: Serialize the change operation into a timestamped JSON-LD object, serialize large-scale change data using Protobuf, and reduce transmission overhead by gzip compression;

[0165] S54: Automatically generate a new version after incremental update submission, record change summary and diff hash, implement rollback by reverse application of historical diff, query the graph state at a certain time point using Cypher statements, visualize the difference between the new version and the historical version, highlight the added / deleted entity relationships, and support attribute change diff viewing;

[0166] Record key operations including entity creation, relationship modification, version merging, etc., and store them in a separate audit log library to meet compliance requirements. Access to historical versions is subject to fine-grained permission management through RBAC (role-based access control) to restrict sensitive operations;

[0167] Set incremental storage strategy, bucket storage: store incremental data in time windows (such as daily / weekly) to avoid large single files, index optimization: establish B + tree index for entity ID, timestamp to accelerate version query, use LSM tree structure to realize efficient incremental writing and historical version merging;

[0168] Only store the difference between adjacent versions, reconstruct historical versions through chain structure, save storage space, store recent versions in high-performance SSD, and migrate historical versions to low-cost storage media such as HDFS.

[0169] A knowledge graph construction system based on a large language model is used to implement the knowledge graph construction method based on the large language model, comprising a data acquisition module, a data preprocessing module, a learning framework construction module, a dynamic adjustment module, an update enhancement module, and a knowledge graph generation module. The data acquisition module is connected with the data preprocessing module, the data preprocessing module is connected with the learning framework construction module, the dynamic adjustment module is connected with the update enhancement module, and the dynamic adjustment module is connected with the update enhancement module.

[0170] The data acquisition module is used to acquire multimodal data such as text, images, and audio. The data preprocessing module is used to convert image and audio data into semantic vectors, fuse them with text data, and process the fused data. The learning framework construction module is used to construct a reinforcement learning framework, using the large model as an intelligent agent, with the accuracy and completeness of entity relationship extraction as reward indicators, learning the optimal extraction strategy through interaction with the data environment, and using the contrastive learning method to enhance the large language model's ability to distinguish similar entities and relationships, extracting entities and relationships from the preprocessed data to generate entity relationship candidate sets. The dynamic adjustment module is used to preset a large-scale knowledge graph as a teacher model, and the new entity relationship knowledge to be fused as a student model, so that the student model learns the knowledge distribution of the teacher model, fuses the newly extracted entity relationship with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights in the fusion process to achieve conflict resolution and semantic alignment. The update and enhancement module leverages the large language model to dynamically generate knowledge validation rules based on the knowledge graph's historical update data and domain knowledge. This module analyzes the evolution of entity relationships within the knowledge graph, identifies potential knowledge errors and gaps, and enhances the optimized knowledge graph, supplementing missing relationships and correcting erroneous ones. The knowledge graph generation module incrementally updates the optimized knowledge representation to the knowledge graph storage system, integrating it with version control mechanisms to generate a final knowledge graph with historical traceability.

[0171] In summary, the knowledge graph construction method and system provided by the present invention based on the large language model realizes the automatic construction and dynamic maintenance of the knowledge graph through multimodal data fusion, joint optimization of reinforcement learning and contrastive learning, teacher-student model knowledge transfer, time series dynamic analysis and incremental update mechanism. Construct a reinforcement learning framework, and train a large model to extract entity relationships with accuracy and completeness as reward indicators; use a large-scale knowledge graph as a teacher model, and realize conflict resolution and semantic alignment between new knowledge and existing graphs through the graph attention network; dynamically generate verification rules based on historical data and domain knowledge, and combine Fourier transform and CUSUM algorithm to detect relationship periodicity and mutation points; finally, generate a traceable knowledge graph through incremental update and version control. It improves the accuracy of multimodal data processing, the dynamic adaptability of entity relationship extraction and the efficiency of knowledge graph maintenance.

Claims

1. A knowledge graph construction method based on a large language model, characterized in that: The following steps are involved: S1: Acquire multimodal data of text, images, and audio, convert the image and audio data into semantic vectors, fuse them with the text data, and process the fused data; S2: Build a reinforcement learning framework, using the large model as an intelligent agent and entity relationship extraction accuracy and completeness as reward indicators. By interacting with the data environment, we learn the optimal extraction strategy. We use contrastive learning methods to enhance the large language model's ability to distinguish similar entities and relationships. We extract entities and relationships from preprocessed data to generate entity relationship candidate sets. S3: A large-scale knowledge graph is pre-set as the teacher model, and the new entity relationship knowledge to be integrated is used as the student model. The student model learns the knowledge distribution of the teacher model, integrates the newly extracted entity relationships with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights during the integration process to achieve conflict resolution and semantic alignment. S4: Leveraging a large language model based on the historical update data and domain knowledge of the knowledge graph, dynamically generates knowledge verification rules, analyzes the evolution trend of entity relationships in the knowledge graph, identifies potential knowledge errors and omissions, and enhances the optimized knowledge graph to supplement missing relationships and correct erroneous relationships. S5: Update the optimized knowledge representation to the knowledge graph storage system in an incremental manner, and combine it with the version control mechanism to generate the final knowledge graph with historical traceability capabilities; The specific process of step S2 is as follows: S21: converting the fused feature vector into an input format that can be processed by the large model, and concatenating the converted fused feature vector with the corresponding text description as the current state of the environment; S22: Define the action space as the results of all entity relationship extractions. An action is an entity pair and its relationship identified from the data. The large model represents the action by generating a corresponding text sequence. The large language model is used as an intelligent agent. The intelligent agent receives the current state of the environment, generates an action based on internal parameters, and executes the action in the environment. S23: Calculate the degree of match between the entity relationships extracted by the agent and the correct entity relationships in the annotated data, and give corresponding rewards or penalties based on the degree of match; if the extracted entity relationships are completely correct, a high reward is given; if there are partial errors, a low reward is given based on the degree of error; if there are complete errors, a penalty is given; Measure the extent to which the extraction results cover all entity relationships in the labeled data, and give corresponding rewards or penalties based on the coverage; If all correct entity relationships are extracted, rewards are given; if there are relationships that are not extracted, penalties are given according to the number of missing relationships. S24: Dynamically adjust the weights of accuracy and completeness rewards based on task requirements and data characteristics, while also taking extraction speed into consideration. Speed ​​rewards are awarded if extraction is completed in a short period of time while ensuring accuracy and completeness. S25: Based on the current strategy, the agent generates and executes actions according to the environment state. The environment calculates rewards based on the labeled data and reward function, and feeds back the next environment state to the agent. S26: Use the policy gradient algorithm to update the large model parameters based on the reward and state transition information, with the goal of maximizing the long-term cumulative reward; S27: Select sample pairs containing similar entities or relations from the preprocessed data, divide them into positive sample pairs with entity relations of the same or similar semantics and negative sample pairs with entity relations of different semantics, input the sample pairs into the large language model, obtain the feature representation of its output, and optimize it using the contrastive learning loss function; S28: Input the new preprocessed data into the large language model that has been trained by reinforcement learning and optimized by contrastive learning, extract entity relationships according to the learned optimal strategy, generate multiple entity relationship results, filter the extraction results according to the confidence threshold, retain entity relationships with high confidence, and form the final entity relationship candidate set.

2. A method for constructing a knowledge graph based on a large language model according to claim 1, characterized in that: The specific process of obtaining multimodal data of text, image, and audio in step S1 and converting the image and audio data into semantic vectors is as follows: S11: Construct a neural network model to extract local features and global features of the acquired image data to obtain a high-dimensional image feature vector; S12: Convert the audio data to a specified sampling rate, scale its amplitude range to a preset range, set the audio frame length and frame shift to form an overlapping frame structure, set a windowing function to reduce spectrum leakage, perform a fast Fourier transform on the windowed signal per frame to convert it into the frequency domain to obtain an audio amplitude spectrum, construct a triangular filter to perform frequency conversion within a preset frequency range, extract the acoustic features of the audio, and construct an audio feature vector; S13: Construct a shared semantic space by training a fully connected neural network to construct a mapping layer to map the image feature vector and the audio feature vector to the shared semantic space respectively. The vectors of the image data and the audio data in the shared semantic space are semantic vectors.

3. A method for constructing a knowledge graph based on a large language model according to claim 2, characterized in that: The specific process of fusing the data with the text data in step S1 and processing the fused data is as follows: S14: Obtain text vectors for the text data by word embedding, obtain the dimensions of the semantic vectors and text vectors of the image data and audio data, and map the semantic vectors and text vectors to preset dimensions through a mapping layer of a fully connected neural network; S15: Train a multi-layer perceptron weight prediction model, setting weights based on the importance of different modal data. Optimize the task goal of entity relationship extraction accuracy, learn the weights of image, audio, and text vectors, and perform feature fusion based on the weights to obtain a fused feature vector. S16: Calculate the density of the fused feature vector in the feature space, consider vectors with a density lower than the threshold as outliers, set the neighborhood radius and the minimum number of samples, and if the number of samples of a vector in a neighborhood with the vector as the center and the radius as the neighborhood radius is less than the minimum number of samples, it is considered an outlier and removed; S17: Calculate the similarity between the fused feature vectors using cosine similarity, set a similarity threshold, and if the cosine similarity of the two fused feature vectors is greater than the threshold, retain the vector with richer information and delete the redundant vector; S18: Use the large language model to perform semantic analysis on the fused feature vector, input the multimodal data description corresponding to the fused feature vector, and determine whether the semantics are consistent. If the semantic consistency score output by the large language model is lower than the set threshold, the fused feature vector is corrected.

4. The method for constructing a knowledge graph based on a large language model according to claim 1, characterized in that: The specific process of step S3 is as follows: S31: Map entities and relationships to a low-dimensional vector space by presetting a large-scale knowledge graph. Use the student model to represent newly extracted entity relationships using a vector space of the same dimension. Initialize the representation using random embeddings or a pre-trained language model. S32: Construct a mapping matrix and learn alignment parameters by optimizing a preset objective function. The teacher model generates a soft label distribution for each triple in the student model and incorporates hard label loss and soft label loss. S33: Build a graph attention network, concatenate entity vectors and relationship vectors as node initial features, calculate multi-head attention scores, and adjust knowledge fusion weights through a gating mechanism; S34: Calculate the entity similarity matrix. When the similarity exceeds the threshold and the relationship is inconsistent, mark it as a conflict. Construct an alignment loss function, adjust the fusion weight of the conflicting entities based on the attention score, evaluate the knowledge graph completion performance, evaluate the entity alignment accuracy, and use the semantic similarity distribution to evaluate the fusion quality.

5. The method for constructing a knowledge graph based on a large language model according to claim 1, characterized in that: The specific process of step S4 is as follows: S41: Sort the historical update data of the knowledge graph by timestamp, construct the entity relationship evolution sequence, calculate the difference between adjacent snapshots, extract the added / deleted entity relationships, and form a change pattern library; S42: Encode domain knowledge into ontology constraints, generate templates based on preset design rules, input a small number of rule examples into the large model to guide it to generate new rules, quickly adapt to the rule generation needs of different domains through meta-learning, and adjust model parameters to minimize rule generation errors; S43: Capture the temporal dynamics of the knowledge graph, map entity relationships into time-varying embedding vectors, use Fourier transform to detect the periodicity of relationship occurrences, and identify relationship mutation points using a preset CUSUM algorithm; S44: Determine the causal dependencies between entity relationships, simulate the potential impact of relationship changes, convert the generated verification rules into Datalog queries, determine the semantic rationality of entity relationships, combine the TransE knowledge graph embedding model with the text features generated by the large model, predict missing relationships, use the reasoning ability of the large model to complete implicit knowledge, select the optimal correction solution based on confidence, update the knowledge graph, and combine the link prediction results, knowledge generated by the large model, and feedback from domain experts to construct a supplementary candidate set.

6. A method for constructing a knowledge graph based on a large language model according to claim 5, characterized in that: In step S43, the temporal dynamics of the knowledge graph are captured, entity relationships are mapped into time-varying embedding vectors, Fourier transform is used to detect the periodicity of relationship occurrences, and the specific process of detecting relationship mutation points is identified by the preset CUSUM algorithm as follows: S431: Represent the knowledge graph as a set of time series triples, divide the time window, and discretize the continuous time; S432: Introducing time-dependent embedding vectors for entities and relations; S433: For each relationship, extract its embedding vector at each time point, or calculate the interaction strength of the associated entity pair; S434: De-meaning and normalizing the time series, calculating the discrete Fourier transform, and taking the modulus to obtain the amplitude spectrum and frequency resolution; S435: Filter low-frequency noise, retain high-frequency components, set amplitude thresholds, extract significant frequencies, draw amplitude spectrograms, and mark the periods corresponding to significant frequencies; S436: Setting the normalized timing value, presetting the reference value and offset, and calculating the forward cumulative sum and the backward cumulative sum; S437: Preset a control threshold. When the forward cumulative sum and the backward cumulative sum exceed the control threshold, a sudden change alarm is triggered and parameters are initialized. S438: When the cumulative sum exceeds the control threshold, it is determined to be an upward mutation; When the backward cumulative sum exceeds the control threshold, it is determined to be a downward mutation, the mutation time is recorded, and the cumulative sum is reset to 0 to detect subsequent mutations.

7. The method for constructing a knowledge graph based on a large language model according to claim 5, characterized in that: The specific process of step S5 is as follows: S51: Define the change operation set {CREATE, UPDATE, DELETE, MERGE}, where CREATE, UPDATE, DELETE, and MERGE correspond to the creation, update, deletion, and merge operations of entities / relationships, respectively; S52: Use graph edit distance-based difference detection to calculate the difference between the optimized knowledge representation and the existing knowledge graph; S53: Serialize change operations into JSON-LD objects with timestamps, use Protobuf to serialize large-scale change data, and use gzip compression to reduce transmission overhead; S54: Automatically generates a new version after an incremental update is committed, records the change summary and diff hash, implements rollback by reverse-applying the historical diff, uses Cypher statements to query the graph state at a certain point in time, and visualizes the differences between the new version and the historical version; Record key operations including entity creation, relationship modification, and version merging, store them in an independent audit log library, and implement fine-grained permission management for access to historical versions; Set an incremental storage strategy, bucket and store incremental data by time window, and create a tree index for entity ID and timestamp.

8. A knowledge graph construction system based on a large language model, used to implement a knowledge graph construction method based on a large language model according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, data preprocessing module, learning framework construction module, dynamic adjustment module, update enhancement module, and knowledge graph generation module; The data acquisition module is used to acquire multimodal data such as text, image, and audio; The data preprocessing module is used to convert image and audio data into semantic vectors, fuse them with text data, and process the fused data; The learning framework construction module is used to build a reinforcement learning framework, using the large model as an intelligent agent and the accuracy and completeness of entity relationship extraction as reward indicators. The optimal extraction strategy is learned by interacting with the data environment, and the contrastive learning method is used to enhance the large language model's ability to distinguish similar entities and relationships. Entities and relationships are extracted from preprocessed data to generate entity relationship candidate sets; The dynamic adjustment module is used to preset a large-scale knowledge graph as a teacher model and use the new entity relationship knowledge to be integrated as a student model. The student model learns the knowledge distribution of the teacher model, integrates the newly extracted entity relationships with the existing knowledge graph, and uses the graph attention network to dynamically adjust the entity relationship weights during the integration process to achieve conflict resolution and semantic alignment. The update enhancement module is used to dynamically generate knowledge verification rules based on the historical update data and domain knowledge of the knowledge graph using the large language model, analyze the evolution trend of entity relationships in the knowledge graph, discover potential knowledge errors and omissions, enhance the optimized knowledge graph, supplement missing relationships, and correct erroneous relationships; The knowledge graph generation module is used to update the optimized knowledge representation to the knowledge graph storage system in an incremental manner, and combine with the version control mechanism to generate a final knowledge graph with historical traceability capabilities.

Images