Conference record data searching method and system based on AI

Abstract

The invention discloses an AI-based conference record data searching method and system, and the method comprises the steps: carrying out the feature extraction and alignment through a multi-mode fusion neural network according to the voice, text, image and video data collected in a conference process, and obtaining a semantic representation vector; according to the semantic representation vector, combining context information of the conference scene, and utilizing a pre-trained context perception model to perform semantic enhancement processing to generate an enhanced semantic vector with context association; according to the enhanced semantic vector, combining with an external knowledge base, and utilizing a dynamic knowledge graph construction algorithm to generate a knowledge graph related to the conference theme in real time; and according to the knowledge graph, intelligent retrieval and recommendation of conference record data are carried out by using a graph neural network. By utilizing the embodiment of the invention, the intelligent retrieval efficiency and accuracy of the conference record can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of AI technology, and in particular to an AI-based conference record data search method and system. Background Art

[0002] With the advancement of information technology, the form of meeting records has shifted from traditional paper records to more intelligent electronic records. However, with the increase in the number of meeting participants and the complexity of meeting content, how to effectively organize, store, and retrieve meeting record data has become a huge challenge.

[0003] Traditional meeting record-keeping methods typically rely on manual recording and note-taking, which is not only time-consuming and labor-intensive but also prone to omissions and errors. Manually sifting through reams of meeting records when searching for specific information is often inefficient and unable to meet the needs of rapid decision-making and information retrieval. Furthermore, existing meeting record retrieval systems often focus on a single modality when processing multimodal data (such as voice, text, images, and video), lacking the ability to comprehensively understand and process multiple forms of information, resulting in insufficient accuracy and completeness in information retrieval. Summary of the Invention

[0004] The purpose of the present invention is to provide an AI-based meeting record data search method and system to address the deficiencies in the existing technology and improve the efficiency and accuracy of intelligent retrieval of meeting records.

[0005] One embodiment of the present application provides an AI-based method for searching meeting record data, the method comprising: Based on the voice, text, image, and video data collected during the meeting, a multimodal fusion neural network is used to extract and align features to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority integration of key information. Based on the semantic representation vector and combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association. The context-aware model captures the temporal dependencies in the meeting records by introducing time series modeling technology, thereby improving the accuracy of semantic understanding; Based on the enhanced semantic vectors and in combination with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the conference topic in real time. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time, thereby ensuring the timeliness and integrity of the knowledge graph. Based on the knowledge graph, a graph neural network is used to intelligently retrieve and recommend meeting record data. The graph neural network introduces an adaptive path search algorithm to dynamically adjust the retrieval path according to the user's query intent, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

[0006] Optionally, the method of extracting and aligning features using a multimodal fusion neural network based on the voice, text, image, and video data collected during the meeting to obtain a semantic representation vector includes: The speech, text, image, and video data collected during the meeting were preprocessed separately. The speech data was subjected to noise reduction and speech recognition processing to obtain a speech feature vector. The text data was subjected to word segmentation, stop word removal, and word vectorization processing to obtain a text feature vector. The image data was subjected to target detection and feature extraction to obtain an image feature vector. The video data was subjected to key frame extraction and action recognition to obtain a video feature vector. Based on the preprocessed speech, text, image and video data, feature extraction is performed respectively using independent encoders in the multimodal fusion neural network, wherein the speech feature vector is encoded by the speech encoder to obtain a speech semantic vector, the text feature vector is encoded by the text encoder to obtain a text semantic vector, the image feature vector is encoded by the image encoder to obtain an image semantic vector, and the video feature vector is encoded by the video encoder to obtain a video semantic vector; Based on the extracted semantic vectors of each modality, a cross-modal attention mechanism is used to perform feature alignment. For each modality’s semantic vector, the attention weights between it and the semantic vectors of other modalities are calculated to generate weighted modality features. The multi-head attention mechanism is used to interactively fuse the weighted modal features to generate preliminary fusion features; Based on the preliminary fusion features, a decoder in the multimodal fusion neural network is used to generate a unified semantic representation vector. The preliminary fusion features are input into the decoder, and through multiple layers of nonlinear transformation and normalization, a unified semantic representation vector is obtained. The unified semantic representation vector contains the multimodal key information of the meeting minutes and retains the semantic relevance between the modalities. The semantic vectors whose attention weights are higher than the preset threshold are filtered out, and the filtered semantic vectors are secondary fused with the unified semantic representation vector, and finally a semantic representation vector containing key information is output.

[0007] Optionally, performing semantic enhancement processing based on the semantic representation vector and combining contextual information of the conference scene with a pre-trained context-aware model to generate an enhanced semantic vector with contextual association includes: Extract the context information of the meeting scene and encode the context information into a context feature vector; The semantic representation vector and context feature vector are input into a pre-trained context-aware model. The context-aware model captures the temporal dependencies in the meeting transcripts by introducing time series modeling techniques. The context-aware model dynamically adjusts the semantic representation vector based on the time series information to generate a context-relevant intermediate semantic vector. Based on the intermediate semantic vector and the context feature vector, an attention mechanism is used to perform semantic enhancement processing, wherein the attention weight of the context feature vector to the intermediate semantic vector is calculated to generate a weighted context semantic vector; The weighted contextual semantic vector is fused with the intermediate semantic vector to generate an enhanced semantic vector with contextual association. The enhanced semantic vector contains the contextual information of the meeting scene and retains the global information of the semantic representation vector.

[0008] Optionally, the method of generating a knowledge graph related to the conference topic in real time using a dynamic knowledge graph construction algorithm based on the enhanced semantic vector and in combination with an external knowledge base includes: The enhanced semantic vector is input into the pre-trained entity recognition model to extract the key entity set in the meeting transcript, where each entity contains entity type and context information; The enhanced semantic vector and the extracted entity set are input into the pre-trained relation extraction model to extract the relationship set between entities, where each relationship contains the relationship type and the entity pair involved; Based on the entity set and relationship set, the entities and relationships are aligned. Based on the aligned entities and relationships, in combination with the external knowledge base, entity linking and relationship expansion are performed. The relevant entities and relationships in the external knowledge base are mapped to the current meeting scenario to generate the initial knowledge graph. Based on the real-time input of meeting minutes, the initial knowledge graph is dynamically updated using an incremental learning mechanism. New entities and relationships are detected and added to the initial knowledge graph. A node embedding update algorithm is then used to dynamically adjust the representation of nodes and edges in the initial knowledge graph to ensure the timeliness of the knowledge graph. According to the changes in conference topics, the knowledge graph is pruned and reconstructed using a graph optimization algorithm. This involves removing redundant nodes and edges to ensure the simplicity of the knowledge graph, merging similar entities and relationships to ensure the integrity of the knowledge graph, and ultimately generating an optimized knowledge graph.

[0009] Optionally, the intelligent retrieval and recommendation of meeting record data using a graph neural network based on the knowledge graph includes: Based on the query statement entered by the user, natural language processing technology is used to analyze the query intent and express the query intent as a query vector; Input the knowledge graph into the graph neural network to initialize the feature representation of entity nodes and relationship edges; Based on the query vector, the adaptive path search algorithm is used to dynamically adjust the retrieval path in the graph neural network; Calculate the similarity between the query vector and each entity node, generate node weights, and dynamically select entity nodes and relationship edges that are highly relevant to the query intent based on the node weights to generate a retrieval path; Extract relevant meeting record fragments from the knowledge graph based on the retrieval path; Based on the retrieval path and meeting record fragments, a graph neural network is used to generate relevant recommended content. The recommended content includes entities, relationships, and meeting topics related to the query intent. Finally, the meeting record fragments and related recommended content that match the query intent are output.

[0010] Another embodiment of the present application provides an AI-based meeting record data search system, the system comprising: An extraction module is used to extract and align features from the voice, text, image, and video data collected during the meeting using a multimodal fusion neural network to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weights of each modality by introducing a cross-modal attention mechanism to ensure the priority integration of key information. An enhancement module is configured to perform semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model captures the temporal dependencies in the meeting minutes by introducing time series modeling technology, thereby improving the accuracy of semantic understanding; A construction module is used to generate a knowledge graph related to the conference topic in real time based on the enhanced semantic vector and in combination with an external knowledge base using a dynamic knowledge graph construction algorithm. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. A retrieval module is used to intelligently retrieve and recommend meeting record data based on the knowledge graph using a graph neural network. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that highly match the query intent.

[0011] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when run.

[0012] Yet another embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0013] Compared with the existing technology, the present invention provides an AI-based meeting record data search method, which uses a multimodal fusion neural network to perform feature extraction and alignment based on the voice, text, image and video data collected during the meeting to obtain a semantic representation vector; based on the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with context association; based on the enhanced semantic vector, combined with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the meeting topic in real time; based on the knowledge graph, a graph neural network is used to perform intelligent retrieval and recommendation of meeting record data, thereby improving the efficiency and accuracy of intelligent retrieval of meeting records. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A hardware block diagram of a computer terminal for an AI-based conference record data search method provided by an embodiment of the present invention; Figure 2 A flowchart of an AI-based conference record data search method provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an AI-based meeting record data search system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0016] The embodiment of the present invention first provides an AI-based meeting record data search method, which can be applied to electronic devices such as computer terminals, specifically ordinary computers.

[0017] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal for an AI-based conference record data search method provided by an embodiment of the present invention. Figure 1 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0018] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause a processor to execute any one of the AI-based meeting record data search methods.

[0019] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0020] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any AI-based meeting record data search method.

[0021] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0022] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0023] See also Figure 2 , an embodiment of the present invention provides an AI-based conference record data search method, which may include the following steps: S201: Based on the voice, text, image, and video data collected during the meeting, a multimodal fusion neural network is used to extract and align features to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority fusion of key information. First, the system preprocesses the collected multimodal data to ensure data quality and parsability. Voice data undergoes noise reduction and speech recognition to extract meaningful speech features. Text data is converted into feature vectors through operations such as word segmentation and stop word removal. Image and video data undergo object detection and action recognition, respectively, to extract valid feature vectors. Next, these different modal data are input into a multimodal fusion neural network, which generates a unified semantic representation vector through feature alignment between the modalities. This network utilizes a cross-modal attention mechanism to dynamically adjust the weights of each modality, ensuring that key information is prioritized when generating the final semantic representation vector, reflecting the combined significance of multiple information sources.

[0024] This processing method, which uses multimodal fusion neural networks, enables in-depth analysis and semantic understanding of conference data, significantly improving the usability of information. Traditional single-modal analysis can overlook certain key information or lead to information loss. However, multimodal fusion allows for the effective integration of different forms of information, such as voice, text, images, and video, to achieve a more comprehensive semantic understanding. Especially in conference scenarios, due to the diverse and complex sources of information, the use of multimodal fusion can not only improve the accuracy of information retrieval and analysis, but also create richer context for subsequent data processing and analysis, making the subsequent processing of meeting records more intelligent and efficient.

[0025] Specifically, the voice, text, image, and video data collected during the meeting can be preprocessed separately, including noise reduction and voice recognition processing on the voice data to obtain a voice feature vector, word segmentation, stop word removal, and word vectorization on the text data to obtain a text feature vector, target detection and feature extraction on the image data to obtain an image feature vector, and key frame extraction and action recognition on the video data to obtain a video feature vector. In this step, the system will focus on preprocessing various types of data collected during the meeting to improve the effectiveness and accuracy of subsequent analysis. First, the voice data undergoes noise reduction to remove background noise and improve the accuracy of speech recognition. Then, speech recognition technology is used to convert the speech into text feature vectors, capturing the key information contained in the speech. Text data is then segmented and stop-word removed to remove irrelevant content and generate word vectors for subsequent processing. For image data, object detection technology is used to identify meeting participants and important visual content, extracting their feature vectors. For video data, keyframe extraction and action recognition are used to obtain keyframe content and action information, resulting in video feature vectors.

[0026] This preprocessing step significantly improves the usability of data from different modalities, laying a solid foundation for multimodal fusion. By extracting features from speech, text, images, and video, the system can effectively capture important information from the meeting, generating clearer and more meaningful feature vectors. These preprocessed feature vectors provide critical input for subsequent feature fusion and semantic understanding, enabling the resulting semantic representation vectors to more accurately reflect the meeting content, improving the precision of data retrieval and analysis.

[0027] At the initial stage of this step, the system performs noise reduction on the collected speech data to improve subsequent speech recognition accuracy. Noise reduction typically uses spectral subtraction or adaptive filtering. For example, spectral subtraction first converts the audio signal into a spectrogram, then estimates the noise component and subtracts it based on the energy levels of different frequencies. Finally, the processed signal is restored to a time domain signal. Through this process, the system can effectively reduce the impact of ambient noise on speech recognition, ensuring that the subsequent speech features obtained are clearer.

[0028] The clean, noise-reduced speech signal is then fed into a speech recognition model for processing. Currently popular speech recognition models often employ deep neural networks (DNNs), such as long short-term memory (LSTM) networks or convolutional neural networks (CNNs). These models analyze the temporal characteristics of audio signals to convert them into text. This processing generates a speech feature vector that not only reflects the speaker's content but also retains a certain degree of emotional identity, providing richer information for subsequent analysis.

[0029] In addition to voice data, when processing text data, the system first performs word segmentation and stop word removal. Word segmentation is the process of dividing long text into logically meaningful words, typically through rule-based segmentation or statistical methods. Stop word removal involves removing words that are not helpful for semantic analysis based on a predefined stop word list. After completing these steps, the system uses word embedding technology to convert the text into feature vectors. For example, the Word2Vec model is used to generate word vectors, so that similar words have similar vector representations, laying the foundation for subsequent feature extraction. For image data, the system uses computer vision technology and convolutional neural networks (CNNs) for object detection, analyzing key objects appearing in the image, and ultimately extracting image feature vectors. For video data, keyframe extraction is first performed. Then, motion recognition algorithms are used to analyze the motion changes between keyframes to obtain video feature vectors, ensuring that rich feature information is extracted from multimodal data.

[0030] Based on the preprocessed speech, text, image and video data, feature extraction is performed respectively using independent encoders in the multimodal fusion neural network, wherein the speech feature vector is encoded by the speech encoder to obtain a speech semantic vector, the text feature vector is encoded by the text encoder to obtain a text semantic vector, the image feature vector is encoded by the image encoder to obtain an image semantic vector, and the video feature vector is encoded by the video encoder to obtain a video semantic vector; In this step, the system uses independent encoders in the multimodal fusion neural network to process the feature vectors of each modality to extract the corresponding semantic vectors. Each encoder is trained specifically for the features of its input modality, enabling it to effectively capture the deep semantics contained in speech, text, image, and video data. For example, the speech encoder encodes speech feature vectors into speech semantic vectors using a convolutional neural network (CNN) or a recurrent neural network (RNN), thereby extracting the emotion, intonation, and other semantic information in the speech. Similarly, the text encoder uses pre-trained language models such as BERT or GPT to convert text feature vectors into text semantic vectors, capturing the semantic relationships and contextual information in the text. The image and video encoders use corresponding convolutional networks to perform deep learning on image feature vectors and video feature vectors, respectively, to extract important visual information from images and videos.

[0031] By deeply processing the feature vectors of different modalities through independent encoders, the system can obtain semantic vectors for each modality, allowing the information of each modality to be expressed independently and accurately. This independent processing method helps the system better understand the unique characteristics of each modality, especially in a multimodal environment, and can effectively reduce interference between modalities and improve the accuracy of overall feature extraction. Ultimately, the obtained multimodal semantic vectors lay a good foundation for subsequent cross-modal alignment and fusion, ensuring the accuracy and effectiveness of information integration.

[0032] In this step, the system uses multiple independent encoders to encode the preprocessed feature vectors to extract higher-dimensional semantic information. First, for the speech feature vectors, the system uses a specially designed speech encoder, typically based on deep learning architectures such as recurrent neural networks (RNNs) or long short-term memory networks (LSTMs). These network architectures are able to capture the temporal characteristics of speech and, through multi-layered neural networks, extract higher-level semantic features layer by layer. Ultimately, the speech semantic vector output by this encoder represents the core information of the speech content and, to a certain extent, reflects the speaker's emotional changes.

[0033] Furthermore, for text feature vectors, the system leverages pre-trained text encoders (such as BERT or GPT) to understand the text content through deep semantic analysis. This process also considers word order, ensuring that the contextual relationships between individual words are fully preserved during the generation of text semantic vectors. After processing, the text semantic vectors output by the text encoder provide important information support for subsequent multimodal feature fusion.

[0034] The processing of the image encoder and video encoder is relatively similar. The system uses convolutional neural networks (CNNs) to encode image and video feature vectors, respectively. For image data, the encoder can recognize visual features such as color, shape, and specific objects (for example, the facial features of participants or charts presented in the meeting). Here, the network gradually extracts high-level features of the image through multiple layers of convolution and pooling operations to obtain image semantic vectors. For video data, the system inputs key frames into the video encoder in sequence, using a temporal network to capture actions and scene changes. This dynamic feature extraction method ensures that the video semantic vector can not only represent the information of static images, but also capture changes in time series, thereby providing a full picture for further analysis of the meeting scene.

[0035] Based on the extracted semantic vectors of each modality, a cross-modal attention mechanism is used to perform feature alignment. For each modality’s semantic vector, the attention weights between it and the semantic vectors of other modalities are calculated to generate weighted modality features. In this step, the system applies a cross-modal attention mechanism to achieve feature alignment between different modalities. The core of this mechanism is that the system calculates the attention weight between the semantic vector of each modality and the semantic vectors of other modalities. This process identifies the importance of each modality by calculating the similarity between the modalities and dynamically adjusts their weights. For example, if the attention weight between the speech semantic vector and the text semantic vector is high, then in the subsequent fusion, the text feature will obtain a higher weight, making the text information more prominent in the overall semantic representation. Through this mechanism, the system can focus on key information in a targeted manner, thereby enhancing the effect of feature fusion and reducing the possible noise impact between modalities.

[0036] The introduction of the cross-modal attention mechanism significantly improves the quality of multimodal feature alignment, ensuring that each modality maximizes its importance in generating a unified semantic representation. This mechanism not only allows the system to dynamically perceive the relationships between modalities but also enables it to adaptively adjust feature weights in different contexts, improving the flexibility and accuracy of information fusion. By effectively aligning the semantic vectors of different modalities, the resulting semantic representation vector is more complete and meaningful, facilitating subsequent semantic processing and applications.

[0037] The core of this step is to align the features of the semantic vectors of different modalities through a cross-modal attention mechanism to achieve effective information fusion. In this process, the system first calculates the similarity between the modalities, usually using methods such as cosine similarity or dot product. For example, for speech and image modalities, the system evaluates the similarity between the semantic vectors of the two to determine the relationship between them. This similarity calculation process lays the foundation for subsequent weighting operations. By performing this evaluation on the semantic vectors of each modality, we can identify which modalities are more important in a specific context and are more conducive to the overall understanding of the information.

[0038] Once the similarities are calculated, the system uses these similarity values to generate attention weights. This weight determines the importance of each modality in the final fusion stage. For example, if the Yaoyin modality contributes significantly to the overall semantic expression in a particular context, the system will assign it a higher weight to ensure that the modal information is preserved during the fusion process and avoid information loss. This method also allows the system to adaptively adjust the weights of each modality in different scenarios, achieving dynamic information optimization.

[0039] Finally, all calculated weighted modal features serve as input for the subsequent feature fusion stage. These weighted features fully reflect the information from each modality while also preserving important semantic relevance, enabling the overall model to more comprehensively and accurately understand meeting content. By introducing a cross-modal attention mechanism, the system effectively reduces intermodal noise interference, thereby improving the accuracy and effectiveness of overall data fusion.

[0040] The multi-head attention mechanism is used to interactively fuse the weighted modal features to generate preliminary fusion features; In this step, the system uses a multi-head attention mechanism to interactively fuse the previously generated weighted modal features. The core idea of the multi-head attention mechanism is to run multiple attention mechanisms in parallel, each calculating different attention representations to capture the diversity and complex relationships between features. Each head independently focuses on different aspects of the weighted modal features. One head may focus on the combination of speech and image, while another head may focus on the information interaction between text and video. This parallel processing ensures that the multimodal features can fully and meticulously reflect the relationships between each feature during the fusion process.

[0041] By introducing a multi-head attention mechanism, the system can more effectively capture the complex relationships between features across modalities and generate richer preliminary fusion features. This approach not only enhances the complementarity of information across modalities but also provides a more comprehensive input for the subsequent generation of a unified semantic representation. This interactive fusion strategy ensures that the final output feature vector not only contains information from each modality but also reveals potential new information generated by the interaction of different modalities, thereby enhancing the expressive power of the semantic representation.

[0042] In this step, the system uses a multi-head attention mechanism to interactively fuse the weighted modal features. The multi-head attention mechanism aims to extract multi-dimensional semantic information by analyzing the relationships between weighted features in parallel through multiple independent attention heads. Specifically, the system splits the weighted features into multiple sub-features and uses each sub-feature head to calculate the intrinsic correlations between them. This allows each head to focus on different feature interactions. For example, one head might emphasize the relationship between speech and text, while another might connect information between image and video features.

[0043] In actual implementation, assume the system generates a weighted value for each modality feature—for example, a weight of 0.7 for speech features, 0.5 for text features, 0.6 for image features, and 0.4 for video features. The system then feeds these features into multiple attention heads, one of which might focus on the interaction between speech and text features, calculating their weighted sum to generate a fused feature representation. In this way, the system can identify which information is most important and which can be ignored, ensuring that the final fused feature highlights the core elements of the meeting content.

[0044] By concatenating or summing the features output by each attention head to generate preliminary fused features, the system is able to capture richer cross-modal information. This interactive fusion not only improves the correlation between different modalities, but also helps users better understand the content of the meeting in practical applications. For example, in a meeting about project progress, the voice may mention "we need to solve a technical problem," and the project report displayed in the image may also contain corresponding technical problem images. Through the multi-head attention mechanism, the system will ensure that this key information can be effectively integrated, helping users to retrieve meeting content more quickly and accurately.

[0045] Based on the preliminary fusion features, a decoder in the multimodal fusion neural network is used to generate a unified semantic representation vector. The preliminary fusion features are input into the decoder, and through multiple layers of nonlinear transformation and normalization, a unified semantic representation vector is obtained. The unified semantic representation vector contains the multimodal key information of the meeting minutes and retains the semantic relevance between the modalities. In this step, the system uses the decoder of the multimodal fusion neural network to further process the preliminary fusion features to generate the final unified semantic representation vector. The decoder is designed to convert the preliminary features into the final semantic representation through multiple layers of nonlinear transformations. This process typically involves multiple fully connected layers, activation functions, and normalization layers to ensure that the generated semantic representation can effectively integrate key information from each modality and preserve the semantic relevance between different modalities. For example, during the processing process, the decoder will compress the complex information of the preliminary fusion features into a more refined semantic representation through layer-by-layer adjustments, reflecting the overall content and structure of the meeting.

[0046] The generation of this unified semantic representation vector is a key step in enabling intelligent analysis of meeting transcripts. By fusing the features of each modality, the resulting semantic representation vector significantly enriches the system's knowledge base, providing a solid foundation for subsequent semantic retrieval and recommendation. The resulting semantic representation not only reflects the overall content of the meeting but also preserves the unique information conveyed by each modality. This enables the system to effectively capture underlying information relationships in complex and ever-changing meeting scenarios, thereby enhancing the intelligent processing and utilization of meeting transcripts.

[0047] In this step, the system inputs the preliminary fused features into the decoder of the multimodal fusion neural network to generate a unified semantic representation vector. The decoder is typically composed of a multi-layer fully connected network (FCN), with each layer processed by a nonlinear activation function (such as ReLU or Sigmoid) to enhance the model's expressive power. This process compresses the high-dimensional information of multimodal features into a more compact and meaningful representation, effectively capturing the key information in the meeting transcript.

[0048] For example, when the preliminary fused features are input into the decoder, the system will gradually reduce the feature dimensions through multiple fully connected layers to extract the optimal data representation. The first layer may reduce the feature dimensions from thousands to hundreds, and the second layer will reduce it to dozens, and finally generate a unified semantic representation vector of fixed length. This vector is not just a simple superposition of the features of each modality, but a holistic understanding of the content of the meeting. After model training, it can well reflect the semantic relevance between the various modalities. For example, when the theme of the meeting revolves around "product design", the generated vector may contain design-related keywords such as "user experience", "market research", "technical implementation", etc.

[0049] Next, to improve the effectiveness of the semantic representation vector, the system normalizes the output results to ensure that each eigenvalue is on the same scale. This process not only reduces the performance issues caused by differences in the magnitude of eigenvalues in subsequent tasks, but also improves the stability of the model output. For example, normalizing the output vector through the Softmax function can make the final semantic representation vector reflect the relative importance of each feature. Therefore, the unified semantic representation vector finally generated can not only comprehensively cover the multimodal key information of the meeting minutes, but also maintain the structure and consistency of the information when understanding it.

[0050] The semantic vectors whose attention weights are higher than the preset threshold are filtered out, and the filtered semantic vectors are secondary fused with the unified semantic representation vector, and finally a semantic representation vector containing key information is output.

[0051] In this step, the system screens the previously generated semantic vectors and performs a secondary fusion. Specifically, the system sets a preset attention weight threshold and compares the attention weight of each semantic vector. If a semantic vector's weight exceeds this threshold, it is considered key information and selected for the secondary fusion process. This process not only helps remove redundant information but also highlights the most important content of the meeting, ensuring that the final output semantic representation vector effectively reflects the key information.

[0052] Through screening and secondary fusion, the system enhances the information density and quality of the final semantic representation vector. This process effectively eliminates unnecessary information noise, allowing the final semantic representation vector to more efficiently serve subsequent intelligent retrieval and recommendation. Focusing key information on the final results not only improves the accuracy of subsequent analysis but also greatly enhances the user experience and efficiency when searching and utilizing meeting minutes, which is of great significance for post-meeting decision-making and information management.

[0053] In this step, the system filters all calculated semantic vectors based on the previously generated attention weights, ensuring that the final output semantic representation vector accurately reflects the key information in the meeting transcript. First, the system sets a preset threshold, such as 0.5, calculates the attention weight for each semantic vector, and retains vectors with weights greater than the threshold. This process helps the system filter out redundant information and ensures that the final output focuses on more representative content.

[0054] In practice, assume the system calculates attention weights of 0.6, 0.4, 0.7, and 0.5 for speech, text, image, and video modalities, respectively. In this case, the semantic vectors for text and video are excluded because their weights are lower than 0.5, and the system retains only the semantic vectors for speech and image modalities. This filtering ensures that the output semantic information is focused on the important parts, avoiding the influence of irrelevant information.

[0055] After the screening is completed, the system will perform a secondary fusion of the screened semantic vectors with the previously generated unified semantic representation vector. This process usually uses weighted averaging or concatenation to further strengthen these important semantic information. For example, the system may assign weights of 0.6 and 0.4 to the semantic vectors of speech and image modalities, respectively, and then add these two vectors to obtain the final output vector. Through this secondary fusion, the system ensures that the output semantic representation vector contains richer and more accurate key information, thereby providing higher accuracy and high-quality user experience in subsequent semantic retrieval or recommendation. The final output will be a semantic representation vector that integrates multimodal information and is highly relevant, providing a solid foundation for data search and analysis of meeting records.

[0056] S202: performing semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model captures temporal dependencies in the meeting minutes by introducing time series modeling technology to improve the accuracy of semantic understanding. According to the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association, wherein the context-aware model captures the temporal dependencies in the meeting minutes by introducing time series modeling technology, thereby improving the accuracy of semantic understanding. The core of this step is to further semantically enrich the generated semantic representation vector through the pre-trained context-aware model. Contextual features play a vital role in meeting minutes, because meetings are often held at specific times and occasions, and the content and topics involved are dynamically changing. Through time series modeling technology, the system can effectively identify the temporal relationship between sentences. For example, a speech may be a response to a previous speech, or a continuation of a previous discussion. The capture of such temporal dependencies will significantly improve the comprehensive understanding of the content of the meeting, so that the enhanced semantic vector finally generated contains important scene information, enhancing the ability to interpret the content of the meeting.

[0057] The enhanced semantic vector generated through this step not only maintains semantic consistency with the original semantic representation vector, but also effectively incorporates the contextual information of the meeting scene. This process ensures that the system can dynamically adjust its semantic understanding based on the context when faced with real-world scenarios. For example, in a technical discussion meeting, a certain speech involves a specific technical issue, and subsequent speeches may focus on this issue. Combined with contextual information, the system can accurately identify the speech content related to the technical issue, improve the accuracy and relevance of information during retrieval, and thus greatly improve the efficiency of users' search for meeting records.

[0058] Specifically, the context information of the meeting scene can be extracted and encoded into a context feature vector; In this step, the system identifies and extracts contextual information about the meeting scene, converting it into a contextual feature vector. This contextual information includes not only the meeting topic, participants, and discussion content, but also contextual factors such as time, location, and participant emotions. For example, in a team planning meeting, participants might discuss a new product's marketing strategy. The system needs to consider these factors, encode them, and convert them into a contextual feature vector. This representation provides important background information beyond the meeting content, facilitating subsequent semantic processing.

[0059] The introduction of contextual feature vectors enables the system to better capture meeting-related details during semantic enhancement, which positively impacts subsequent semantic understanding and information retrieval. By incorporating dynamically changing contextual information into semantic understanding, the system can generate more customized information retrieval results based on different scenarios, thereby improving user experience and information relevance.

[0060] In this step, the system first needs to identify and extract contextual information related to the meeting scenario. This includes the meeting's topic, time, location, participants, and key points discussed. This information can be identified by analyzing keywords and phrases mentioned in the meeting transcript. For example, in a meeting about new product development, keywords such as "product prototype," "market research," and "competitive analysis" might be mentioned. The system uses natural language processing technology to extract these keywords from the meeting transcript, forming a preliminary set of contextual information.

[0061] Next, the system encodes the extracted context information according to a preset structure to generate a context feature vector. During this process, word embedding technology can be used to convert the extracted keywords into vector representations. For example, models such as Word2Vec or GloVe can be used to convert each keyword into a numerical vector, and a feature vector representing the entire context information is generated by taking the mean or weighted average. Through this processing, the system can ensure that the context information is not only recognized, but also can be used in a numerical form for subsequent calculations and analysis.

[0062] Finally, the system needs to be supplemented with additional contextual information, such as participant sentiment and engagement. This can be accomplished by analyzing the emotional tendencies of speech during the meeting. For example, sentiment analysis tools can be used to categorize speech and quantify positive, negative, and neutral sentiment. By combining this emotional information with basic meeting information, the resulting contextual feature vector will fully reflect the meeting's contextual characteristics, providing rich context for subsequent semantic enhancement processing.

[0063] The semantic representation vector and context feature vector are input into a pre-trained context-aware model. The context-aware model captures the temporal dependencies in the meeting transcripts by introducing time series modeling techniques. The context-aware model dynamically adjusts the semantic representation vector based on the time series information to generate a context-relevant intermediate semantic vector. In this step, the system leverages a pre-trained context-aware model to combine the extracted semantic representation vectors with contextual feature vectors. Through time series modeling and analysis, the system captures the temporal dependencies within the meeting transcripts. Trained on historical data, the model identifies relationships between different time points and adjusts the representation vectors to the appropriate context. This dynamic adjustment allows contextual features to be fully utilized, resulting in a more semantically rich and accurate intermediate semantic vector.

[0064] By introducing a context-aware model, the system can not only understand static semantic content but also grasp the dynamic changes of information over time. This capture of temporal dependencies significantly improves the understanding of meeting minutes, allowing the system to better reflect the context of information, thereby improving the relevance and accuracy of information retrieval.

[0065] In this step, the system first merges the extracted semantic representation vector with the previously generated context feature vector and provides it as input to a pre-trained context-aware model. This model, typically based on deep learning techniques, automatically learns the underlying temporal dependencies in the input data. The system employs structures such as LSTM (Long Short-Term Memory) networks or Transformers, leveraging their regression capabilities to effectively identify the relationship between individual statements and preceding and following statements in meeting transcripts.

[0066] During training, the model learns the temporal characteristics of speech based on historical data. For example, if a participant in a meeting mentions "user feedback" while discussing product features and then later discusses "improvement plans," the model can capture the relationship between the two and effectively improve its semantic understanding of the subsequent content. The system dynamically adjusts the input vector through temporal modeling to generate intermediate semantic vectors. During this process, the model is continuously updated and optimized to adapt to changing contexts.

[0067] Finally, the resulting intermediate semantic vector carries contextual information. This vector not only preserves the content characteristics of the original semantic representation but also reflects its dynamic connection to the context, making subsequent semantic enhancement more accurate. This process ensures that, in real-world applications, the system not only understands information statically but can dynamically adjust and optimize its representation based on changing context.

[0068] Based on the intermediate semantic vector and the context feature vector, an attention mechanism is used to perform semantic enhancement processing, wherein the attention weight of the context feature vector to the intermediate semantic vector is calculated to generate a weighted context semantic vector; In this step, the system combines the intermediate semantic vector and the contextual feature vector using the attention mechanism. The attention mechanism is a key concept in deep learning that dynamically adjusts the weights of various components based on the importance of the input data. Here, the system first calculates the attention weights of the contextual feature vector on the intermediate semantic vector and then generates a weighted contextual semantic vector based on these weights. This allows the system to precisely focus on key information and improve the quality of semantic representation.

[0069] This step introduces an attention mechanism, enabling the system to adaptively emphasize the role of specific information in the context when performing semantic enhancement. By calculating attention weights, the system better identifies which features in a given context are critical to understanding the meeting content, thereby generating more accurate contextual semantic vectors and ultimately achieving higher-quality semantic enhancement results.

[0070] In this step, the system uses an attention mechanism to enhance the intermediate semantic vector and contextual feature vector. First, the system calculates the similarity between the contextual feature vector and the intermediate semantic vector to determine which contextual information has a greater impact on the semantic representation. Typically, this similarity can be calculated using methods such as dot product or cosine similarity. For example, if a speech in a meeting mentions "market research" frequently, the system will assign a higher attention weight to the "market research" information based on the similarity of this feature vector.

[0071] Next, based on the calculated attention weights, the system weights the intermediate semantic vectors. During this process, contextual features with higher weights will have a greater impact on the final generated intermediate semantic vector. For example, stronger attention weights will strengthen features related to "market research" in the final contextual semantic vector, while features with lower weights will be weakened. This process effectively improves semantic accuracy and relevance.

[0072] Finally, the resulting weighted contextual semantic vector incorporates optimized contextual information and intermediate semantic features. This vector not only reflects the relationship between modalities but also captures the impact of context on semantics, making the subsequent fusion phase more precise. This approach ensures that the system fully leverages contextual information when processing meeting transcripts, generating semantic representations that better meet practical needs.

[0073] The weighted contextual semantic vector is fused with the intermediate semantic vector to generate an enhanced semantic vector with contextual association. The enhanced semantic vector contains the contextual information of the meeting scene and retains the global information of the semantic representation vector.

[0074] In this step, the system effectively fuses the weighted contextual semantic vector with the intermediate semantic vector to generate an enhanced semantic vector with contextual relevance. Through this fusion, the system can integrate the contextual information and semantic information from the previous step to form a more comprehensive and accurate representation, which is convenient for subsequent semantic processing and retrieval.

[0075] This enhanced semantic vector not only incorporates contextual information about the meeting scene, but also further improves semantic accuracy and relevance while maintaining the original global information of the semantic representation vector. This will make subsequent semantic analysis and information retrieval more intelligent and effective, helping users find the meeting records and information they need more quickly and accurately.

[0076] In this step, the system fuses the weighted contextual semantic vector and the intermediate semantic vector into a unified enhanced semantic vector. The system needs to first select an appropriate fusion strategy, which can be achieved using a variety of methods such as weighted summation, concatenation, or splicing. For example, if weighted summation is selected, the system will set a fusion weight to balance the influence of the weighted contextual semantic vector and the intermediate semantic vector. Assuming that the weight of the contextual semantic vector is set to 0.6 and the weight of the intermediate semantic vector is 0.4, the system will perform a weighted sum of the two vectors according to this ratio to generate the final enhanced semantic vector.

[0077] The system then needs to perform normalization to ensure that the generated enhanced semantic vectors remain numerically consistent. This normalization step can be achieved through methods such as L2 normalization to avoid the impact of large or small values that may occur in subsequent calculations. Ultimately, the enhanced semantic vectors after this process will be more stable, providing a good foundation for subsequent analysis.

[0078] Finally, the generated enhanced semantic vector integrates contextual information, ensuring that it reflects the meeting scenario while retaining global semantic information. This vector not only provides input for subsequent knowledge graph construction and data retrieval, but also significantly improves the system's accuracy and efficiency in understanding and analyzing meeting content. For example, in a meeting discussing product strategy, the enhanced semantic vector will clearly reflect the key discussion points, relevant context, and their temporal relationships, making subsequent knowledge mining and refinement in this area smoother.

[0079] S203: Based on the enhanced semantic vector and in combination with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the conference topic in real time. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. In this step, based on the enhanced semantic vectors generated previously, the system uses a dynamic knowledge graph construction algorithm to generate a knowledge graph related to the meeting topic in real time. This process first inputs the enhanced semantic vectors into a pre-trained entity recognition model. The model then analyzes the key information extracted from the meeting minutes to identify important entities related to the meeting topic. These entities represent the core concepts and relationships in the meeting discussion, such as participants, topics, and decisions. Next, the extracted entities and relationships are integrated to form an initial knowledge graph. The system also uses an incremental learning mechanism to continuously update the nodes and edges in the knowledge graph to ensure the timeliness and completeness of the knowledge graph. This mechanism allows the system to dynamically adjust the graph at different stages of the meeting, providing continuously updated knowledge support for future decisions and discussions.

[0080] By introducing a dynamic knowledge graph construction algorithm, this method significantly improves the intelligence and practicality of meeting record data search. The real-time generated knowledge graph not only reflects the key content of the meeting process, but also continuously evolves with the introduction of new information, ensuring the accuracy and timeliness of relevant data. For example, as the meeting progresses, new decisions or opinions may be proposed. The system can instantly capture these changes and incorporate them into the knowledge graph, so that in subsequent information retrieval, users can obtain more accurate and representative meeting records and knowledge. Ultimately, this process not only improves the availability of information, but also provides valuable reference for meeting participants and decision makers.

[0081] Specifically, the enhanced semantic vector can be input into a pre-trained entity recognition model to extract a set of key entities in the meeting minutes, where each entity contains entity type and context information; In this step, the system first inputs the generated enhanced semantic vectors into a pre-trained entity recognition model. This model has been trained on a large amount of data and can accurately identify various entities in text, such as organizations, individuals, or events. In meeting records, the enhanced semantic vectors contain rich information, which the system uses to identify important entities mentioned in the meeting. Through model analysis, each identified entity is labeled as a specific type and attached with corresponding contextual information. This helps to understand the specific role and significance of these entities in the meeting discussion, such as whether a participant may be related to the speech on a specific topic.

[0082] The execution of this step lays a solid foundation for the subsequent construction of the knowledge graph. By extracting a set of key entities, the system can accurately capture the core content of the meeting, making subsequent analysis and processing more effective. For example, when processing meeting records related to product development, identified entities such as "product manager" and "technical team" will guide subsequent decision makers to better understand the context of the discussion. This also means that the system can reflect the contributions of each participant and related discussion topics in real time during the dynamically changing meeting process, providing direct support for knowledge accumulation and decision-making.

[0083] When implementing this step, the system first needs to process the enhanced semantic vector into a format suitable for input into the pre-trained entity recognition model. To this end, the system will perform feature extraction on the enhanced semantic vector and construct a feature matrix containing key information. This step involves standardizing the semantic vector to ensure that its value does not fluctuate when the model is processed. Then, the input feature matrix will be passed to the pre-trained deep learning model layer by layer so that the model can recognize the entities therein. Suppose the system is processing a meeting record about a new product launch. The enhanced semantic vector may contain information such as "product manager", "market analysis", and "user feedback". The pre-trained model will be able to extract the corresponding entities from this information.

[0084] By inputting the enhanced semantic vector, the entity recognition model analyzes the context and identifies key entities in the meeting. For example, if the meeting discussed "new product design," the model might identify entities such as "designer," "team goals," and "project timeline." Each identified entity is not only labeled with a specific entity type, such as "position" or "project," but also includes contextual information (such as the time of first mention and the content of the discussion), forming a detailed entity set. This entity set lays the foundation for subsequent relationship extraction and knowledge graph construction.

[0085] Finally, the system stores the extracted entity set in a structured format for subsequent access. This structured data, which may be stored in JSON or a database, is queryable and updateable, supporting other parts of the system. In subsequent steps, the system extracts links and relationships from external knowledge bases based on these entities. This reliable storage method also ensures that entity information can be dynamically updated in the future and reflected in the knowledge graph in real time.

[0086] The enhanced semantic vector and the extracted entity set are input into the pre-trained relation extraction model to extract the relationship set between entities, where each relationship contains the relationship type and the entity pair involved; In this step, the system inputs the obtained enhanced semantic vectors, along with the extracted key entity set, into a relation extraction model. This relation extraction model uses advanced machine learning algorithms to identify relationships between different entities in a text. For example, by learning associated vocabulary and semantic structures, the model can identify relation types such as "discuss," "suggest," or "oppose." Through this analysis, the system effectively identifies interactions between participants, connections between different topics, and the logical relationships between various decisions and recommendations, forming a rich set of relationships.

[0087] This process provides the necessary structure for building more complex knowledge graphs. By extracting entity relationships, the system not only identifies individual entities but also understands the dynamic interactions between them. For example, in a meeting related to market strategy, identifying the relationship "Product Manager – Proposal – Market Research Suggestion" can help the team understand the source and context of the decision-making advice, which is crucial for subsequent discussions and decision analysis. This dynamic presentation of relationships makes the knowledge graph more vivid and useful, truly reflecting the flow of information and interactive relationships during the meeting.

[0088] In this step, the system first combines the enhanced semantic vector with the set of key entities extracted in the previous step to form an input dataset containing rich context. Using the deep learning relationship extraction model, the system can analyze the interaction between the two datasets. This process usually involves preprocessing the input data to ensure that the necessary context and semantic information can be effectively transmitted. For example, for a specific meeting record, the system will integrate the contextual information of the two entities "Product Manager" and "Market Analysis" to form an input sample for relationship extraction, which may be a statement such as "The product manager recommends market analysis."

[0089] Next, the pre-trained relationship extraction model analyzes the input data to identify potential relationships between entities. The system automatically labels different relationship types, such as "suggestion," "collaboration," or "feedback." For example, for "product manager" and "market analysis," the model might identify the "proposal" relationship and output "product manager – proposal – market analysis." The success of this extraction process often depends on the quality of the model training and the richness of the input data. Therefore, in practical applications, the model needs to be trained on a large amount of real-world meeting data to improve its accuracy.

[0090] Finally, the extracted set of relationships is stored in the system's database along with the entity set, forming the foundation for subsequent knowledge graph construction. Each relationship not only contains entity pairs but also stores contextual information about the relationship, allowing users to quickly gain a detailed understanding of these relationships when they query them. Successful implementation of this step helps build a dynamic knowledge graph, enabling the system to continuously update and expand its knowledge base during real-time meetings, ensuring the accuracy and timeliness of the information.

[0091] Based on the entity set and relationship set, the entities and relationships are aligned. Based on the aligned entities and relationships, in combination with the external knowledge base, entity linking and relationship expansion are performed. The relevant entities and relationships in the external knowledge base are mapped to the current meeting scenario to generate the initial knowledge graph. In this step, the system needs to align the extracted entity and relationship sets with the external knowledge base. This process is to ensure that the information in the knowledge graph is not only relevant to the current meeting, but also consistent in the broader knowledge context. The alignment process involves comparing the identified entities with known entities in the external knowledge base to determine whether they have the same meaning or refer to the same object. At the same time, the system can also extract relevant relationships and map them into the knowledge graph to ensure that the information recorded in the graph is comprehensive and accurate.

[0092] By aligning entities and relationships, the system can integrate the content discussed in the meeting with a wider range of knowledge, enhancing the depth and breadth of the knowledge graph. This allows users to not only retrieve the content within the meeting minutes, but also access a richer set of external information during subsequent data queries and extractions. For example, when discussing product market strategies, linking to relevant market analysis reports in external knowledge bases not only strengthens the basis for decision-making but also provides participants with a broad range of perspectives and background knowledge. This linking gives the knowledge graph greater application value and can provide users with multi-level data support.

[0093] In this step, the system's task is to align the internally extracted entities and relationships with the external knowledge base to ensure information consistency. First, the system will feature the extracted entities and compare their feature vectors with the entity features in the external knowledge base. This may be achieved through some similarity calculation methods, such as cosine similarity or Manhattan distance, which can effectively find entities with similar meanings and align them. For example, when processing the entity "product manager", the system may look for entities in the external knowledge base that also point to product management positions, such as "product director" or "project leader".

[0094] Once alignment is complete, the system performs entity linking based on the identified similarities, enriching the endogenous data by introducing information from external knowledge bases. During this process, the system also checks the relationships between entities to ensure that all relationships are connected in an appropriate manner. For example, if an external knowledge base contains a detailed explanation of "market analysis," but only brief information about the entity is provided in the meeting minutes, the system will link the relevant information from the external knowledge base to ensure the integrity of the knowledge graph.

[0095] Finally, based on the aligned entities and relationships, the system generates a preliminary knowledge graph. This knowledge graph integrates information from various sources into a structured graph, which can be displayed as nodes and edges. When users query, this preliminary knowledge graph allows them to quickly access information related to the meeting topic, thereby facilitating efficient decision-making. This step not only increases the information richness of the knowledge graph but also enhances its effectiveness, allowing users to extract more comprehensive and valuable data from it after the meeting.

[0096] Based on the real-time input of meeting minutes, the initial knowledge graph is dynamically updated using an incremental learning mechanism. New entities and relationships are detected and added to the initial knowledge graph. A node embedding update algorithm is then used to dynamically adjust the representation of nodes and edges in the initial knowledge graph to ensure the timeliness of the knowledge graph. During this step, the system monitors the input of meeting transcripts in real time and dynamically updates the initial knowledge graph through an incremental learning mechanism. The system analyzes new speeches during the meeting and identifies newly added entities and relationships. This new information is integrated into the existing knowledge graph, ensuring it reflects the latest discussions and decision-making changes. Simultaneously, the system utilizes a node embedding update algorithm to dynamically adjust the representation of existing nodes and edges to maintain the graph's currency and accuracy.

[0097] This step ensures that the knowledge graph evolves from a static state to a dynamic state, adapting to user needs and environmental changes in real time. By continuously updating the knowledge graph, the system can continuously improve the relevance and accuracy of information, providing users with deeper insights. For example, if a new project or decision change arises during a meeting, the system can immediately identify and update the relevant nodes, quickly reflecting the impact of this information on the decision. This flexible update mechanism makes the system more intelligent and user-friendly, greatly facilitating user application in real work.

[0098] When implementing this step, the system will build a dynamic learning mechanism to ensure that the knowledge graph can respond to changes in meeting input in real time. When a new meeting record is collected, the system will first run the entity recognition and relationship extraction model to analyze the new input to extract the newly added entities and relationships. The system will compare it with the existing knowledge graph to detect whether there is new information that needs to be added. For example, if the newly joined "sales team" is mentioned in the latest meeting, and this information is not in the previous knowledge graph, the system will identify it as a new entity.

[0099] Once new information is identified, the system stores and processes it through an incremental learning mechanism. This means the system not only adds new entities and their relationships to the knowledge graph, but also continuously updates background knowledge and adjusts existing nodes and edges. The node embedding update algorithm plays a key role here—it dynamically adjusts the representation of each node based on new data, ensuring that the importance and relevance of each node in the knowledge graph reflect the latest state.

[0100] Finally, the successful implementation of this dynamic update process ensures the timeliness of the knowledge graph. The system can effectively maintain and update the knowledge graph, ensuring that it remains accurate and up-to-date while reflecting the flow of information during meetings. In this way, the knowledge graph is no longer just a static presentation tool, but can adapt to rapidly changing work scenarios, allowing participants to access the most relevant and up-to-date information in every meeting, greatly improving work efficiency.

[0101] According to the changes in conference topics, the knowledge graph is pruned and reconstructed using a graph optimization algorithm. This involves removing redundant nodes and edges to ensure the simplicity of the knowledge graph, merging similar entities and relationships to ensure the integrity of the knowledge graph, and ultimately generating an optimized knowledge graph.

[0102] In this step, the system optimizes the existing knowledge graph based on the changing topics of the conferences. By introducing a graph optimization algorithm, the system analyzes the nodes and edges in the knowledge graph, identifying and removing redundant information. For example, if a node appears repeatedly in multiple conferences but contains no new information, the system marks it as redundant and removes it from the graph. At the same time, the system examines the relationships between different nodes, particularly those representing similar entities or identical relationships, so that they can be merged. Through this process, the system ensures that the knowledge graph remains efficient and accurate while improving its decision-making support capabilities.

[0103] By pruning and reconstructing the knowledge graph, the system can effectively improve the usability and responsiveness of the knowledge graph. A concise knowledge graph not only makes information retrieval and analysis more efficient, but also reduces the complexity of data processing and the consumption of computing resources. Furthermore, the process of merging similar entities and relationships ensures the integrity of the knowledge graph, allowing users to obtain focused information on relevant topics when querying, rather than being overwhelmed by redundant content. This efficient and precise information structure allows users to easily obtain the information they need even when the topic of a meeting changes, greatly improving work efficiency.

[0104] In this step, the system uses graph optimization algorithms to process the knowledge graph, pruning and reconstructing it. First, the system analyzes the structure of the current knowledge graph and identifies redundant nodes and edges. For example, duplicate participant entities or similar discussion topics may appear in multiple meeting transcripts. The system uses network analysis techniques, such as community detection algorithms, to identify this redundant information and mark it as deletable. This process not only reduces information duplication but also improves knowledge graph processing efficiency.

[0105] Next, the system merges similar entities and relationships. Using a clustering algorithm, the system groups entities with similar functions and merges them to create a single representative entity. This approach ensures that the information reflected in the knowledge graph is neither distorted nor redundant. For example, if there are multiple nodes in the knowledge graph representing "Product Manager" and "Product Director," but they actually refer to the same role, the system will automatically merge these two nodes into one. This process improves the cleanliness of the graph and ensures the centralized and effective presentation of information.

[0106] Finally, the pruned and merged knowledge graph undergoes a final structural optimization to ensure it can effectively support user queries. The system then uses visualization technology to present the optimized knowledge graph to users, providing an easy-to-understand graphical interface that allows them to quickly find the content they need amidst complex information. This optimization process significantly enhances the user experience of the knowledge graph, enabling users to quickly access the most relevant information across various conference topics while ensuring data integrity and accuracy.

[0107] S204: Based on the knowledge graph, a graph neural network is used to intelligently retrieve and recommend meeting record data. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

[0108] In this method, intelligent retrieval and recommendation of meeting minutes data using a graph neural network (GNN) is achieved by incorporating the user's query intent. First, the system performs natural language processing on the user's query, parsing the user's specific needs to form a query vector. Next, the knowledge graph is fed into the GNN as input, initializing feature representations for entity nodes and relationship edges. The GNN dynamically adjusts the search path based on the query vector. By calculating the similarity between the query vector and each entity node and generating node weights, it selects entity nodes and relationship edges that are highly relevant to the query intent. This approach makes the retrieval process more efficient and intelligent, precisely matching user needs. Using a graph neural network for intelligent retrieval and recommendation not only improves information retrieval accuracy but also enhances the user experience. When users can quickly find meeting minutes and related recommendations that closely match their query intent, the entire information retrieval process becomes smoother and more efficient. Furthermore, by dynamically adjusting the search path, this method ensures that users always obtain the latest and most relevant information, even as the knowledge graph is constantly updated. This flexibility enables the system to adapt to changing conference topics and user needs, truly realizing intelligent data management and application.

[0109] Specifically, the query intent can be analyzed based on the query statement entered by the user using natural language processing technology, and the query intent can be represented as a query vector; In this step, the system receives the user's query and applies advanced natural language processing (NLP) technology to parse the query intent. First, through the text parsing module, the system will segment the user's query, remove stop words, and then identify keywords and phrases. This process may involve technologies such as named entity recognition and part-of-speech tagging to ensure that the system can accurately capture the user's intent. For example, if the user enters the query "meeting minutes on new product launches," the system will extract the keyword "new product launches" and understand that the user's primary focus is on product and launch information.

[0110] The system then converts the extracted keywords into a query vector, a high-dimensional numerical representation that captures semantic information. The resulting query vector incorporates contextual and semantic information, enabling effective matching with entity nodes in the knowledge graph. This representation can be achieved using word embedding techniques (such as Word2Vec or BERT), making the query vector not only structured but also relevant to the nodes in the knowledge graph.

[0111] By parsing user queries into query vectors, the system achieves a deep understanding of user needs, thereby enhancing the accuracy and interactivity of retrieval. This process enables information retrieval to move beyond keyword matching to a more comprehensive understanding of semantics, thereby improving the efficiency with which users find the information they need. For example, if a user searches for "new product release," the system can link all meeting minutes related to the topic, rather than simply matching text. This intelligent query parsing lays the foundation for subsequent retrieval and recommendations, significantly improving the user experience.

[0112] In this step, the system first receives the user's query and preprocesses it. The user's query may contain a variety of information, and the system uses a word segmentation algorithm to break the input text into individual terms. For example, if the user enters the query "meeting minutes regarding new product launches," the system will break it down into terms such as "about," "new product," "release," and "meeting minutes." The system then performs stop word removal, removing meaningless words like "about," while retaining the key words "new product," "release," and "meeting minutes."

[0113] The system then converts these keywords into word vectors, typically using models such as Word2Vec, GloVe, or BERT for word embedding. Taking BERT as an example, the system inputs each retained keyword into the model to generate a corresponding high-dimensional vector representation. Each vector not only contains the semantic information of the word but also takes into account the contextual relationship, so that the correlation between words can be reflected. Assuming that the phrase "new product" is highly correlated with "innovation" in a certain context, the BERT model will be able to capture this relationship. Ultimately, these word vectors will be aggregated into an overall query vector that represents the user's complete query intent.

[0114] Finally, the system normalizes the generated query vector to ensure it can be effectively matched with entity nodes in the knowledge graph during subsequent searches. This process may include normalization and dimensionality adjustment to maintain a consistent numerical range. This process ensures that subsequent similarity calculations accurately reflect the user's query intent, laying a solid foundation for subsequent searches.

[0115] Input the knowledge graph into the graph neural network to initialize the feature representation of entity nodes and relationship edges; In this step, the system imports the structural information of the entire knowledge graph into a graph neural network (GNN). A knowledge graph is essentially composed of nodes (entities) and edges (relationships), and GNNs can leverage this graph structure to perform complex reasoning and computation. First, the system initializes the features of each entity node and relationship edge in the knowledge graph. For example, an entity node may contain type, attributes, and contextual information, while a relationship edge may contain information about the interaction between two entities, such as weights.

[0116] For each node, the system generates a feature vector, ensuring that these features not only reflect the node's basic information but also capture its relative position and importance within the entire graph. Furthermore, the system uses network embedding techniques to map relationship edges into vectors, effectively integrating edge features with node features. Through this process, GNNs provide an important foundation for subsequent retrieval and recommendation.

[0117] This step provides the necessary structure and information for further computation within the graph neural network. By initializing the feature representations for entity nodes and relationship edges, the GNN can better adapt to the user's query intent during subsequent reasoning and computation. This flexibility and adaptability means the system can more accurately match user needs. For example, when searching for a specific meeting topic or discussion content, it can quickly identify relevant entities and relationships, thereby improving the accuracy and relevance of the information.

[0118] In this step, the system first converts the knowledge graph's data structure into a format that can be processed by a graph neural network (GNN). A knowledge graph consists of numerous entity nodes and the edges between them. The system first traverses all nodes in the knowledge graph and extracts the basic features of each node. These features may include node type (such as "product," "meeting," "person," etc.), attributes (such as product price, meeting time, etc.), and contextual information. This generates a feature matrix, laying the foundation for subsequent GNN operations.

[0119] Next, the system assigns a feature representation to each relationship edge. For example, the edge's features can be represented as the strength of the interaction between two entities or the type of connection. This process involves parsing the attributes of the edges to ensure that the necessary contextual information is provided for each edge. In this way, the edge's features can effectively supplement the node information and help the system better understand the associations between entities. For example, in the relationship between "Product Manager" and "Market Analysis", the characteristics of the relationship may be reflected as "Responsible" and "Analysis", helping to build a richer network structure in real time.

[0120] Finally, the system inputs the feature representations of all initialized entity nodes and relationship edges into the graph neural network. Through the GNN's propagation and aggregation mechanisms, high-dimensional vector representations of the nodes and edges are formed. During this process, the GNN optimizes the feature representations, gradually extracting the contextual relationships between nodes and edges across the entire graph through multiple layers of nonlinear transformations. These optimization steps significantly improve the quality of subsequent retrieval and recommendations, enabling the system to quickly identify and respond to user needs within complex data structures.

[0121] Based on the query vector, the adaptive path search algorithm is used to dynamically adjust the retrieval path in the graph neural network; In this step, the system utilizes an adaptive path search algorithm to dynamically adjust the search path based on the query vector entered by the user. The user's query vector not only represents the user's intent but also includes similarities with each node in the knowledge graph. Based on this information, the system optimizes the search process to ensure efficient and accurate data retrieval. The adaptive path search algorithm analyzes the similarity between the user's query vector and the feature representations of each node in the knowledge graph to determine the optimal search path.

[0122] This dynamic path adjustment mechanism means that when processing specific queries, the system doesn't just rely on a fixed search path, but can adjust in real time based on the current query needs. When the system detects that the user's query intent has a high degree of similarity with certain specific nodes, it will prioritize exploring paths related to those nodes rather than following a fixed query process. For example, when a user searches for "market analysis report," the system will adjust its search path to directly access nodes highly related to "market analysis" and collect relevant information.

[0123] This process significantly improves the flexibility and accuracy of information retrieval. By dynamically adjusting the search path, the system can quickly find the most relevant entities and information based on the user's specific needs. This adaptability not only enhances the user experience but also strengthens the system's ability to handle complex queries, ensuring users receive information quickly and efficiently. This makes the retrieval process more user-friendly, ensuring users can obtain the information they need regardless of the situation, thereby more effectively supporting decision-making.

[0124] In this step, the system combines the query vector with the structure of the graph neural network to implement an adaptive path search algorithm. First, the system calculates the similarity between the query vector and each entity node in the knowledge graph. Common calculation methods include cosine similarity and Euclidean distance. In this way, the system can identify entity nodes that are highly relevant to the user's query intent. For example, if the user's query intent is "marketing meeting", the system will calculate the similarity between the query vector and all "marketing" related nodes in the knowledge graph to find the most matching node.

[0125] Once the similarity list is generated, the system assigns a weight to each entity node, with more relevant nodes receiving higher weights. Next, an adaptive path search algorithm analyzes these weights and dynamically selects the optimal path for search. This means that rather than invariably traversing all nodes, the system focuses on the nodes most relevant to the query intent and their adjacent edges. Once high-weighted nodes are identified, the system automatically generates a search path, prioritizing these entities to obtain information directly relevant to the user's query intent.

[0126] Finally, the system performs information retrieval based on the generated search path, collecting the information represented by the nodes and their connected edges. This process ensures efficient and accurate retrieval, allowing users to quickly obtain relevant meeting minutes and their background information. For example, if the path search identifies a high-weight node for "Marketing Meeting," the system will directly retrieve meeting minutes related to that node, significantly reducing search time and improving search efficiency.

[0127] Calculate the similarity between the query vector and each entity node, generate node weights, and dynamically select entity nodes and relationship edges that are highly relevant to the query intent based on the node weights to generate a retrieval path; In this step, the system calculates the similarity between the user's query vector and each entity node in the knowledge graph and generates node weights based on this. Similarity calculations typically use methods such as cosine similarity or Euclidean distance to quantify the similarity between the query vector and the entity node features. The system converts these similarity values into weights, with nodes with higher weights indicating greater relevance to the user's query intent. This allows the system to prioritize nodes that are highly relevant to the user's intent for subsequent processing.

[0128] After the calculations are complete, the system dynamically selects entity nodes and relationship edges based on the weights, forming an optimized search path. This path encompasses not only relevant entity nodes but also the edges (i.e., relationships) between them, ensuring efficient multi-dimensional querying of the knowledge graph. For example, if the user's query intent is "latest product development," the system will be more inclined to select nodes related to "product development," ensuring a precise and efficient search path.

[0129] This process provides a scientific basis and data support for constructing retrieval paths. Dynamically selecting entity nodes and relationship edges that are highly relevant to the query intent not only improves the accuracy of information retrieval but also ensures that users can quickly access relevant knowledge. This approach enables the system to efficiently respond to specific queries, optimize the user experience, and enhance the timeliness and effectiveness of information acquisition.

[0130] In this step, the system first calculates the similarity between the query vector and each entity node in the knowledge graph. This process compares the query vector with the feature vector of each node and uses a similarity calculation formula, such as cosine similarity or dot product, to evaluate the degree of association between them. Based on the calculation results, the system generates a similarity matrix that records the similarity value between each node and the query vector. Assuming the user's query is about "Annual Sales Summary", the system will identify nodes related to this topic, such as "Sales Report", "Sales Team Meeting", etc.

[0131] The system then converts these similarity values into weights, with nodes with higher weights indicating greater relevance to the user's query intent. The resulting node weights are used to determine the search path. The system prioritizes high-weighted nodes and their directly connected edges to ensure that the retrieved information is as relevant as possible to the user's query. The weight calculation process considers not only the node itself but also other nodes and relationships directly connected to it, ensuring the validity of the entire search path.

[0132] After the system generates node weights, it dynamically selects entity nodes and relationship edges to construct a search path. By selecting the nodes most relevant to the query intent, the system ensures accurate and effective information, avoiding unnecessary traversals and redundant data. For example, if a user queries "quarterly financial analysis," the system will select nodes such as "financial report" and "budget review" based on their weights, forming an effective search path to quickly retrieve relevant information. This dynamic selection mechanism significantly reduces query response times while improving the relevance and accuracy of information.

[0133] Extract relevant meeting record fragments from the knowledge graph based on the retrieval path; In this step, the system leverages the previously generated search path to extract meeting transcript snippets from the knowledge graph that are relevant to the user's query. This process involves in-depth analysis of each node in the path to capture information closely related to the query intent. Based on node weights and associations, the system identifies all transcripts connected to the appropriate nodes and aggregates them into a set of results. Each meeting transcript snippet may include information such as the meeting's topic, participants, discussions, and decisions, providing users with detailed insights into the meeting.

[0134] During the extraction process, the system ensures the completeness and context of the information, taking into account the interrelationships between meeting notes. For example, if a node represents a "marketing strategy discussion," the system extracts all meeting notes related to that node and aggregates them to provide users with a comprehensive view. This extraction process is automated, and the system efficiently obtains the necessary information from the knowledge graph to quickly respond to user queries.

[0135] By extracting relevant meeting transcript snippets from the knowledge graph, the system not only provides users with direct answers but also ensures the accuracy and relevance of the information. This process enables users to quickly find all meeting content that matches their query, making it easy to obtain the required data in a complex information environment. This efficient information extraction mechanism enhances users' decision-making capabilities and makes the system more intelligent and user-friendly in managing and querying meeting transcripts.

[0136] In this step, the system leverages the search path generated in the previous step to perform in-depth information extraction for each entity node. The system first traverses each node along the path, extracting relevant meeting record fragments from each node. These records may include information such as the meeting's topic, participants, discussions, and decisions, ensuring that the extracted content fully reflects the core of the meeting. For example, if the path includes the "Financial Report" node, the system will extract all meeting records related to that node, forming a detailed set of information on "financial status."

[0137] During the extraction process, the system leverages the metadata stored in the knowledge graph to ensure that the information obtained includes both directly relevant content and possible contextual connections. For example, if a "Financial Report" node is connected to a "Sales Summary" node, the system extracts both types of information simultaneously, allowing users to gain a comprehensive understanding of the company's overall financial performance. This step aims to ensure the diversity and contextual integrity of the information, allowing users to form a more systematic understanding of the meeting discussion.

[0138] Finally, the system consolidates all extracted meeting record fragments into a specific summary or report for easy user review. This aggregation can be customized to meet user needs, such as arranging meeting records chronologically or categorizing them by participants. This allows users to quickly find all necessary information relevant to their query, improving query efficiency. This aggregation process enhances the intelligence of data extraction, allowing users to easily obtain the data they need in complex information environments.

[0139] Based on the retrieval path and meeting record fragments, a graph neural network is used to generate relevant recommended content. The recommended content includes entities, relationships, and meeting topics related to the query intent. Finally, the meeting record fragments and related recommended content that match the query intent are output.

[0140] In this step, the system uses a graph neural network (GNN) to perform deep learning on previously extracted meeting transcript snippets and search paths to generate recommendations relevant to the user's query intent. First, the system analyzes the generated search paths and identifies entities and relationships related to the meeting transcript snippets. Using the GNN, the system further mines latent relationships within the knowledge graph, identifying additional entities and their interrelationships that are highly relevant to the user's query intent, such as similar meeting topics, participants, and background information. This process not only enhances the relevance of the content but also provides a broader perspective. During the recommendation generation process, the system uses the GNN to infer additional information related to the original meeting transcript snippets. For example, if a user queries for meeting transcripts related to "new product launches" and extracts relevant snippets, the GNN considers entities and relationships in the knowledge graph related to "new product" or "release plans" to generate possible recommendations, such as "market feedback," "competitive analysis reports," or links to related conferences. This network aggregates features across nodes through an information propagation mechanism, resulting in richer and more diverse recommendations.

[0141] This process not only enhances the user's information acquisition experience but also strengthens the system's intelligent recommendation capabilities. By leveraging graph neural networks to generate relevant recommendations, the system can provide more diverse information based on user queries, helping users understand the query from a broader perspective. This recommendation approach effectively improves user decision-making efficiency, ensuring that users can discover potentially valuable supplementary information while accessing specific meeting minutes. This intelligent recommendation mechanism makes the system more user-friendly and significantly enhances its responsiveness to user needs.

[0142] In this step, the system uses the search path and the extracted meeting transcript snippets to generate recommendations relevant to the user's query intent using a graph neural network (GNN). First, the system encodes the key entities and relationships in the extracted meeting transcript snippets into the GNN input data. This input data contains information about the user's query intent and the entities and relationships extracted from the relevant records. The system then encodes each element for processing within the GNN.

[0143] Next, the system uses the GNN's node propagation mechanism to analyze the entities and relationships within the current meeting minutes, thereby identifying potentially relevant entities and topics. This process involves feature updates and information aggregation within the graph neural network. Based on node connections, the system identifies other entities highly relevant to the query intent. For example, if the meeting minutes a user searches for include information about a "new product launch," the GNN will consider other records related to this topic, such as "market feedback meetings" or "user experience surveys," generating a diverse range of recommendations.

[0144] Finally, the system integrates the generated recommendations with the user's original query results to output the final result set. This result set not only covers the meeting record fragments required by the user, but also includes recommended related topics, entities, and other relationship information, allowing users to understand the relevant content from multiple dimensions. This recommendation not only improves the accessibility of information but also strengthens user decision-making support, ensuring that users can access a wider range of background information while consulting specific meeting records. For example, if a user searches for "annual financial meeting", the final results may not only include relevant meeting records, but also recommend discussions related to "annual budget" and "financial health", improving users' information acquisition capabilities and decision-making efficiency.

[0145] It can be seen that based on the voice, text, image and video data collected during the meeting, a multimodal fusion neural network is used to perform feature extraction and alignment to obtain a semantic representation vector; based on the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association; based on the enhanced semantic vector, combined with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the meeting topic in real time; based on the knowledge graph, a graph neural network is used to perform intelligent retrieval and recommendation of meeting record data, thereby improving the efficiency and accuracy of intelligent retrieval of meeting records.

[0146] Another embodiment of the present invention provides an AI-based conference record data search system, see Figure 3 , the system may include: Extraction module 301 is used to extract and align features from the voice, text, image, and video data collected during the meeting using a multimodal fusion neural network to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority integration of key information. Enhancement module 302 is configured to perform semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model uses time series modeling technology to capture temporal dependencies in meeting records and improve the accuracy of semantic understanding. Construction module 303 is used to generate a knowledge graph related to the conference topic in real time based on the enhanced semantic vector and in combination with an external knowledge base using a dynamic knowledge graph construction algorithm. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. The retrieval module 304 is used to intelligently retrieve and recommend meeting record data based on the knowledge graph using a graph neural network. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

[0147] It can be seen that based on the voice, text, image and video data collected during the meeting, a multimodal fusion neural network is used to perform feature extraction and alignment to obtain a semantic representation vector; based on the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association; based on the enhanced semantic vector, combined with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the meeting topic in real time; based on the knowledge graph, a graph neural network is used to perform intelligent retrieval and recommendation of meeting record data, thereby improving the efficiency and accuracy of intelligent retrieval of meeting records.

[0148] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0149] Specifically, in this embodiment, the above-mentioned storage medium may be configured to store a computer program for performing the following steps: S201: Based on the voice, text, image, and video data collected during the meeting, a multimodal fusion neural network is used to extract and align features to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority fusion of key information. S202: performing semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model captures temporal dependencies in the meeting minutes by introducing time series modeling technology to improve the accuracy of semantic understanding. S203: Based on the enhanced semantic vector and in combination with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the conference topic in real time. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. S204: Based on the knowledge graph, a graph neural network is used to intelligently retrieve and recommend meeting record data. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

[0150] It can be seen that based on the voice, text, image and video data collected during the meeting, a multimodal fusion neural network is used to perform feature extraction and alignment to obtain a semantic representation vector; based on the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association; based on the enhanced semantic vector, combined with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the meeting topic in real time; based on the knowledge graph, a graph neural network is used to perform intelligent retrieval and recommendation of meeting record data, thereby improving the efficiency and accuracy of intelligent retrieval of meeting records.

[0151] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0152] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0153] Specifically, in this embodiment, the processor may be configured to execute the following steps through a computer program: S201: Based on the voice, text, image, and video data collected during the meeting, a multimodal fusion neural network is used to extract and align features to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority fusion of key information. S202: performing semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model captures temporal dependencies in the meeting minutes by introducing time series modeling technology to improve the accuracy of semantic understanding. S203: Based on the enhanced semantic vector and in combination with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the conference topic in real time. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. S204: Based on the knowledge graph, a graph neural network is used to intelligently retrieve and recommend meeting record data. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

[0154] It can be seen that based on the voice, text, image and video data collected during the meeting, a multimodal fusion neural network is used to perform feature extraction and alignment to obtain a semantic representation vector; based on the semantic representation vector, combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association; based on the enhanced semantic vector, combined with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the meeting topic in real time; based on the knowledge graph, a graph neural network is used to perform intelligent retrieval and recommendation of meeting record data, thereby improving the efficiency and accuracy of intelligent retrieval of meeting records.

[0155] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings. The above is only a preferred embodiment of the present invention, but the scope of implementation of the present invention is not limited to what is shown in the drawings. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which do not exceed the spirit covered by the description and drawings, should be within the scope of protection of the present invention.

Claims

1. A method for searching conference record data based on AI, characterized in that: The method comprises: Based on the voice, text, image, and video data collected during the meeting, a multimodal fusion neural network is used to extract and align features to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weight of each modal data by introducing a cross-modal attention mechanism to ensure the priority integration of key information. Based on the semantic representation vector and combined with the contextual information of the meeting scene, a pre-trained context-aware model is used to perform semantic enhancement processing to generate an enhanced semantic vector with contextual association. The context-aware model captures the temporal dependencies in the meeting records by introducing time series modeling technology, thereby improving the accuracy of semantic understanding; Based on the enhanced semantic vectors and in combination with an external knowledge base, a dynamic knowledge graph construction algorithm is used to generate a knowledge graph related to the conference topic in real time. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time, thereby ensuring the timeliness and integrity of the knowledge graph. Based on the knowledge graph, a graph neural network is used to intelligently retrieve and recommend meeting record data. The graph neural network introduces an adaptive path search algorithm to dynamically adjust the retrieval path according to the user's query intent, and outputs meeting record fragments and related recommended content that are highly matched with the query intent.

2. The method according to claim 1, characterized in that The method uses a multimodal fusion neural network to extract and align features based on the voice, text, image, and video data collected during the meeting to obtain a semantic representation vector, including: The speech, text, image, and video data collected during the meeting were preprocessed separately. The speech data was subjected to noise reduction and speech recognition processing to obtain a speech feature vector. The text data was subjected to word segmentation, stop word removal, and word vectorization processing to obtain a text feature vector. The image data was subjected to target detection and feature extraction to obtain an image feature vector. The video data was subjected to key frame extraction and action recognition to obtain a video feature vector. Based on the preprocessed speech, text, image and video data, feature extraction is performed respectively using independent encoders in the multimodal fusion neural network, wherein the speech feature vector is encoded by the speech encoder to obtain a speech semantic vector, the text feature vector is encoded by the text encoder to obtain a text semantic vector, the image feature vector is encoded by the image encoder to obtain an image semantic vector, and the video feature vector is encoded by the video encoder to obtain a video semantic vector; Based on the extracted semantic vectors of each modality, a cross-modal attention mechanism is used to perform feature alignment. For each modality’s semantic vector, the attention weights between it and the semantic vectors of other modalities are calculated to generate weighted modality features. The multi-head attention mechanism is used to interactively fuse the weighted modal features to generate preliminary fusion features; Based on the preliminary fusion features, a decoder in the multimodal fusion neural network is used to generate a unified semantic representation vector. The preliminary fusion features are input into the decoder, and through multiple layers of nonlinear transformation and normalization, a unified semantic representation vector is obtained. The unified semantic representation vector contains the multimodal key information of the meeting minutes and retains the semantic relevance between the modalities. The semantic vectors whose attention weights are higher than the preset threshold are filtered out, and the filtered semantic vectors are secondary fused with the unified semantic representation vector, and finally a semantic representation vector containing key information is output.

3. The method according to claim 2, characterized in that The method of performing semantic enhancement processing based on the semantic representation vector and combining the contextual information of the conference scene with a pre-trained context-aware model to generate an enhanced semantic vector with context association includes: Extract the context information of the meeting scene and encode the context information into a context feature vector; The semantic representation vector and context feature vector are input into a pre-trained context-aware model. The context-aware model captures the temporal dependencies in the meeting transcripts by introducing time series modeling techniques. The context-aware model dynamically adjusts the semantic representation vector based on the time series information to generate a context-relevant intermediate semantic vector. Based on the intermediate semantic vector and the context feature vector, an attention mechanism is used to perform semantic enhancement processing, wherein the attention weight of the context feature vector to the intermediate semantic vector is calculated to generate a weighted context semantic vector; The weighted contextual semantic vector is fused with the intermediate semantic vector to generate an enhanced semantic vector with contextual association. The enhanced semantic vector contains the contextual information of the meeting scene and retains the global information of the semantic representation vector.

4. The method according to claim 3, characterized in that The method of generating a knowledge graph related to the conference topic in real time using a dynamic knowledge graph construction algorithm based on the enhanced semantic vector and in combination with an external knowledge base includes: The enhanced semantic vector is input into the pre-trained entity recognition model to extract the key entity set in the meeting transcript, where each entity contains entity type and context information; The enhanced semantic vector and the extracted entity set are input into the pre-trained relation extraction model to extract the relationship set between entities, where each relationship contains the relationship type and the entity pair involved; Based on the entity set and relationship set, the entities and relationships are aligned. Based on the aligned entities and relationships, in combination with the external knowledge base, entity linking and relationship expansion are performed. The relevant entities and relationships in the external knowledge base are mapped to the current meeting scenario to generate the initial knowledge graph. Based on the real-time input of meeting minutes, the initial knowledge graph is dynamically updated using an incremental learning mechanism. New entities and relationships are detected and added to the initial knowledge graph. A node embedding update algorithm is then used to dynamically adjust the representation of nodes and edges in the initial knowledge graph to ensure the timeliness of the knowledge graph. According to the changes in conference topics, the knowledge graph is pruned and reconstructed using a graph optimization algorithm. This involves removing redundant nodes and edges to ensure the simplicity of the knowledge graph, merging similar entities and relationships to ensure the integrity of the knowledge graph, and ultimately generating an optimized knowledge graph.

5. The method according to claim 4, characterized in that The intelligent retrieval and recommendation of meeting record data using a graph neural network based on the knowledge graph includes: Based on the query statement entered by the user, natural language processing technology is used to analyze the query intent and express the query intent as a query vector; Input the knowledge graph into the graph neural network to initialize the feature representation of entity nodes and relationship edges; According to the query vector, the adaptive path search algorithm is used to dynamically adjust the retrieval path in the graph neural network; Calculate the similarity between the query vector and each entity node, generate node weights, and dynamically select entity nodes and relationship edges that are highly relevant to the query intent based on the node weights to generate a retrieval path; Extract relevant meeting record fragments from the knowledge graph based on the retrieval path; Based on the retrieval path and meeting record fragments, a graph neural network is used to generate relevant recommended content. The recommended content includes entities, relationships, and meeting topics related to the query intent. Finally, the meeting record fragments and related recommended content that match the query intent are output.

6. An AI-based meeting record data search system, characterized in that: The system comprises: An extraction module is used to extract and align features from the voice, text, image, and video data collected during the meeting using a multimodal fusion neural network to obtain a semantic representation vector. The multimodal fusion neural network dynamically adjusts the weights of each modality by introducing a cross-modal attention mechanism to ensure the priority integration of key information. An enhancement module is configured to perform semantic enhancement processing based on the semantic representation vector and the contextual information of the meeting scene using a pre-trained context-aware model to generate an enhanced semantic vector with contextual association. The context-aware model captures the temporal dependencies in the meeting minutes by introducing time series modeling technology, thereby improving the accuracy of semantic understanding; A construction module is used to generate a knowledge graph related to the conference topic in real time based on the enhanced semantic vector and in combination with an external knowledge base using a dynamic knowledge graph construction algorithm. The dynamic knowledge graph construction algorithm introduces an incremental learning mechanism to update the nodes and edges in the knowledge graph in real time to ensure the timeliness and integrity of the knowledge graph. A retrieval module is used to intelligently retrieve and recommend meeting record data based on the knowledge graph using a graph neural network. The graph neural network dynamically adjusts the retrieval path according to the user's query intent by introducing an adaptive path search algorithm, and outputs meeting record fragments and related recommended content that highly match the query intent.

7. The system according to claim 6, characterized in that The extraction module is specifically used to: The speech, text, image, and video data collected during the meeting were preprocessed separately. The speech data was subjected to noise reduction and speech recognition processing to obtain a speech feature vector. The text data was subjected to word segmentation, stop word removal, and word vectorization processing to obtain a text feature vector. The image data was subjected to target detection and feature extraction to obtain an image feature vector. The video data was subjected to key frame extraction and action recognition to obtain a video feature vector. Based on the preprocessed speech, text, image and video data, feature extraction is performed respectively using independent encoders in the multimodal fusion neural network, wherein the speech feature vector is encoded by the speech encoder to obtain a speech semantic vector, the text feature vector is encoded by the text encoder to obtain a text semantic vector, the image feature vector is encoded by the image encoder to obtain an image semantic vector, and the video feature vector is encoded by the video encoder to obtain a video semantic vector; Based on the extracted semantic vectors of each modality, a cross-modal attention mechanism is used to perform feature alignment. For each modality’s semantic vector, the attention weights between it and the semantic vectors of other modalities are calculated to generate weighted modality features. The multi-head attention mechanism is used to interactively fuse the weighted modal features to generate preliminary fusion features; Based on the preliminary fusion features, a decoder in the multimodal fusion neural network is used to generate a unified semantic representation vector. The preliminary fusion features are input into the decoder, and through multiple layers of nonlinear transformation and normalization, a unified semantic representation vector is obtained. The unified semantic representation vector contains the multimodal key information of the meeting minutes and retains the semantic relevance between the modalities. The semantic vectors whose attention weights are higher than the preset threshold are filtered out, and the filtered semantic vectors are secondary fused with the unified semantic representation vector, and finally a semantic representation vector containing key information is output.

8. The system according to claim 7, characterized in that The enhancement module is specifically used to: Extract the context information of the meeting scene and encode the context information into a context feature vector; The semantic representation vector and context feature vector are input into a pre-trained context-aware model. The context-aware model captures the temporal dependencies in the meeting transcripts by introducing time series modeling techniques. The context-aware model dynamically adjusts the semantic representation vector based on the time series information to generate a context-relevant intermediate semantic vector. Based on the intermediate semantic vector and the context feature vector, an attention mechanism is used to perform semantic enhancement processing, wherein the attention weight of the context feature vector to the intermediate semantic vector is calculated to generate a weighted context semantic vector; The weighted contextual semantic vector is fused with the intermediate semantic vector to generate an enhanced semantic vector with contextual association. The enhanced semantic vector contains the contextual information of the meeting scene and retains the global information of the semantic representation vector.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images