Intelligent meeting management system and method based on large model

Abstract

The application discloses an intelligent conference management system and method based on a large model, relates to the technical field of intelligent conference management, and comprises the following steps: extracting and processing a project sequence of multiple conferences to obtain an enhanced representation of a current conference; recognizing audio data of the current conference to obtain a text segment sequence; projecting the text segment sequence and the enhanced conference representation to obtain a context-aware segment representation and calculate a dynamic importance weight to obtain a first candidate decision point set; iteratively compressing the first candidate decision point set through a double-constraint strategy to obtain a second candidate decision point set; for each decision point in the second candidate decision point set, extracting a feature vector of the decision point from multi-modal data of the current conference and projecting the feature vector into a projection vector; calculating a cosine similarity between the projection vectors to obtain a final confidence; and associating the final confidence with the corresponding decision point to obtain a third candidate decision point set.

Description

Technical Field

[0001] This invention relates to the field of intelligent meeting management technology, and in particular to an intelligent meeting management system and method based on a large model. Background Technology

[0002] In recent years, with the deepening of enterprise digital transformation and the popularization of remote collaboration, intelligent meeting management technology has become a key link in improving organizational operational efficiency. Traditional meeting management relies on manual recording and summarizing, which is difficult to cope with multi-source heterogeneous data, such as the explosive growth of audio recordings, videos, and behavior logs. Intelligent analysis technology based on big data models, through natural language understanding and multimodal fusion, provides a new path for automatically extracting meeting decision points, generating meeting minutes, and tracking tasks. How to accurately identify key decision moments from lengthy meeting content and associate them with multi-dimensional contextual information is the core problem that urgently needs to be solved to achieve efficient meeting management.

[0003] Existing meeting management methods mostly focus on speech transcription or keyword extraction for single meetings, lacking the ability to globally represent and analyze the correlation of historical meeting project sequences. This makes it impossible to capture long-term dependencies and evolutionary patterns between meetings. At the same time, traditional methods usually use fixed thresholds or simple statistical features when screening key decision points, ignoring the dynamic contextual importance of text fragments. Furthermore, they do not jointly constrain the number and fidelity of candidate decision points, which can easily lead to redundancy or omissions. Moreover, relying solely on text modality for decision point verification makes it difficult to handle scenarios with speech recognition ambiguity or semantic ambiguity. They also lack the means to obtain cross-validation from multimodal data such as video and behavioral logs, resulting in insufficient accuracy in the confidence assessment of the final decision points. Summary of the Invention

[0004] The technical problem solved by this invention is that existing intelligent meeting management methods lack the ability to represent the context of historical meeting sequences, lack dynamic importance perception and resource constraints in decision point extraction, and lack multimodal fusion verification methods, resulting in low accuracy of key decision point identification, high information redundancy, and unreliable confidence assessment.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: an intelligent meeting management method based on a large model, comprising the following steps: Step S1: Extract and process the project sequence of multiple meetings to obtain the enhanced representation of the current meeting, and identify the audio data of the current meeting to obtain a text segment sequence; Step S2: Project the text segment sequence and the enhanced meeting representation to obtain the context-aware segment representation, calculate the dynamic importance weight of the context-aware segment representation, obtain the first candidate decision point set based on the dynamic importance weight, and iteratively compress the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. Step S3: For each decision point in the second candidate decision point set, extract the feature vector of the decision point from the multimodal data of the current meeting, project the feature vector into a projection vector, calculate the cosine similarity between the projection vectors, obtain the final confidence score, and associate the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

[0006] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, S1 specifically includes: Step S101, the extraction process specifically includes: Extract the time features for each meeting item, including the number of days between adjacent meetings and the meeting duration; Connect two consecutive meeting items to form the first hyperedge, connect three consecutive meeting items to form the second hyperedge, and connect four consecutive meeting items to form the third hyperedge; Based on the first, second, and third hyperedges, generate an association matrix between nodes and hyperedges. The rows of the association matrix correspond to the nodes of the meeting items, and the columns correspond to each hyperedge. When the value of an element in the incidence matrix is ​​1, it means that the node belongs to the corresponding hyperedge; when the value of an element in the incidence matrix is ​​0, it means that the node does not belong to the corresponding hyperedge. The association matrix is ​​input into the hypergraph attention network, which sequentially performs bidirectional information transfer from node to hyperedge and from hyperedge to node, and outputs a preliminary contextual feature representation for each meeting item. Step S102: The preliminary contextual feature representation and corresponding temporal features of each meeting are fused using an attention mechanism to obtain fused features; After adding inverse positional encoding to the fused features, a self-attention mechanism is used to extract session features, resulting in an enhanced meeting representation of the current meeting; Acquire the audio recording data of the current meeting, perform speech recognition on the audio recording data using the Whisper model to obtain the original text record, and segment the original text record according to sentence boundaries to obtain the text segment sequence of the current meeting; The sentence boundaries are periods, question marks, or exclamation marks.

[0007] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S2 specifically includes: Step S201: Use the BERT-base model to convert each text segment into a text embedding vector of the first dimension; The text embedding vector is mapped to a first projection vector in the second dimension through a first linear projection matrix; The enhanced conference representation is mapped to a second-dimensional second projection vector through a second linear projection matrix; The first projection vector is concatenated with the second projection vector to obtain a third-dimensional context-aware fragment representation; The enhanced conference representation is then mapped to a third-dimensional third projection vector via a third linear projection matrix.

[0008] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S2 further includes: Calculate the dynamic importance weight of each context-aware segment representation to obtain the importance weight of each segment; The expression for dynamic importance weights is: ; in, Indicates the first The importance weights of each context-aware segment Indicates the first Normalized word frequencies of a text segment Indicates the first Context-aware fragment representation of a text segment Represents the third projection vector. Represents the L2 norm. This represents a very small constant.

[0009] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S2 further includes: From all segments with importance weights, segments with importance weights greater than or equal to the current threshold are selected to form the first candidate decision point set; Calculate the fidelity and resource count of the first candidate decision point set; The fidelity calculation process specifically includes: Obtain context-aware segment representations of all text segments and construct the first matrix; Based on the first set of candidate decision points, generate a second matrix with the same size as the first matrix; In the second matrix, the rows corresponding to the segments within the first candidate decision point set retain the context-aware segment representation, while the remaining rows are set to zero vectors; Calculate the mean square error between the first matrix and the second matrix, and use the mean square error as the fidelity. The resource count is the number of segments in the first candidate decision point set.

[0010] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S2 further includes: The dual-constraint strategy is applied to the first set of candidate decision points, and the dual-constraint strategy specifically includes: The current threshold is iteratively adjusted, and the iterative compression specifically includes: If the fidelity is greater than the fidelity limit and the resource count is less than the resource limit, then subtract the learning rate from the current threshold. If the resource count is greater than the resource limit and the fidelity is less than the fidelity limit, then add the learning rate to the current threshold. If the fidelity is greater than the fidelity limit and the resource count is greater than the resource limit, then add half of the learning rate to the current threshold. If the fidelity is less than the fidelity limit and the resource count is less than the resource limit, subtract the learning rate from the current threshold. Repeatedly compress until the convergence condition is met or the maximum number of iterations is reached, to obtain the second candidate decision point set. Each segment in the second candidate decision point set includes the text content of the corresponding segment and the corresponding timestamp. Convergence conditions include: The absolute difference between fidelity and fidelity limit is less than the first preset threshold, and the relative difference between resource count and resource limit is less than the second preset threshold.

[0011] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S3 specifically includes: Step S301: Obtain each decision point in the second candidate decision point set. Each decision point includes text content and a corresponding timestamp. Acquire multimodal data from the current meeting, including video data and behavioral log data; Based on the timestamp, extract 5 video frames before and after the timestamp from the video data to obtain 11 video frames. Scale each video frame to a uniform 224×224 pixel RGB format. The ViT-B / 16 model is used to convert each frame into a first-dimensional image frame feature, resulting in 11 first-dimensional image frame feature vectors. Attention pooling is used to aggregate the feature vectors of 11 image frames into a single image feature vector. Based on the timestamp, extract the sequence of behavioral events within 5 seconds before and after the timestamp from the behavioral log data, and map each behavioral event to a fourth-dimensional embedding vector through a learnable behavioral embedding table; The number of rows in the behavior embedding table is equal to the total number of predefined behavior event categories; Average pooling is performed on all embedded vectors in the behavioral event sequence to obtain the fourth-dimensional behavioral feature vector; The text content of the decision point is converted into a first-dimensional text embedding vector using the BERT-base model.

[0012] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S3 further includes: Step S302, the feature vector includes text embedding vector, image feature vector and behavior feature vector; The text embedding vector is projected into a fifth-dimensional text projection vector through the third linear projection matrix; The image feature vector is projected into a fifth-dimensional image projection vector through the fourth linear projection matrix; The behavior feature vector is projected into a fifth-dimensional behavior projection vector through the fifth linear projection matrix; Projection vectors include text projection vectors, image projection vectors, and behavior projection vectors.

[0013] As a preferred embodiment of the intelligent meeting management method based on a large model described in this invention, step S3 further includes: Step S303: Calculate the cosine similarity between the text projection vector and the image projection vector to obtain the text-image similarity. Calculate the cosine similarity between the text projection vector and the behavior projection vector to obtain the text-behavior similarity; The original confidence score is obtained by weighted summation of text-image similarity and text-behavior similarity. The original confidence level is linearly mapped to the [0,1] interval to obtain the final confidence level; The final confidence level is associated with each decision point in the second candidate decision point set to obtain the third candidate decision point set.

[0014] The intelligent meeting management system based on a large model includes extraction, calculation, and association modules; The extraction module extracts and processes the project sequences from multiple meetings to obtain an enhanced representation of the current meeting, and identifies the audio recording data of the current meeting to obtain a sequence of text segments. The calculation module projects the text fragment sequence and the enhanced meeting representation to obtain the context-aware fragment representation, calculates the dynamic importance weight of the context-aware fragment representation, obtains the first candidate decision point set based on the dynamic importance weight, and iteratively compresses the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. The association module extracts the feature vector of each decision point in the second candidate decision point set from the multimodal data of the current meeting, projects the feature vector into a projection vector, calculates the cosine similarity between the projection vectors to obtain the final confidence score, and associates the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

[0015] The beneficial effects of this invention are as follows: To address the existing fixed threshold problem, this invention introduces dynamic importance weights based on word frequency and context awareness, and designs a dual-constraint iterative compression strategy to achieve joint optimization of fidelity and resource counting. While controlling the number of decision points, it retains key semantics. By fusing multi-level associations and reverse positional encodings from multiple meetings through a hypergraph attention network, it enhances the representational ability of historical meeting sequences. By employing multimodal projection and weighted cosine similarity, and fusing video and behavioral logs for cross-validation, it significantly improves the confidence of decision points and the accuracy of meeting management. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of an intelligent meeting management method based on a large model, provided as an embodiment of the present invention.

[0017] Figure 2 This is a basic flowchart of an intelligent meeting management system based on a large model, provided as an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example 1, referring to Figure 1 As an embodiment of the present invention, an intelligent meeting management method based on a large model is provided, comprising the following steps: Step S1: Extract and process the project sequence of multiple meetings to obtain the enhanced representation of the current meeting, and identify the audio data of the current meeting to obtain a text segment sequence; Step S2: Project the text segment sequence and the enhanced meeting representation to obtain the context-aware segment representation, calculate the dynamic importance weight of the context-aware segment representation, obtain the first candidate decision point set based on the dynamic importance weight, and iteratively compress the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. Step S3: For each decision point in the second candidate decision point set, extract the feature vector of the decision point from the multimodal data of the current meeting, project the feature vector into a projection vector, calculate the cosine similarity between the projection vectors, obtain the final confidence score, and associate the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

[0020] In one embodiment, step S1 extracts and processes the project sequences of multiple meetings to obtain an enhanced representation of the current meeting, and simultaneously identifies the recording data of the current meeting to obtain a text segment sequence, providing a semantic basis for subsequent decision point extraction, including temporal dependencies. Step S2 addresses the limitations of fixed threshold selection of decision points by using dynamic importance weights to perceive the criticality of text segments. Specifically, it projects the text segment sequence and the enhanced meeting representation to obtain a context-aware segment representation, calculates dynamic importance weights, obtains a first candidate decision point set based on the dynamic importance weights, and then iteratively compresses it using a dual-constraint strategy to obtain a second candidate decision point set, thereby achieving adaptive selection of decision points while controlling information fidelity and resource counting. Step S3 introduces a multimodal cross-validation mechanism to solve the problem of unreliable single-text modality validation. Specifically, for each decision point in the second candidate decision point set, it extracts a feature vector from the multimodal data collected from the meeting, projects the feature vectors, calculates the cosine similarity between the projected vectors to obtain the final confidence score, and associates the confidence score with the corresponding decision point to obtain a third candidate decision point set, thereby improving the accuracy of the decision point confidence score. The entire design framework achieves a complete closed loop from meeting context to accurate identification of key decision points and reliability assessment by extracting multiple meeting project sequences, dynamically weighting importance and compressing with dual-constraint strategies, and linking multimodal confidence verification. This improves the accuracy of key decision point identification in intelligent meeting management and reduces information redundancy.

[0021] Step S1 specifically includes: Step S101, the extraction process specifically includes: Extract the time features for each meeting item, including the number of days between adjacent meetings and the meeting duration; Connect two consecutive meeting items to form the first hyperedge, connect three consecutive meeting items to form the second hyperedge, and connect four consecutive meeting items to form the third hyperedge; Based on the first, second, and third hyperedges, generate an association matrix between nodes and hyperedges. The rows of the association matrix correspond to the nodes of the meeting items, and the columns correspond to each hyperedge. When the value of an element in the incidence matrix is ​​1, it means that the node belongs to the corresponding hyperedge; when the value of an element in the incidence matrix is ​​0, it means that the node does not belong to the corresponding hyperedge. The association matrix is ​​input into the hypergraph attention network, which sequentially performs bidirectional information transfer from node to hyperedge and from hyperedge to node, and outputs a preliminary contextual feature representation for each meeting item. Step S102: The preliminary contextual feature representation and corresponding temporal features of each meeting are fused using an attention mechanism to obtain fused features; After adding inverse positional encoding to the fused features, a self-attention mechanism is used to extract session features, resulting in an enhanced meeting representation of the current meeting; Acquire the audio recording data of the current meeting, perform speech recognition on the audio recording data using the Whisper model to obtain the original text record, and segment the original text record according to sentence boundaries to obtain the text segment sequence of the current meeting; Sentence boundaries are periods, question marks, or exclamation marks.

[0022] In one embodiment, step S1 involves constructing a multi-scale hyperedge to capture complex high-order dependencies between meeting items and using inverse position encoding to enhance the current meeting's inverse perception of the global sequence, providing an enhanced representation including temporal semantics for subsequent decision point extraction. This step aims to address the problem that traditional methods cannot effectively utilize historical meeting context information, laying a reliable semantic foundation for dynamic importance weight calculation and multimodal verification. Step S101 specifically includes: extracting the temporal features of each meeting item, including the number of days between adjacent meetings and the meeting duration; wherein the number of days between adjacent meetings is the natural day difference between the current meeting and the previous meeting, and the meeting duration is in minutes, calculated from the meeting start and end times; the temporal features of each meeting item are used as the initial feature vector (two-dimensional vector) of that meeting item node; or, when meeting items include other metadata (such as meeting... When generating a list of topics and attendees, this metadata can be embedded and encoded before being concatenated with time features to form a higher-dimensional initial feature vector. In this embodiment, to simplify calculations, two-dimensional time features are directly used as the initial features of nodes. To capture the relationships between projects at different granularities, two consecutive conference projects are connected as the first hyperedge, three consecutive conference projects as the second hyperedge, and four consecutive conference projects as the third hyperedge. When the length of the historical conference project sequence is less than the number of consecutive projects required by the hyperedge (for example, when the sequence length is only 3, it is impossible to build a hyperedge with 4 consecutive projects), only hyperedges that can be composed of existing projects are built. Missing hyperedges do not participate in the subsequent generation of the association matrix, and the hypergraph attention network still performs message passing based on the actual existing hyperedges. When the sequence length is less than 2, no hyperedges are built, and the initial features of each conference project are directly output as the preliminary context feature representation.The hyperedge lengths of 2, 3, and 4 were determined through comparative experiments. Specifically, the experiments tested seven hyperedge length configurations on a dataset of 500 real-world conferences: using only length 2, only length 3, only length 4, a combination of lengths 2 and 3, a combination of lengths 2 and 4, a combination of lengths 3 and 4, and a combination of lengths 2, 3, and 4. In each configuration, a hypergraph attention network was used to output preliminary contextual feature representations for each conference item. These preliminary contextual feature representations were then used as input to train a linear regressor to predict the number of days between adjacent conferences and the conference duration for the current conference item. The linear regressor used mean squared error as the loss function, and the optimizer was standard stochastic gradient descent. The learning rate was fixed at 0.01, the training epochs were 100, and the batch size was 32. The mean squared error (MSE) between predicted and actual values ​​was used as the evaluation metric. Experimental results showed that the MSE was consistently higher than 0.35 when using only a single length. The MSE decreased to a minimum of 0.28 when using a combination of two lengths. Furthermore, the MSE reached 0.23 when using a combination of lengths 2, 3, and 4, a reduction of approximately 18% compared to the two-length combination. Therefore, the combination of lengths 2, 3, and 4 was chosen as the lowest MSE among all configurations, indicating its strongest ability to capture multi-scale dependencies between meeting items, thus providing the most accurate meeting context representation for subsequent steps. Nodes were generated based on the first, second, and third hyperedges. The association matrix between nodes and hyperedges is generated, where rows correspond to nodes of conference projects and columns correspond to each hyperedge. A node's value is 1 if it belongs to a hyperedge, otherwise it is 0. This association matrix is ​​then input into a hypergraph attention network, which employs a two-stage message passing mechanism. Specifically, the first stage involves information aggregation from nodes to hyperedges. For each hyperedge, based on all associated conference project nodes, the attention mechanism calculates the weights of each node. The weight calculation method is as follows: first, the current feature of each node is multiplied by a linear transformation matrix W1, mapping the node features to the same dimensional space as the learnable hyperedge feature vector. Then, a dot product is calculated with a learnable hyperedge feature vector, followed by softmax normalization. Finally, the features of each node are processed... The first stage involves weighted summation to obtain the updated representation of the hyperedge. The second stage involves updating the information from the hyperedge to the node. For each conference project node, the updated representations of all hyperedges including that node are collected. The attention mechanism is used again to calculate the importance weight of each hyperedge to that node. Here, the hyperedge feature specifically refers to the updated representation of the hyperedge calculated in the first stage, rather than the initial learnable hyperedge feature vector. The weight is determined by the dot product of the node feature and the updated representation of the hyperedge. After weighted aggregation to obtain the combined feature, the combined feature is first multiplied by the linear transformation matrix W2 to adjust its feature dimension to match the node output feature dimension. Then, the updated feature of the node is obtained through nonlinear transformation. The nonlinear transformation uses the ReLU activation function and includes a learnable bias term.The two-stage message passing mechanism can be repeated in multiple layers. In this embodiment, one layer is set. All attention weights are calculated using the standard softmax function. The learnable parameters include the initial hyperedge feature vector, two linear transformation matrices (W1 and W2), and the bias term in the nonlinear transformation. The hypergraph attention network sequentially performs bidirectional information passing from node to hyperedge and from hyperedge to node, outputting a preliminary contextual feature representation for each meeting item. This allows each meeting item to aggregate neighbor information within hyperedges of different scales, enhancing the feature's ability to represent long-term and short-term dependencies. During the training process of the hypergraph attention network, the goal is to predict the temporal features (adjacent interval days and meeting duration) of the next meeting item, using mean squared error (MSE) as the loss function. The network parameters are optimized through backpropagation; step S102 includes: fusing the preliminary context feature representation and corresponding time feature of each meeting using an attention mechanism to obtain fused features. Specifically, the fusion method is as follows: the time feature (two-dimensional vector) is mapped to a vector of the same dimension as the preliminary context feature representation through a linear layer, and then the mapped vector is used as the query, and the preliminary context feature representation is used as the key and value. Scaled dot product attention is calculated: the dot product of the query and the key is divided by the square root of the dimension and then normalized by softmax to obtain the attention weights. The values ​​are then weighted and summed to output the fused features. This attention mechanism dynamically focuses on the preliminary context feature table according to the time feature. The different dimensions in the representation highlight information related to the current meeting's time pattern. The attention mechanism adaptively allocates weights based on the relevance between time features and the initial contextual feature representation, making the fusion result more focused on temporal patterns useful for subsequent sessions. After adding inverse positional encoding to the fused features, a self-attention mechanism is used for session feature extraction to obtain an enhanced meeting representation of the current meeting. Specifically, session feature extraction includes: adding inverse positional encoding to the fused features to obtain a feature sequence with inverse positional information. The inverse positional encoding is generated by assigning a symmetrical encoding vector to each position according to the reverse order of the item sequence of multiple meetings. The forward positional encoding uses sine and cosine positional encoding. The fusion features are added element-wise to obtain a feature sequence with inverse positional information. This feature sequence corresponds to each item in the item sequence of multiple meetings. The feature sequence with inverse positional information is then input into a multi-head self-attention network layer for session feature extraction. The output is a session feature sequence with the same length and feature dimension as the input sequence. The length of the input sequence is equal to the total number of items in the multiple meetings being processed. This number can be dynamically changed based on the actual historical meeting records. The multi-head self-attention network layer supports parallel processing of variable-length sequences through positional encoding and masking mechanisms, without requiring a fixed sequence length. The multi-head self-attention network layer includes eight attention heads. The session feature sequence is then average-pooled along the sequence dimension to obtain an aggregated feature vector.The aggregated feature vector is mapped to a 256-dimensional vector through a fully connected linear layer. This 256-dimensional vector is the enhanced representation of the current meeting. Then, the recording data of the current meeting is obtained, and the recording data is used for speech recognition through the Whisper model to obtain the original text record. The original text record is segmented according to sentence boundaries to obtain the text segment sequence of the current meeting. The sentence boundaries are periods, question marks, or exclamation marks. This segmentation method preserves complete semantic units and avoids the impact of sentence segmentation errors on the subsequent importance weight calculation. By combining multi-scale hypergraph attention with inverse positional encoding, the ability of the current meeting augmented representation to capture the evolutionary patterns of historical meeting sequences is significantly improved, resulting in an approximately 18% increase in recall for subsequent decision point selection compared to traditional methods that only use features from a single meeting.

[0023] Step S2 specifically includes: Step S201: Use the BERT-base model to convert each text segment into a text embedding vector of the first dimension; The text embedding vector is mapped to a first projection vector in the second dimension through a first linear projection matrix; The enhanced conference representation is mapped to a second-dimensional second projection vector through a second linear projection matrix; The first projection vector is concatenated with the second projection vector to obtain a third-dimensional context-aware fragment representation; The enhanced conference representation is then mapped to a third-dimensional third projection vector via a third linear projection matrix.

[0024] Step S2 also includes: Calculate the dynamic importance weight of each context-aware segment representation to obtain the importance weight of each segment; The expression for dynamic importance weights is: ; in, Indicates the first The importance weights of each context-aware segment Indicates the first Normalized word frequencies of a text segment Indicates the first Context-aware fragment representation of a text segment Represents the third projection vector. Represents the L2 norm. This represents a very small constant.

[0025] Step S2 also includes: From all segments with importance weights, segments with importance weights greater than or equal to the current threshold are selected to form the first candidate decision point set; Calculate the fidelity and resource count of the first candidate decision point set; The fidelity calculation process specifically includes: Obtain context-aware segment representations of all text segments and construct the first matrix; Based on the first set of candidate decision points, generate a second matrix with the same size as the first matrix; In the second matrix, the rows corresponding to the segments within the first candidate decision point set retain the context-aware segment representation, while the remaining rows are set to zero vectors; Calculate the mean square error between the first matrix and the second matrix, and use the mean square error as the fidelity. The resource count is the number of segments in the first candidate decision point set.

[0026] Step S2 also includes: A two-constraint strategy is applied to the first set of candidate decision points. The two-constraint strategy specifically includes: The current threshold is iteratively adjusted, and the iterative compression specifically includes: If the fidelity is greater than the fidelity limit and the resource count is less than the resource limit, then subtract the learning rate from the current threshold. If the resource count is greater than the resource limit and the fidelity is less than the fidelity limit, then add the learning rate to the current threshold. If the fidelity is greater than the fidelity limit and the resource count is greater than the resource limit, then add half of the learning rate to the current threshold. If the fidelity is less than the fidelity limit and the resource count is less than the resource limit, subtract the learning rate from the current threshold. Repeatedly compress until the convergence condition is met or the maximum number of iterations is reached, to obtain the second candidate decision point set. Each segment in the second candidate decision point set includes the text content of the corresponding segment and the corresponding timestamp. Convergence conditions include: The absolute difference between fidelity and fidelity limit is less than the first preset threshold, and the relative difference between resource count and resource limit is less than the second preset threshold.

[0027] In one embodiment, step S2 involves projecting text fragments and enhanced meeting representations onto a unified space and calculating dynamic importance weights to achieve context-sensitive filtering of text fragments. Simultaneously, a dual-constraint strategy is introduced for adaptive iterative compression between fidelity and resource count, balancing information retention and the number of decision points. This step addresses the problem that traditional fixed-threshold methods cannot dynamically adapt to changes in meeting content, leading to omissions or redundancies of key decision points, thus providing a refined and high-fidelity set of candidate decision points for subsequent multimodal confidence assessment. Step S201 specifically includes: using a BERT-base model to convert each text fragment into a first-dimensional text embedding vector (768 dimensions); and mapping this text embedding vector through a first linear projection matrix. The first projection vector of the second dimension is mapped to the second projection vector through the second linear projection matrix, which has 512 rows and 768 columns. The second dimension is 512-dimensional. The enhanced meeting representation is mapped to the second projection vector of the second dimension through the second linear projection matrix, which has 512 rows and 256 columns. The first projection vector and the second projection vector are concatenated to obtain the context-aware fragment representation of the third dimension, which has 1024 dimensions. The enhanced meeting representation is then mapped to the third projection vector of the third dimension through the third linear projection matrix, which has 1024 rows and 256 columns. This allows the text fragments to be integrated with the global meeting context, significantly improving the semantic relevance of subsequent importance weights. The dynamic importance weight of each context-aware fragment representation is calculated using the following expression: ; in, Indicates the first The importance weight of each segment Indicates the first The normalized word frequency of a text segment is calculated by summing the frequencies of all words in that segment throughout the entire meeting, dividing by the segment length, and then normalizing to the [0,1] interval. Indicates the first The context-aware segment representation of each text fragment is a 1024-dimensional vector. This represents the third projection vector, which is a 1024-dimensional vector. Represents the L2 norm. This represents a minimum constant, a preset minimum constant used to avoid numerical calculation anomalies caused by a denominator of zero. This constant is much smaller than the L2 norm product of normal feature vectors (usually greater than 1), and will not have a noticeable impact on the weight calculation results, while ensuring the numerical stability of the algorithm under all input conditions. This expression measures the criticality of a segment by the product of the word frequency and the direction cosines of the two vectors. The larger the direction cosine, the more consistent the semantics of the segment are with the third projection vector. The initial threshold was set to 0.5, determined experimentally. Specifically, the experiment was conducted on a dataset of 500 real conferences. Each conference had key decision points manually labeled as real labels (i.e., the reference standard). The test threshold ranged from 0.1 to 0.9, with a step size of 0.05, for a total of 17 different threshold values. For each candidate threshold, text segments with importance weights greater than or equal to that threshold were selected to form a candidate decision point set. Then, the precision (the ratio of correctly extracted decision points to the total number of extracted candidate decision points) and recall (the ratio of correctly extracted decision points to the number of manually labeled real decision points) for that conference were calculated. Furthermore, the F1 score (the harmonic mean of precision and recall) was calculated. This was performed on a dataset of 500 conferences. The average F1 score at various thresholds was calculated. Experimental results showed that when the threshold was 0.3 to 0.4, the average F1 score was approximately 0.78 to 0.81, with a recall of approximately 0.90 to 0.93 and a precision of approximately 0.67 to 0.72. When the threshold was 0.6 to 0.7, the average F1 score was approximately 0.80 to 0.79, with a precision of approximately 0.85 to 0.88 and a recall of approximately 0.72 to 0.75. At the threshold of 0.5, the average F1 score reached its maximum of 0.86, with a precision of 0.85 and a recall of 0.87, achieving the best balance between precision and recall. Therefore, 0.5 was chosen as the optimal initial threshold. Segments with an importance weight greater than or equal to the current threshold were selected from all segments with varying importance weights to form the first candidate decision point set. The calculation of the fidelity and resource count of the first candidate decision point set includes: The fidelity calculation process involves obtaining context-aware segment representations of all text fragments to form a first matrix with a size of N×1024, where N is the total number of text fragments and 1024 is the dimension of the context-aware segment representation, i.e., the third dimension; A second matrix with the same size as the first matrix is ​​generated based on the first candidate decision point set. In the second matrix, the rows corresponding to the fragments in the first candidate decision point set retain the context-aware segment representation, while the remaining rows are set to zero vectors; The mean square error between the first and second matrices is calculated, and this mean square error is used as the fidelity. A smaller fidelity indicates more complete information retention; the resource count is the number of text fragments in the first candidate decision point set. The threshold is adjusted by iteratively. Lowering the threshold increases the resource count and decreases the fidelity, thus reducing information loss. Conversely, raising the threshold decreases the resource count and increases the fidelity, thus increasing information loss. A dual-constraint strategy is implemented for the first candidate decision point set, specifically: the current threshold is iteratively adjusted. First, the resource limit is set to 20, based on the average number of manually labeled decision points per meeting being 18 to 22. Therefore, the median value of 20 is taken, providing a moderate adjustment space when the resource limit is not frequently reached, while taking into account the decision point requirements of most meetings. The remaining parameters (fidelity limit, learning rate, first preset threshold, and second preset threshold) are determined through sequential experiments. Each experiment is based on the resource limit of 20 as a fixed condition and is conducted on the basis of the determined parameters. The fidelity cap was set experimentally, specifically as follows: A dataset of 500 real conference sessions was divided into a training set (400 sessions) and a validation set (100 sessions) with 80% allocated to training and 20% to validation. A fixed resource limit of 20 was set. The learning rate, first preset threshold, and second preset threshold were initially set to empirical values ​​(learning rate 0.05, first preset threshold 0.01, second preset threshold 0.1). The fidelity caps were tested at 0.10, 0.15, 0.20, and 0.25 respectively. A double-constraint iterative compression algorithm was run. The F1 score of the final decision point set was calculated on the validation set. Experimental results showed that the F1 score was 0.82 when the fidelity cap was 0.10, 0.88 when it was 0.15, 0.86 when it was 0.20, and 0.83 when it was 0.25. Therefore, 0.15 was selected as the fidelity cap. At this value, the F1 score on the validation set was the highest (0.88), which is 0.06 higher than 0.10, and 0.02 and 0.05 higher than 0.20 and 0.25 respectively, resulting in the best overall performance. The learning rate was set based on the experiment, specifically including: a fixed resource limit of 20 and a fidelity limit of 0.15. The first and second preset thresholds still used empirical initial values ​​(0.01 and 0.1). Learning rates of 0.01, 0.03, 0.05, and 0.08 were tested respectively. The average number of steps required to reach convergence in each iteration and the deviation of the fidelity from the resource limit after convergence were recorded. The experimental results showed that: at 0.01, the average number of steps was 87, with a deviation of (0.008, 0.12); at 0.03, the average number of steps was 42, with a deviation of (0.008, 0.12). At 0.05, the convergence rate is 28 steps with a deviation of (0.005, 0.08); at 0.08, the convergence rate is 22 steps but with significant oscillations and a deviation of (0.015, 0.14). Considering both convergence speed and stability, 0.05 is chosen. Setting the learning rate to 0.05 achieves the best balance between the number of convergence steps (28 steps) and the post-convergence deviation (fidelity deviation 0.005, resource count deviation 0.08). Compared to 0.03, the convergence speed is improved by 33%, and compared to 0.08, oscillations are avoided and the deviation is smaller. The first preset threshold was set based on experiments, specifically including: a fixed resource limit of 20, a fidelity limit of 0.15, and a learning rate of 0.05. The second preset threshold was temporarily set to an empirical value of 0.1. Tests were conducted at the first preset thresholds of 0.005, 0.01, and 0.02, recording the average number of iterations and the absolute difference in fidelity after convergence. The results showed that at 0.005, 52 steps were required, with an absolute difference of 0.004, but 12% of the samples failed to converge within 50 steps; at 0.01, 28 steps were required, with an absolute difference of 0.004. With an absolute difference of 0.008, the non-convergence rate is 3%; with 0.02, it takes 16 steps and the absolute difference is 0.017, which is too large. Therefore, 0.01 is chosen. Setting it to 0.01 ensures that the vast majority of samples (97%) can converge, while the absolute difference of fidelity after convergence (0.008) is close to the tolerance limit of 0.01, and the number of iteration steps is moderate (28 steps). 0.005 leads to an excessively high convergence failure rate, while 0.02 causes the absolute difference of fidelity to exceed the tolerance range. The second preset threshold was set based on experiments, specifically including: a fixed resource limit of 20, a fidelity limit of 0.15, a learning rate of 0.05, and a first preset threshold of 0.01. The second preset thresholds of 0.05, 0.1, and 0.15 were tested respectively, and the average number of iterations and the relative difference of resource counts were recorded. The results showed that at 0.05, there were 46 steps and a relative difference of 0.04, with 8% of the samples failing to converge; at 0.1, there were 30 steps and a relative difference of 0.08, with a non-convergence rate of 2%; at 0.15, there were 22 steps and a relative difference of 0.13, which exceeded the tolerance range of 10%. Therefore, 0.1 was chosen. Setting it to 0.1 ensures that 98% of the samples can converge, and the relative difference after convergence (0.08) is less than the tolerance range of 10%, and the number of iterations (30 steps) is moderate. However, 0.05 has a high convergence failure rate, and 0.15 causes the relative difference of resource counts to exceed the tolerance range. The current threshold is iteratively adjusted. The iterative compression specifically includes: if the fidelity is greater than the fidelity limit and the resource count is less than the resource limit, then the learning rate is subtracted from the current threshold; if the resource count is greater than the resource limit and the fidelity is less than the fidelity limit, then the learning rate is added to the current threshold; if the fidelity is greater than the fidelity limit and the resource count is greater than the resource limit, then half of the learning rate is added to the current threshold, in which case the resource count is reduced first; if the fidelity is less than the fidelity limit and the resource count is less than the resource limit, then the learning rate is subtracted from the current threshold. The iterative compression is repeated until the convergence condition is met (the absolute difference between the fidelity and the fidelity limit is less than the first preset threshold of 0.01, and the relative difference between the resource count and the resource limit is less than the second preset threshold of 0.1) or the maximum number of iterations of 50 is reached (to ensure termination), resulting in a second set of candidate decision points. Each segment includes the text content of the corresponding segment and the corresponding timestamp. The dual-constraint strategy dynamically optimizes between fidelity and resource count by adaptively adjusting the threshold, so that the selected decision points maintain high information fidelity while avoiding excessive redundancy. The combination of dynamic importance weights and dual-constraint iterative compression significantly improves the F1 score of decision point extraction by an average of about 15% compared to the fixed threshold method, while reducing the resource count variance by about 30%, thus significantly improving the accuracy and stability of key decision point selection.

[0028] Step S3 specifically includes: Step S301: Obtain each decision point in the second candidate decision point set. Each decision point includes text content and a corresponding timestamp. Acquire multimodal data from the current meeting, including video data and behavioral log data; Based on the timestamp, extract 5 video frames before and after the timestamp from the video data to obtain 11 video frames. Scale each video frame to a uniform 224×224 pixel RGB format. The ViT-B / 16 model is used to convert each frame into a first-dimensional image frame feature, resulting in 11 first-dimensional image frame feature vectors. Attention pooling is used to aggregate the feature vectors of 11 image frames into a single image feature vector. Based on the timestamp, extract the sequence of behavioral events within 5 seconds before and after the timestamp from the behavioral log data, and map each behavioral event to a fourth-dimensional embedding vector through a learnable behavioral embedding table; The number of rows in the behavior embedding table is equal to the total number of predefined behavior event categories; Average pooling is performed on all embedded vectors in the behavioral event sequence to obtain the fourth-dimensional behavioral feature vector; The text content of the decision point is converted into a first-dimensional text embedding vector using the BERT-base model.

[0029] Step S3 also includes: Step S302, the feature vector includes text embedding vector, image feature vector and behavior feature vector; The text embedding vector is projected into a fifth-dimensional text projection vector through the third linear projection matrix; The image feature vector is projected into a fifth-dimensional image projection vector through the fourth linear projection matrix; The behavior feature vector is projected into a fifth-dimensional behavior projection vector through the fifth linear projection matrix; Projection vectors include text projection vectors, image projection vectors, and behavior projection vectors.

[0030] Step S3 also includes: Step S303: Calculate the cosine similarity between the text projection vector and the image projection vector to obtain the text-image similarity. Calculate the cosine similarity between the text projection vector and the behavior projection vector to obtain the text-behavior similarity; The original confidence score is obtained by weighted summation of text-image similarity and text-behavior similarity. The original confidence level is linearly mapped to the [0,1] interval to obtain the final confidence level; The final confidence level is associated with each decision point in the second candidate decision point set to obtain the third candidate decision point set.

[0031] In one embodiment, step S3 introduces two non-textual modal data—video frames and behavior logs—to perform multi-angle cross-validation on each decision point in the second candidate decision point set, overcoming the misjudgment of confidence caused by single textual modality when speech recognition is ambiguous or semantically vague. This step aims to address the lack of multi-modal fusion verification methods in existing approaches, assigning reliable confidence scores to the selected decision points, and outputting a high-precision, low-false-detection third candidate decision point set, thus achieving a complete closed loop for intelligent meeting management. Step S301 specifically includes: obtaining each decision point in the second candidate decision point set, where each decision point includes text content and a corresponding timestamp. Acquire multimodal data of the current meeting, including video data and behavior log data; extract 5 video frames before and after the timestamp from the video data, resulting in 11 video frames. The video data is collected at a fixed frame rate (e.g., 30 frames / second), or if the frame rate is not fixed, extract all video frames in a time window of 0.167 seconds before and after the timestamp (corresponding to 5 frames at 30fps), and uniformly scale them to the same number (e.g., 11 frames through interpolation). The selection of 5 frames before and after the timestamp was set experimentally. The specific experimental process includes: on a dataset of 500 real meetings, each meeting has been manually labeled with the corresponding decision point. At the moment of the critical action, three window sizes were tested: 3 frames before and after (7 frames total), 5 frames before and after (11 frames total), and 7 frames before and after (15 frames total). For each decision point's timestamp, video frames were extracted according to each window size, and the same pre-trained action recognition model (using a SlowFast network) was used to determine whether the extracted frame sequence contained manually annotated critical actions. The action capture rate was defined as: for all decision points, the number of decision points in the frame sequence containing critical actions divided by the total number of decision points. The computational cost was measured by the average time (milliseconds) of a single extraction and processing. Experimental results showed that the action capture rate was 80% when there were 3 frames before and after the critical action. First, the average time is 18 milliseconds; the motion capture rate is 92% with an average time of 32 milliseconds when there are 5 frames before and after; the motion capture rate is 95% with an average time of 51 milliseconds when there are 7 frames before and after. Considering that the capture rate of 5 frames is only 3 percentage points lower than that of 7 frames, but the time is reduced by about 37%, and the 92% capture rate already meets the actual application requirements, the 5 frames before and after are chosen as the balance point; each frame of video image is uniformly scaled to 224×224 pixel RGB format; the ViT-B / 16 model is used to convert each frame into the first dimension of image frame features, the first dimension is 768-dimensional, resulting in 11 768-dimensional image frame feature vectors;The 11 image frame feature vectors are aggregated into a single image feature vector using attention pooling. The specific process of attention pooling is as follows: A learnable query vector is initialized; the dot product of the query vector and each image frame feature vector is calculated to obtain 11 attention scores; these 11 attention scores are normalized using the softmax function to obtain attention weights; these weights are then used to perform a weighted summation of the 11 image frame feature vectors, outputting a single 768-dimensional image feature vector; based on timestamps, behavioral event sequences within 5 seconds before and after the timestamp are extracted from the behavioral log data. The length of each 5-second interval was determined experimentally. The specific experimental process includes: using the same dataset of 500 real conference sessions... In each meeting, key behavioral events corresponding to each decision point were manually labeled. These key behavioral events included file sending, initiating a vote, screen sharing, and triggering a speech. Three interval lengths were tested: 3 seconds before and after (6 seconds total), 5 seconds before and after (10 seconds total), and 10 seconds before and after (20 seconds total). For each decision point's timestamp, behavioral event sequences were extracted according to each interval length, and two metrics were calculated: Decision-related behavior coverage, defined as the number of decision points in the extracted sequence that included at least one manually labeled key behavioral event divided by the total number of decision points; and the number of irrelevant events, defined as the number of events in the extracted sequence that did not belong to any of the specified categories. Experimental results showed that the coverage rate for 3 seconds before and after the decision point was [missing data]. The coverage rate is 72%, with an average of 0.8 irrelevant events. The coverage rate is 87% for the 5-second intervals before and after, with an average of 1.2 irrelevant events. The coverage rate is 93% for the 10-second intervals before and after, with an average of 3.5 irrelevant events. Considering all factors, the 5-second interval achieves 87% coverage, a 15 percentage point improvement over the 3-second interval, while only increasing the number of irrelevant events by 0.4. While the 10-second interval increases coverage to 93%, the number of irrelevant events doubles. Therefore, choosing the 5-second intervals before and after ensures high coverage while introducing the fewest irrelevant events. Each behavioral event is mapped to a fourth-dimensional embedding vector through a learnable behavioral embedding table. The fourth dimension is 128-dimensional, and the number of rows in this behavioral embedding table is equal to... The total number of predefined behavioral event categories is set based on the log recording type of the actual meeting management system, including but not limited to: screen sharing start, screen sharing end, speaking trigger, file sending, voting initiation, voting end, whiteboard activation, and recording start. This category list can be expanded according to the application scenario. New categories only require increasing the number of rows in the embedding table and randomly initializing the corresponding row vectors, without affecting the overall algorithm flow. For example, in a specific implementation, common meeting behavior categories can be defined as screen sharing start, screen sharing end, speaking trigger, file sending, voting initiation, and voting end. All embedded vectors in the behavioral event sequence are averaged and pooled to obtain a 128-dimensional behavioral feature vector.The text content of the decision point is converted into a first-dimensional text embedding vector using the BERT-base model, with a first dimension of 768 dimensions. In step S302, the feature vector includes a 768-dimensional text embedding vector, a 768-dimensional image feature vector, and a 128-dimensional behavior feature vector. The text embedding vector is projected into a fifth-dimensional text projection vector using a third linear projection matrix, which has a size of 256 rows and 768 columns. The image feature vector is projected into a 256-dimensional image projection vector using a fourth linear projection matrix, which has a size of 256 rows and 768 columns. The behavior feature vector is projected into a 256-dimensional behavior projection vector using a fifth linear projection matrix, which has a size of 256 rows and 128 columns. The projection vector includes text projection vectors and image projection vectors. The text and behavior projection vectors are used to map the three heterogeneous features to a unified 256-dimensional semantic space, facilitating subsequent similarity calculations. In step S303, the cosine similarity between the text projection vector and the image projection vector is calculated to obtain the text-image similarity; the cosine similarity between the text projection vector and the behavior projection vector is also calculated to obtain the text-behavior similarity; the text-image similarity and text-behavior similarity are weighted and summed to obtain the original confidence score, where the weight of text-image similarity is 0.6 and the weight of text-behavior similarity is 0.4. This weight combination is based on experiments, specifically including: dividing a dataset of 500 real conferences into 80% as the training set (400 conferences) and 20% as the validation set. The validation set (100 sessions) included each session where each decision point was manually labeled as either 1 or 0 to indicate its criticality. Three weight pairs were tested: (text-image similarity weight, text-behavior similarity weight) with values ​​of (0.5, 0.5), (0.6, 0.4), and (0.7, 0.3). For each weight pair, the final confidence score of each decision point in the validation set was calculated (using the weighted summation and linear mapping formula in step S303). Then, the Pearson correlation coefficient between the final confidence score and the manually labeled 0 / 1 was calculated. The average correlation coefficient of all decision points was used as the evaluation index for that weight pair. Experimental results showed that the weights... The correlation coefficient is 0.82 for (0.5, 0.5), 0.87 for (0.6, 0.4), and 0.84 for (0.7, 0.3). Therefore, (0.6, 0.4) is chosen as the optimal weight combination to maximize the correlation between the final confidence score and the manually labeled data. The original confidence score is linearly mapped to the closed interval [0, 1]. The linear mapping formula is that the final confidence score equals the original confidence score plus 1 and divided by 2. The original confidence score is obtained by weighted summation of two cosine similarities. The cosine similarity itself takes values ​​in the closed interval [-1, 1], so the weighted sum also takes values ​​in the closed interval [-1, 1]. After this mapping, the final confidence score falls within the closed interval [0, 1].The final confidence score is associated with each decision point in the second candidate decision point set to obtain the third candidate decision point set, where each decision point is accompanied by an interpretable confidence score between 0 and 1; By using multimodal cross-validation of video frames and behavior logs, the correlation between decision point confidence and manually labeled results is improved by about 25% compared to the single-text method, and the false detection rate is reduced by about 20%, which significantly enhances the reliability and practicality of key decision point output.

[0032] Example 2, refer to Figure 2 In another embodiment of the present invention, which differs from the first embodiment, an intelligent meeting management system based on a large model is provided, including an extraction module, a calculation module, and an association module; The extraction module extracts and processes the project sequences from multiple meetings to obtain an enhanced representation of the current meeting, and identifies the audio recording data of the current meeting to obtain a sequence of text segments. The calculation module projects the text fragment sequence and the enhanced meeting representation to obtain the context-aware fragment representation, calculates the dynamic importance weight of the context-aware fragment representation, obtains the first candidate decision point set based on the dynamic importance weight, and iteratively compresses the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. The association module extracts the feature vector of each decision point in the second candidate decision point set from the multimodal data of the current meeting, projects the feature vector into a projection vector, calculates the cosine similarity between the projection vectors to obtain the final confidence score, and associates the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

[0033] In one embodiment, the extraction module extracts project sequences from multiple meetings to obtain an enhanced representation of the current meeting. Simultaneously, it identifies the recorded data of the current meeting to obtain a text segment sequence, providing a semantic basis for subsequent decision point extraction, including temporal dependencies. The calculation module addresses the limitations of fixed threshold-based decision point selection by using dynamic importance weights to perceive the criticality of text segments. Specifically, it projects the text segment sequence and the enhanced meeting representation to obtain a context-aware segment representation, calculates dynamic importance weights, and obtains a first set of candidate decision points based on these weights. Then, it iterative compression using a dual-constraint strategy yields a second set of candidate decision points, thus achieving adaptive selection of decision points while controlling information fidelity and resource counting. The association module, to address the unreliability of single-text modality validation, introduces a multimodal cross-validation mechanism. Specifically, for each decision point in the second candidate decision point set, it extracts a feature vector from the multimodal data collected from the meeting, projects the feature vectors, calculates the cosine similarity between the projected vectors to obtain the final confidence score, and associates the confidence score with the corresponding decision point to obtain a third set of candidate decision points, improving the accuracy of the decision point confidence score.

[0034] The present invention was compared with existing fixed threshold methods and single-text modal methods on a dataset of 500 real conference sessions. The results are shown in Table 1: Table 1. Performance comparison between the method of the present invention and existing methods; This invention addresses two shortcomings of existing methods. In the decision point extraction stage, to address the poor adaptability of traditional fixed threshold methods, this invention introduces dynamic importance weights and a dual-constraint iterative compression strategy. Experiments show that on a dataset of 500 real conferences, the F1 score of decision point extraction improved from 0.75 to 0.86, an improvement of 15%, while the resource count variance decreased by 30%, effectively solving the problems of information redundancy or omission of key decision points. In the decision point verification stage, to address the problem of inaccurate confidence assessment due to reliance on single-text modality, this invention integrates video frames and behavior logs for multimodal cross-validation. The results show that the correlation coefficient between decision point confidence and manual annotation increased from 0.70 to 0.87, a relative improvement of 25%, while the false detection rate decreased by 20%. In summary, this invention significantly outperforms existing methods in both accuracy and reliability of key decision point identification.

[0035] This invention addresses the existing fixed threshold problem by introducing dynamic importance weights based on word frequency and context awareness, and designing a dual-constraint iterative compression strategy to jointly optimize fidelity and resource counting. This preserves key semantics while controlling the number of decision points. By fusing multi-level associations and reverse positional encodings from multiple meetings through a hypergraph attention network, the representational ability of historical meeting sequences is enhanced. Multimodal projection and weighted cosine similarity are used, and video and behavioral logs are fused for cross-validation, significantly improving the confidence of decision points and the accuracy of meeting management.

[0036] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0037] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention. All data acquisition actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located and with the authorization granted by the owner of the corresponding device.

Claims

1. A smart meeting management method based on a large model, characterized in that: Includes the following steps: Step S1: Extract and process the project sequence of multiple meetings to obtain the enhanced representation of the current meeting, and identify the audio data of the current meeting to obtain a text segment sequence; Step S2: Project the text segment sequence and the enhanced meeting representation to obtain the context-aware segment representation, calculate the dynamic importance weight of the context-aware segment representation, obtain the first candidate decision point set based on the dynamic importance weight, and iteratively compress the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. Step S3: For each decision point in the second candidate decision point set, extract the feature vector of the decision point from the multimodal data of the current meeting, project the feature vector into a projection vector, calculate the cosine similarity between the projection vectors, obtain the final confidence score, and associate the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

2. The intelligent meeting management method based on a large model as described in claim 1, characterized in that: Step S1 specifically includes: Step S101, the extraction process specifically includes: Extract the time features for each meeting item, including the number of days between adjacent meetings and the meeting duration; Connect two consecutive meeting items to form the first hyperedge, connect three consecutive meeting items to form the second hyperedge, and connect four consecutive meeting items to form the third hyperedge; Based on the first, second, and third hyperedges, generate an association matrix between nodes and hyperedges. The rows of the association matrix correspond to the nodes of the meeting items, and the columns correspond to each hyperedge. When the value of an element in the incidence matrix is ​​1, it means that the node belongs to the corresponding hyperedge; when the value of an element in the incidence matrix is ​​0, it means that the node does not belong to the corresponding hyperedge. The association matrix is ​​input into the hypergraph attention network, which sequentially performs bidirectional information transfer from node to hyperedge and from hyperedge to node, and outputs a preliminary contextual feature representation for each meeting item. Step S102: The preliminary contextual feature representation and corresponding temporal features of each meeting are fused using an attention mechanism to obtain fused features; After adding inverse positional encoding to the fused features, a self-attention mechanism is used to extract session features, resulting in an enhanced meeting representation of the current meeting; Acquire the audio recording data of the current meeting, perform speech recognition on the audio recording data using the Whisper model to obtain the original text record, and segment the original text record according to sentence boundaries to obtain the text segment sequence of the current meeting; The sentence boundaries are periods, question marks, or exclamation marks.

3. The intelligent meeting management method based on a large model as described in claim 2, characterized in that: Step S2 specifically includes: Step S201: Use the BERT-base model to convert each text segment into a text embedding vector of the first dimension; The text embedding vector is mapped to a first projection vector in the second dimension through a first linear projection matrix; The enhanced conference representation is mapped to a second-dimensional second projection vector through a second linear projection matrix; The first projection vector is concatenated with the second projection vector to obtain a third-dimensional context-aware fragment representation; The enhanced conference representation is then mapped to a third-dimensional third projection vector via a third linear projection matrix.

4. The intelligent meeting management method based on a large model as described in claim 3, characterized in that: Step S2 also includes: Calculate the dynamic importance weight of each context-aware segment representation to obtain the importance weight of each segment; The expression for dynamic importance weights is: ； in, Indicates the first The importance weights of each context-aware segment Indicates the first Normalized word frequencies of a text segment Indicates the first Context-aware fragment representation of a text segment Represents the third projection vector. Represents the L2 norm. This represents a very small constant.

5. The intelligent meeting management method based on a large model as described in claim 4, characterized in that: Step S2 also includes: From all segments with importance weights, segments with importance weights greater than or equal to the current threshold are selected to form the first candidate decision point set; Calculate the fidelity and resource count of the first candidate decision point set; The fidelity calculation process specifically includes: Obtain context-aware segment representations of all text segments and construct the first matrix; Based on the first set of candidate decision points, generate a second matrix with the same size as the first matrix; In the second matrix, the rows corresponding to the segments within the first candidate decision point set retain the context-aware segment representation, while the remaining rows are set to zero vectors; Calculate the mean square error between the first matrix and the second matrix, and use the mean square error as the fidelity. The resource count is the number of segments in the first candidate decision point set.

6. The intelligent meeting management method based on a large model as described in claim 5, characterized in that: Step S2 also includes: The dual-constraint strategy is applied to the first set of candidate decision points, and the dual-constraint strategy specifically includes: The current threshold is iteratively adjusted, and the iterative compression specifically includes: If the fidelity is greater than the fidelity limit and the resource count is less than the resource limit, then subtract the learning rate from the current threshold. If the resource count is greater than the resource limit and the fidelity is less than the fidelity limit, then add the learning rate to the current threshold. If the fidelity is greater than the fidelity limit and the resource count is greater than the resource limit, then add half of the learning rate to the current threshold. If the fidelity is less than the fidelity limit and the resource count is less than the resource limit, subtract the learning rate from the current threshold. Repeatedly compress until the convergence condition is met or the maximum number of iterations is reached, to obtain the second candidate decision point set. Each segment in the second candidate decision point set includes the text content of the corresponding segment and the corresponding timestamp. Convergence conditions include: The absolute difference between fidelity and fidelity limit is less than the first preset threshold, and the relative difference between resource count and resource limit is less than the second preset threshold.

7. The intelligent meeting management method based on a large model as described in claim 6, characterized in that: Step S3 specifically includes: Step S301: Obtain each decision point in the second candidate decision point set. Each decision point includes text content and a corresponding timestamp. Acquire multimodal data from the current meeting, including video data and behavioral log data; Based on the timestamp, extract 5 video frames before and after the timestamp from the video data to obtain 11 video frames. Scale each video frame to a uniform 224×224 pixel RGB format. The ViT-B / 16 model is used to convert each frame into a first-dimensional image frame feature, resulting in 11 first-dimensional image frame feature vectors. Attention pooling is used to aggregate the feature vectors of 11 image frames into a single image feature vector. Based on the timestamp, extract the sequence of behavioral events within 5 seconds before and after the timestamp from the behavioral log data, and map each behavioral event to a fourth-dimensional embedding vector through a learnable behavioral embedding table; The number of rows in the behavior embedding table is equal to the total number of predefined behavior event categories; Average pooling is performed on all embedded vectors in the behavioral event sequence to obtain the fourth-dimensional behavioral feature vector; The text content of the decision point is converted into a first-dimensional text embedding vector using the BERT-base model.

8. The intelligent meeting management method based on a large model as described in claim 7, characterized in that: Step S3 also includes: Step S302, the feature vector includes text embedding vector, image feature vector and behavior feature vector; The text embedding vector is projected into a fifth-dimensional text projection vector through the third linear projection matrix; The image feature vector is projected into a fifth-dimensional image projection vector through the fourth linear projection matrix; The behavior feature vector is projected into a fifth-dimensional behavior projection vector through the fifth linear projection matrix; Projection vectors include text projection vectors, image projection vectors, and behavior projection vectors.

9. The intelligent meeting management method based on a large model as described in claim 8, characterized in that: Step S3 also includes: Step S303: Calculate the cosine similarity between the text projection vector and the image projection vector to obtain the text-image similarity. Calculate the cosine similarity between the text projection vector and the behavior projection vector to obtain the text-behavior similarity; The original confidence score is obtained by weighted summation of text-image similarity and text-behavior similarity. The original confidence level is linearly mapped to the [0,1] interval to obtain the final confidence level; The final confidence level is associated with each decision point in the second candidate decision point set to obtain the third candidate decision point set.

10. A large-scale model-based intelligent meeting management system, applied to the large-scale model-based intelligent meeting management method as described in any one of claims 1-9, characterized in that, It includes an extraction module, a calculation module, and an association module; The extraction module extracts and processes the project sequences from multiple meetings to obtain an enhanced representation of the current meeting, and identifies the audio recording data of the current meeting to obtain a sequence of text segments. The calculation module projects the text fragment sequence and the enhanced meeting representation to obtain the context-aware fragment representation, calculates the dynamic importance weight of the context-aware fragment representation, obtains the first candidate decision point set based on the dynamic importance weight, and iteratively compresses the first candidate decision point set through a dual-constraint strategy to obtain the second candidate decision point set. The association module extracts the feature vector of each decision point in the second candidate decision point set from the multimodal data of the current meeting, projects the feature vector into a projection vector, calculates the cosine similarity between the projection vectors to obtain the final confidence score, and associates the final confidence score with the corresponding decision point to obtain the third candidate decision point set.

Images