Intelligent conference memo generation method based on robot

Abstract

The invention provides an intelligent conference memo generation method based on a robot, and the method comprises the steps: collecting a multi-channel voice signal through a microphone array, and enhancing the voice of a target speaker through a beam forming technology; background noise is separated by adopting a self-adaptive filtering algorithm, and the voice signal quality is improved; speech features are extracted, language model parameters are adjusted, and a transliteration text is generated; analyzing the text structure, and extracting conference themes, participants and decision contents to form structured information; calculating semantic similarity among the knowledge graph nodes, and if the semantic similarity is higher than a preset threshold value, associating historical records to generate extension information; constructing a structured memorandum based on the extended information, organizing a conference theme, participants, decision contents and associated historical records, and generating an initial memorandum; and monitoring a memorandum editing operation, updating a knowledge graph node relationship, and generating a final memorandum document. According to the method, the accuracy, integrity and availability of conference records are effectively improved, and the conference efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular, to an intelligent meeting memo generation method based on a robot. Background Art

[0002] The real-time speech transcription, key information extraction, and intelligent memo generation of portable robots in multi-scenario calls are important research directions in the fields of artificial intelligence and speech interaction. The core lies in providing users with convenient meeting record and information management support in complex acoustic environments through efficient speech processing and intelligent analysis technologies. This technology has key application values in scenarios such as business meetings, remote collaboration, and daily communication, and can significantly improve information processing efficiency and user experience. However, existing solutions have obvious limitations in multi-scenario adaptability, real-time performance, and intelligence level. Many systems have a decline in speech recognition accuracy in noisy environments and are difficult to effectively separate the target speaker's voice from background noise. At the same time, the accuracy of key information extraction is insufficient, and the generated memo lacks structure and intelligence, unable to meet users' needs for quick query and editing.

[0003] Therefore, how to achieve efficient processing of speech signals, accurate extraction of key information, and automatic generation of structured intelligent memos in multi-scenario calls by optimizing microphone array noise reduction, real-time speech transcription, natural language processing, and semantic understanding technologies under the limited computing resources of portable robots has become a key issue in this research. Summary of the Invention

[0004] The present invention provides an intelligent meeting memo generation method based on a robot, mainly including S101. Collecting original speech data through a microphone array to obtain multi-channel speech signals, adjusting the directivity parameters based on the speech signals by using beamforming technology, calculating the direction angle of the target speaker, and obtaining enhanced speech signals; S102. Separating background noise by using an adaptive filtering algorithm according to the enhanced speech signals to obtain high signal-to-noise ratio speech signals; S103. Extracting speech features from the high signal-to-noise ratio speech signals, and generating a transcribed text by using a pre-trained model combined with fine-tuning of meeting domain corpus; S104. Analyzing the sentence structure of the transcribed text, and extracting the meeting theme, participants, and resolution content to form a structured information set; S105. Calculating the semantic similarity between knowledge graph nodes according to the structured information set. If the similarity is higher than a second preset threshold, an extended information set is generated by associating historical records; S106. Constructing a structured memo based on the extended information set, organizing the meeting theme, participants, resolution content, and associated historical records, and generating an initial memo document; S107. Monitoring the editing operation of the initial memo document, updating the knowledge graph node relationship, and generating a memo document.

[0005] Preferably, the step S101 includes: Collecting original voice data through the microphone array to obtain multi-channel signals. If there is noise in the multi-channel signals, spatial filtering technology is used for preprocessing to remove background noise and obtain a first voice signal; According to the first voice signal, beamforming technology is used to adjust the directivity parameter, and the spatial focusing is optimized through the minimum variance distortionless response algorithm to obtain a second voice signal; If the direction of the target speaker in the second voice signal is not clear, the direction angle is calculated based on the phase difference analysis through the direction angle estimation technology to determine the position of the target speaker; According to the direction angle, the beamforming parameter is adjusted, and the target speaker signal is enhanced through spatial filtering to obtain a third voice signal; For the third voice signal, noise suppression is performed by combining a deep learning noise reduction model with spectral subtraction technology to remove residual noise and obtain an enhanced voice signal.

[0006] Preferably, the step S102 includes: Based on the voice signal, the voice signal is decomposed by time domain analysis technology to extract the main voice signal; For the main voice signal, the interference frequency distribution is calculated by spectral analysis technology to determine the interference frequency band; According to the interference frequency band, notch filtering technology is used to suppress specific frequency interference to obtain a filtered voice signal; For the filtered voice signal, the filtering coefficient is adjusted by adaptive filtering technology. If the signal-to-noise ratio is lower than the second preset threshold, the filtering coefficient is iteratively updated to obtain a high signal-to-noise ratio voice signal.

[0007] Preferably, the step S103 includes: Extracting time domain features from the high signal-to-noise ratio voice signal, and using a feature extraction method combining Mel frequency cepstral coefficients and convolutional neural network to obtain a time-frequency feature distribution; According to the time-frequency feature distribution, dimensionality reduction processing is performed by principal component analysis technology to obtain dimensionality-reduced voice features; For the dimensionality-reduced voice features, if the variance of the feature vector is lower than the third preset threshold, optimization is performed through regularization technology to obtain optimized voice features; Semantic information is extracted from the optimized voice features, and feature mapping is performed using a pre-trained language model to obtain a semantic feature vector; According to the semantic feature vector, a preliminary transcription text is generated by fine-tuning the pre-trained model combined with conference domain corpus; For the preliminary transcription text, calculate the text semantic similarity through word embedding technology. If the text semantic similarity is lower than the fourth preset threshold, iterate and update the language model parameters until the similarity of the generated transcription text is greater than or equal to the preset threshold.

[0008] Preferably, step S104 includes: Use dependency syntax analysis to parse the sentence structure in the transcription text to obtain a syntactic dependency tree; According to the syntactic dependency tree, extract the subject-predicate-object structure, and identify the subject, predicate, and object through syntactic role annotation technology to obtain the subject-predicate-object structure; If the subject or object in the subject-predicate-object structure contains a noun phrase, decompose the noun phrase through keyword extraction technology to obtain a set of meeting topic words; Obtain named entities from the syntactic dependency tree, and use named entity recognition technology to annotate personal names and organization names to obtain a list of participants; According to the subject-predicate-object structure and the set of meeting topic words, use semantic role annotation technology to identify the core verb and its associated arguments to obtain a set of resolution contents; For the set of resolution contents and the list of participants, generate a set of structured information through information integration technology.

[0009] Preferably, step S105 includes: Obtain knowledge graph nodes from the set of structured information, and use semantic analysis technology to calculate the semantic similarity between nodes to obtain a set of similarities; If the similarity exceeds the preset threshold, extract relevant node association data from the historical record to obtain a set of association records; According to the set of association records, use knowledge graph extension technology to generate extended nodes to obtain a set of extended nodes; Obtain node attributes from the set of extended nodes, and use information integration technology to generate extended information to obtain a set of extended information.

[0010] Preferably, step S106 includes: Obtain meeting data from the set of extended information. The meeting data includes meeting topics, participants, resolution contents, and historical records, and use data extraction technology to generate a set of extracted data; Generate a set of classified data through information classification technology based on the set of extracted data to obtain a set of classified data; Obtain classified data from the set of classified data, and use integrity verification technology to verify whether the classified data meets the preset integrity conditions. If the verification result shows that the classified data meets the integrity conditions, generate an initial memorandum document through a structured template; Retrieve a memo document from an initial memo document, generate a formatted memo using formatting techniques, and determine the relevance between the formatted memo and historical records using association analysis techniques to obtain a relevance analysis result; According to the relevance analysis result, retrieve associated data from historical records, and integrate the associated data into the formatted memo using information integration techniques to obtain an initial memo document.

[0011] Preferably, step S107 includes: Retrieve an editing operation from the initial memo document, extract operation data using operation capture techniques to obtain an operation data set; Extract operation data from the operation data set and generate an operation feature set using data parsing techniques; Retrieve an operation feature from the operation feature set and update the node relationships of the knowledge graph using graph mapping techniques to obtain an updated set of node relationships; Extract node relationships from the updated set of node relationships, and determine whether the node relationships meet preset consistency conditions using relationship verification techniques. If they meet, determine the updated knowledge graph; Retrieve node relationships from the updated knowledge graph and generate a memo document using document generation techniques to obtain a set of memo documents.

[0012] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: The present invention discloses an intelligent conference memo generation method based on a robot. In response to the challenges of speech recognition and content extraction in a multi-person conference environment, beamforming technology and an adaptive filtering algorithm are used to improve the quality of speech signals, and a domain-adaptive language model is used to achieve accurate transcription. Key information of the conference is extracted through semantic analysis and a knowledge graph is constructed, and then a structured memo is generated. The present invention also integrates historical record extension information, optimizes the content of the memo and establishes a semantic index, and finally forms an interactive dynamic memo document. This method can effectively improve the accuracy, integrity, and usability of conference records, provide an intelligent conference management solution for enterprises, and significantly improve conference efficiency and knowledge precipitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flowchart of an intelligent conference memo generation method based on a robot according to the present invention.

[0014] Figure 2 It is a schematic flowchart of step S101 according to the present invention.

[0015] Figure 3 It is a schematic flowchart of step S106 according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and detailedly described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention.

[0017] As shown in the attached Figure 1 , a method for generating an intelligent meeting memorandum based on a robot in this embodiment may specifically include: Step S101, collect the original voice data through a microphone array, obtain a multi-channel voice signal, adjust the directivity parameter by using beamforming technology, calculate the direction angle of the target speaker, and obtain an enhanced voice signal.

[0018] In this embodiment, referring to the attached Figure 2 , collect the original voice data through a microphone array to obtain a multi-channel voice signal. If there is noise in the multi-channel signal, preprocess it by using spatial filtering technology to remove background noise and obtain a first voice signal. According to the first voice signal, adjust the directivity parameter by using beamforming technology, optimize spatial focusing through the minimum variance distortionless response algorithm, and obtain a second voice signal. If the direction of the target speaker in the second voice signal is not clear, calculate the direction angle based on phase difference analysis through direction angle estimation technology to determine the position of the target speaker. According to the direction angle, adjust the beamforming parameter, and enhance the target speaker signal through spatial filtering to obtain a third voice signal. For the third voice signal, combine a deep learning noise reduction model with spectral subtraction technology to suppress noise and remove residual noise to obtain an enhanced voice signal.

[0019] Exemplarily, to collect a multi-channel voice signal through a microphone array, a linear microphone array can be used, which includes 4 microphones with a spacing of 5 cm and is placed inside a portable robot to collect the original voice data. The signal sampling rate is 16 kHz, and time series data is generated for each channel. The collected signal includes the voice of the target speaker, air conditioner noise, and distant noise, and needs to be further processed to improve the voice quality.

[0020] In a possible implementation manner, for the noise in the multi-channel signal, preprocessing can be performed through spatial filtering. For example, use a delay and sum filter to suppress background noise from non-target directions based on the time difference between microphones.

[0021] Specifically, assume that the noise mainly comes from the side. By adjusting the filter parameters, attenuate the signal in the 90-degree direction and retain the front voice to output the first voice signal. This process effectively reduces environmental noise and improves the signal-to-noise ratio.

[0022] Specifically, based on the first voice signal, beamforming technology can be used to optimize the directivity.

[0023] Preferably, the minimum variance distortionless response algorithm is used to focus on the target direction. For example, the beam is set to point to the center of the conference table. The minimum variance distortionless response algorithm suppresses lateral interference by adaptively adjusting the weights and outputs a second speech signal. This signal has higher speech clarity in the target direction and is suitable for multi-person meeting scenarios.

[0024] It should be noted that if the direction of the target speaker in the second speech signal is not clear, it can be located through the direction angle estimation technology. For example, based on the phase difference analysis, the arrival time difference of the signals received by each microphone is calculated to estimate the direction angle. Assuming that the calculated result is 30 degrees, it indicates that the speaker is located slightly to the right in front of the array. The beamforming parameters are adjusted to focus on the 30-degree direction, and the target signal is enhanced through spatial filtering to output a third speech signal. This method improves the separation degree of the target speech and is applicable to dynamic scenarios.

[0025] For example, for the residual noise in the third speech signal, spectral subtraction can be used for noise suppression.

[0026] In one embodiment, the signal spectrum is analyzed to identify the frequency bands dominated by noise, such as the air conditioner noise in the high-frequency band. By subtracting the estimated noise spectrum, the main frequency components of the speech are retained to obtain an enhanced speech signal. The background noise of this signal is significantly reduced, the speech is clearer, and it is suitable for input to the speech recognition system.

[0027] It can be understood that the above technologies are combined to form a complete speech enhancement link, which gradually improves the speech quality from acquisition to final output. Spatial filtering reduces background interference, beamforming focuses on the target, direction angle estimation adjusts dynamically, and spectral subtraction removes noise finely. The overall effect shows that the signal-to-noise ratio is increased by about 10 dB, the speech intelligibility is improved, it is applicable to intelligent conference systems, and ensures the efficiency and robustness of far-field speech interaction.

[0028] Step S102, according to the enhanced speech signal, an adaptive filtering algorithm is used to separate the background noise to obtain a high signal-to-noise ratio speech signal; In the embodiment of the present application, based on the enhanced speech signal, a time-domain analysis technology is used to decompose the speech signal to extract the main speech signal; for the main speech signal, a spectral analysis technology is used to calculate the interference frequency distribution to determine the interference frequency band; according to the interference frequency band, a notch filtering technology is used to suppress specific frequency interference to obtain a filtered speech signal; for the filtered speech signal, an adaptive filtering technology is used to adjust the filtering coefficient. If the signal-to-noise ratio is lower than the second preset threshold, the filtering coefficient is iteratively updated to obtain a high signal-to-noise ratio speech signal.

[0029] Exemplarily, for the time-domain analysis technology to decompose the signal components, the enhanced speech signal can be decomposed into multiple time-domain subbands through short-time Fourier transform to extract the main speech signal.

[0030] In a possible implementation, assume that in a conference room scenario, the signal sampling rate collected by the microphone is 16 kHz, which includes the speech of the target speaker and background noise, such as the low-frequency hum of the air conditioner. Time-domain analysis first identifies the time periods with higher energy in the signal and determines them as the main speech regions. For example, set the time window to 20 milliseconds, analyze the energy peaks of each window, extract the energy-concentrated regions as the main speech signals, and generate the main speech signals. This process effectively separates the speech from the persistent low-energy noise. For example, assume that the spectrum shows that the frequency band from 200 Hz to 300 Hz is the dominant frequency band of the air conditioner noise and can be marked as the interference frequency band. This analysis provides an accurate target for subsequent filtering and avoids misinterfering with the speech frequency band.

[0031] In one embodiment, for the interference frequency band, notch filtering technology can effectively suppress the interference of specific frequencies.

[0032] Preferably, design a notch filter with a center frequency of 250 Hz and a bandwidth of 50 Hz to attenuate the signals in this frequency band. For example, in a conference scenario, in the filtered speech signal output after filtering, the air conditioner noise is significantly reduced, and the main frequency components of the speech, such as the signals from 500 Hz to 2 kHz, are retained. This method precisely targets the interference and maintains the naturalness of the speech.

[0033] For the filtered speech signal, adaptive filtering technology can further optimize the signal quality. In a possible implementation, the recursive least squares (RLS) algorithm is used to dynamically adjust the filtering coefficients. Assume that the initial signal-to-noise ratio is 5 dB, which is lower than the preset threshold of 10 dB. The algorithm iteratively updates the coefficients to gradually enhance the target speech. For example, analyze 100 milliseconds of the signal in each iteration, adjust the filter to suppress the residual high-frequency noise, and output a speech signal with a high signal-to-noise ratio. This process improves the speech clarity and adapts to the dynamic noise environment.

[0034] Step S103, extract speech features from the high signal-to-noise ratio speech signal, and use a pre-trained model combined with conference domain corpus for fine-tuning to generate a transcription text.

[0035] In this embodiment, time-domain features are extracted from the high signal-to-noise ratio speech signal, and a feature extraction method combining Mel Frequency Cepstral Coefficients (MFCC) and Convolutional Neural Network (CNN) is used to decompose the time-domain features to obtain a time-frequency feature distribution. According to the time-frequency feature distribution, principal component analysis technology is used for dimensionality reduction processing to obtain dimensionality-reduced speech features. For the dimensionality-reduced speech features, if the variance of the feature vector is lower than the third preset threshold, regularization technology is used for optimization to obtain optimized speech features. Semantic information is extracted from the optimized speech features, and a pre-trained language model is used for feature mapping to obtain a semantic feature vector. According to the semantic feature vector, a pre-trained model is combined with conference domain corpus for fine-tuning to generate a preliminary transcription text. For the preliminary transcription text, the text semantic similarity is calculated through word embedding technology. If the text semantic similarity is lower than the fourth preset threshold, the language model parameters are iteratively updated until the similarity of the generated transcription text is greater than or equal to the preset threshold. Exemplarily, when extracting time-domain features from a high signal-to-noise ratio speech signal, a feature extraction method combining Mel Frequency Cepstral Coefficients and Convolutional Neural Network can be used to decompose the signal. Specifically, the original speech signal (sampling rate 16 kHz) is framed, with a frame length of 25 ms (400 sample points) and a frame shift of 10 ms (160 sample points). A Hamming window is used to reduce spectral leakage. The FFT of each frame of the signal is obtained to get the power spectrum. The auditory characteristics of the human ear are simulated through 20 triangular filters (frequency range 0 - 8 kHz) to compress the frequency-domain information. After taking the logarithm of the filter bank energy, DCT is performed to obtain 13-dimensional MFCC coefficients, and the first-order and second-order differences are appended, resulting in a 39-dimensional feature vector, which constitutes a time-frequency feature matrix. The time-frequency feature matrix is converted into a time-frequency image (time axis × frequency axis) as the input of the convolutional neural network, and the output is a 64-dimensional time-domain feature vector for subsequent classification or recognition.

[0036] In a possible implementation manner, based on the time-frequency feature distribution, principal component analysis technology is used for dimensionality reduction processing. Principal component analysis calculates the covariance matrix of the features, extracts the principal components that contribute the most to the variance, and retains the main speech features. For example, assuming that the time-frequency features contain 1000-dimensional data, principal component analysis retains the first 50 dimensions, covering 90% of the variance, to generate dimensionality-reduced speech features. This dimensionality reduction reduces the computational complexity while retaining the core speech information.

[0037] It should be noted that if the variance of the feature vector of the dimensionality-reduced speech features is lower than the third preset threshold, such as 0.8, the feature weights can be adjusted through regularization technology. Regularization enhances the discriminability of the features by introducing a penalty term. For example, L2 regularization is used to adjust the weights to enhance the influence of low-variance features and generate optimized speech features. This method improves the robustness of the features and adapts to different speaker styles.

[0038] Specifically, to extract semantic information from optimized speech features, a pre-trained language model can be used for feature mapping. Pre-trained models such as BERT convert speech features into semantic feature vectors through a deep network. For example, the optimized speech features are input into the model to generate semantic feature vectors, capturing the semantics of keywords such as "plan" and "budget" in the meeting. This mapping makes subsequent text generation more semantic-aware.

[0039] In one embodiment, based on the semantic feature vectors, a general speech or language pre-trained model is loaded and the model parameters are adjusted in combination with the meeting corpus, and the speech features are input to generate accurate text in the meeting scenario. Specifically, pre-trained model type adaptation: Speech recognition model: Select an open-source pre-trained model (such as Wav2Vec2.0, DeepSpeech) to directly process speech features to generate text. Language model: If combined with post-processing of speech recognition, a general language model (such as the GPT series, BERT) can be selected to optimize the grammar and semantics of the transcribed text. Load the pre-trained parameters from a public weight library (such as HuggingFace) as the basis for feature extraction and text generation. Adaptive strategies: Parameter-efficient fine-tuning (PEFT): Freeze most of the parameters of the pre-trained model and only fine-tune specific layers (such as the last few layers of the language model) to reduce the computational amount. Adversarial training: Introduce a domain adversarial network (DANN), and use a discriminator to distinguish between general and meeting domain data, enabling the model to learn domain-agnostic features. Transfer learning: Customize tasks for the meeting scenario (such as correcting misspelled words of professional terms and identifying specific sentence patterns in the meeting), and update the parameters through backpropagation. Fine-tuning parameter settings: Learning rate: Adopt a dynamic adjustment strategy (such as the AdamW optimizer, initial learning rate 1e-5); Number of training epochs: Set 3-8 epochs according to the data scale, and combine early stopping on the validation set to prevent overfitting, quickly generating structured meeting transcripts. Through the above solutions, the pre-trained model can effectively adapt to the speech transcription requirements in the meeting domain, significantly improving the accuracy and professionalism of text generation in high signal-to-noise ratio speech scenarios.

[0040] Preferably, for the preliminary transcribed text, the text semantic similarity is calculated through word embedding technology. Word embedding converts text into vectors and calculates the cosine similarity between the vectors. For example, if the similarity between the preliminary transcribed text and the reference text is lower than the fourth preset threshold, such as 0.9, the parameters of the language model are iteratively updated to continuously optimize the transcribed text, where the fourth preset threshold is set based on the extremely high similarity standard in the medical scenario and the requirement for extremely strict semantic consistency. By this method, text semantic consistency is ensured.

[0041] Step S104, parse the sentence structure of the transcribed text, extract the meeting theme, participants, and resolution content, and form a structured information set; In the embodiments of the present application, the dependency syntactic analysis is used to parse the sentence structure in the transcribed text to obtain a syntactic dependency tree; according to the syntactic dependency tree, the subject-predicate-object structure is extracted, and the subject, predicate, and object are identified through the syntactic role annotation technology to obtain the subject-predicate-object structure; if the subject or object in the subject-predicate-object structure contains a noun phrase, the noun phrase is decomposed through the keyword extraction technology to obtain a set of meeting theme words; the named entities are obtained from the syntactic dependency tree, and the person names and organization names are annotated through the named entity recognition technology to obtain a list of participants; according to the subject-predicate-object structure and the set of meeting theme words, the semantic role annotation technology is used to identify the core verb and its associated arguments to obtain a set of resolution contents; for the set of resolution contents and the list of participants, a structured information set is generated through the information integration technology.

[0042] Exemplarily, a syntactic dependency tree is obtained from the transcribed text. The dependency syntactic analysis technology constructs a tree structure by analyzing the grammatical relationships between words in a sentence. For example, in the meeting record text "Manager Zhang proposed to accelerate the project progress", the dependency syntactic analysis identifies "proposed" as the predicate center, "Manager Zhang" as the subject of the proposal, and "accelerate the project progress" as the object, generating a syntactic dependency tree that contains the dependency relationships between words. This tree structure clearly shows the grammatical hierarchy of the sentence and provides a basis for subsequent analysis.

[0043] In a possible implementation manner, the subject-predicate-object structure is extracted according to the syntactic dependency tree. The syntactic role annotation technology identifies the subject, predicate, and object by annotating the grammatical roles of words. For example, in the above sentence, "Manager Zhang" is annotated as the subject, "proposed" as the predicate, and "accelerate the project progress" as the object, obtaining the subject-predicate-object structure. This structure extraction retains the core semantics of the sentence and is convenient for further processing.

[0044] It should be noted that if the subject or object contains a noun phrase, the keyword extraction technology decomposes the noun phrase to generate a set of meeting theme words. For example, "accelerate the project progress" contains the noun phrase "project progress", which is decomposed into "project" and "progress" through keyword extraction, forming a set of meeting theme words. This decomposition focuses on the core content of the meeting and improves the accuracy of theme extraction.

[0045] Specifically, named entities are obtained from the syntactic dependency tree, and the person names and organization names are annotated through the named entity recognition technology to generate a list of participants. For example, in the text "Engineer Li and the R & D department discussed the technical solution", the named entity recognition annotates "Engineer Li" as the person name and "R & D department" as the organization name, generating a list of participants. This method accurately identifies the relevant personnel in the meeting and provides support for information integration.

[0046] In one embodiment, according to the subject-predicate-object structure and the set of meeting topic words, the semantic role labeling technology identifies the core verb and its associated arguments, and generates a set of resolution contents. For example, in the sentence "Director Wang decides to optimize the budget allocation", the semantic role labeling identifies "decides" as the core verb and "optimize the budget allocation" as the argument, and generates the resolution content "optimize the budget allocation". This analysis captures the core intention of the meeting resolution.

[0047] Preferably, for the set of resolution contents and the list of participants, the information integration technology generates a set of structured information. For example, integrating "Manager Zhang", "Project progress", and "Accelerate" into structured information: {Person: Manager Zhang, Topic: Project progress, Resolution: Accelerate}. This structured information clearly organizes the meeting content and facilitates subsequent processing.

[0048] Step S105, according to the set of structured information, calculate the semantic similarity between the nodes of the knowledge graph. If the similarity is higher than the first preset threshold, then generate an extended information set by associating with historical records; In this embodiment, obtain the knowledge graph nodes from the set of structured information, use semantic analysis technology to calculate the semantic similarity between the nodes, and obtain a similarity set; if the similarity exceeds the first preset threshold, extract the relevant node association data from the historical records to obtain an association record set; according to the association record set, use the knowledge graph extension technology to generate extended nodes to obtain an extended node set; obtain the node attributes from the extended node set, and use the information integration technology to generate extended information to obtain an extended information set.

[0049] Exemplarily, obtain the knowledge graph nodes from the set of structured information, and the semantic analysis technology calculates the similarity by analyzing the semantic features of the node content. Assume that the meeting record contains structured information: {Person: Manager Zhang, Topic: Project progress, Resolution: Accelerate}, map it to the knowledge graph nodes, and the node attributes include "Manager Zhang", "Project progress", and "Accelerate". The semantic analysis technology compares "Project progress" with the historical node "Development plan" based on the word vector model and calculates the semantic similarity to be 0.85. The first preset threshold is 0.8. Since 0.85 exceeds the threshold, historical record extraction is triggered, where the first preset threshold is set based on the medical scenario and the need for near-certain associations.

[0050] In a possible implementation, extract the relevant node association data from the historical records to form an association record set. The historical records contain the association data between the node "Development plan" and "Engineer Li" and "Adjust the plan". After extraction, a set is formed: {Development plan, Engineer Li, Adjust}. This set provides a basis for subsequent expansion, focuses on the meeting management field, and avoids interference from irrelevant operations.

[0051] Specifically, the knowledge graph extension technology generates extended nodes based on the associated record set. Based on the similarity between the "development plan" and the "project progress", the extended node "technology development progress" is generated, and the attributes include "technology" and "progress". The extended node set also includes the association between "Engineer Li" and "technology development progress" to ensure logical consistency among the nodes.

[0052] It should be noted that the information integration technology extracts attributes from the extended node set to generate extended information. For "technology development progress", the attributes "technology", "progress" and the associated person "Engineer Li" are integrated to form the extended information: {Subject: technology development progress, Person: Engineer Li, Resolution: accelerate}. This information retains the core semantics of the meeting and is convenient for further processing.

[0053] Step S106, construct a structured memorandum based on the extended information set, organize the meeting theme, participants, resolution content and associated historical records to generate an initial memorandum document; In the embodiment of the present application, refer to the appendix Figure 3 to obtain meeting data from the extended information set, where the meeting data includes the meeting theme, participants, resolution content and historical records, and use data extraction technology to generate an extraction data set; generate a classification data set through information classification technology based on the extraction data set to obtain a classification data set; obtain classification data from the classification data set, and use integrity verification technology to verify whether the classification data meets the preset integrity conditions. If the verification result shows that the classification data meets the integrity conditions, an initial memorandum document is generated through a structured template, where the structured template is usually based on domain requirements, standardization specifications and historical data precipitation; obtain the memorandum document from the initial memorandum document, generate a formatted memorandum through formatting technology, and use association analysis technology to judge the relevance between the formatted memorandum and the historical records to obtain an association analysis result; according to the association analysis result, obtain associated data from the historical records, and use information integration technology to incorporate the associated data into the formatted memorandum to obtain an initial memorandum document.

[0054] Exemplarily, obtain the meeting theme, participants, resolution content and historical records from the extended information set, and the data extraction technology generates an initial data set by parsing the structured data. For example, the extended information set contains the meeting record: the theme is "product optimization", the participants are "Manager Wang, Engineer Chen", the resolution is "adjust the plan", and the historical records contain nodes related to "product development". The data extraction technology parses this information into the initial data set: theme "product optimization", person "Manager Wang, Engineer Chen", resolution "adjust the plan", historical record "product development".

[0055] In a possible implementation manner, the information classification technology classifies the initial data set to generate a classification data set.

[0056] Specifically, the classification technology classifies information into four categories: theme, personnel, resolution, and historical records according to data attributes. For example, the theme is classified as "product optimization", the personnel are classified as "Manager Wang, Engineer Chen", the resolution is classified as "adjustment plan", and the historical record is classified as "product development". The classified data set clearly divides data types, facilitating subsequent processing.

[0057] It should be noted that the integrity verification technology determines whether the classified data meets the preset integrity conditions. The integrity conditions include that the theme is not empty, there is at least one person, and the resolution is clear. For example, the classified data set contains the complete theme "product optimization", two participants, and a clear resolution "adjustment plan", and the verification result is that the conditions are met. If the theme is missing, the verification fails and data needs to be supplemented.

[0058] Preferably, the structured template technology generates a memorandum document based on the classified data that passes the verification.

[0059] For example, using the template to fill in the theme "product optimization", the personnel "Manager Wang, Engineer Chen", and the resolution "adjustment plan", an initial memorandum document is generated: Product Optimization Meeting, Participants Manager Wang and Engineer Chen, Resolution Adjustment Plan. This document concisely records the core content of the meeting.

[0060] Step S107, monitor the editing operations of the initial memorandum document, update the node relationships in the knowledge graph, and generate a memorandum document; In the embodiment of the present application, obtain the editing operations from the initial memorandum document, extract operation data using the operation capture technology to obtain an operation data set; extract operation data from the operation data set, and generate an operation feature set using the data parsing technology; obtain operation features from the operation feature set, and update the node relationships in the knowledge graph using the graph mapping technology to obtain an updated node relationship set; Extract node relationships from the updated node relationship set, and use the relationship verification technology to determine whether the node relationships meet the preset consistency conditions. If they meet, determine the updated knowledge graph; obtain node relationships from the updated knowledge graph, and generate a memorandum document using the document generation technology to obtain a memorandum document set.

[0061] Exemplarily, when obtaining the editing operations from the interactive memorandum document, the operation capture technology extracts operation data by real-time monitoring of user interaction behaviors. For example, in the memorandum document of a product release meeting, the user may add comments to the "user experience optimization" part or modify the responsible person information. The operation capture technology records these behaviors and generates an operation data set, such as "Add comment: Optimize the interface design" "Modify responsible person: Change from Manager Wang to Manager Li". This method ensures that the operation data fully reflects the user's intention and provides a basis for subsequent analysis.

[0062] In a possible implementation, data parsing technology extracts operation data from an operation data set and generates an operation feature set.

[0063] Specifically, the types, targets, and contexts of technical analysis operations are analyzed to generate features such as "operation type: adding comments", "target: user experience optimization", and "context: interface design". For example, when parsing "adding comments: optimizing interface design", the technology identifies its relevance to the "user experience" theme and generates a feature set. This parsing ensures the accuracy of the features and facilitates subsequent knowledge graph updates.

[0064] It should be noted that the knowledge graph mapping technology updates the node relationships of the knowledge graph based on operation features. For example, the original node "user experience optimization" in the knowledge graph is associated with "Manager Wang". Based on the operation feature "modify responsible person: Manager Li", the technology updates the node relationship to "user experience optimization - Manager Li". The updated set of node relationships reflects the latest responsibility assignment. This mapping method maintains the dynamics of the knowledge graph.

[0065] Specifically, the relationship verification technology checks whether the node relationships meet the consistency conditions. For example, it verifies whether "user experience optimization - Manager Li" is consistent with the document context. If the document mentions that Manager Li is responsible for interface design, the condition is met, and the updated knowledge graph is determined. This verification ensures the logical rigor of the knowledge graph.

[0066] In an embodiment, the document generation technology generates a memorandum document using the updated knowledge graph. For example, based on the node "user experience optimization - Manager Li", a document containing the content "User experience optimization is the responsibility of Manager Li" is generated, resulting in a set of memorandum documents. This generation method ensures the consistency between the document content and the knowledge graph.

[0067] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these changes and modifications.

Claims

1. A robot-based intelligent meeting memorandum generation method, characterized in that, The method includes the following steps: S101. Collect original speech data through a microphone array to obtain a multi-channel speech signal. Based on the speech signal, use beamforming technology to adjust the directivity parameter, calculate the direction angle of the target speaker, and obtain an enhanced speech signal; S102. According to the enhanced speech signal, use an adaptive filtering algorithm to separate background noise to obtain a speech signal with a high signal-to-noise ratio; S103. Extract speech features from the speech signal with a high signal-to-noise ratio, and use a pre-trained model combined with fine-tuning of conference domain corpus to generate a transcribed text; S104. Analyze the sentence structure of the transcribed text, extract the conference theme, participants, and resolution content to form a structured information set; S105. According to the structured information set, calculate the semantic similarity between nodes in the knowledge graph. If the similarity is higher than the first preset threshold, associate historical records to generate an extended information set; S106. Based on the extended information set, construct a structured memorandum, organize the conference theme, participants, resolution content, and associated historical records to generate an initial memorandum document; S107. Monitor the editing operation of the initial memorandum document, update the node relationship in the knowledge graph, and generate a memorandum document.

2. The method according to claim 1, wherein The step S101 includes: Collect original speech data through the microphone array to obtain a multi-channel signal. If there is noise in the multi-channel signal, use spatial filtering technology for preprocessing to remove background noise and obtain a first speech signal; According to the first speech signal, use beamforming technology to adjust the directivity parameter, and optimize spatial focusing through the minimum variance distortionless response algorithm to obtain a second speech signal; If the direction of the target speaker in the second speech signal is not clear, use direction angle estimation technology to calculate the direction angle based on phase difference analysis to determine the position of the target speaker; According to the direction angle, adjust the beamforming parameter, and enhance the target speaker signal through spatial filtering to obtain a third speech signal; For the third speech signal, combine a deep learning denoising model with spectral subtraction technology to suppress noise and remove residual noise to obtain an enhanced speech signal.

3. The method according to claim 1, wherein The step S102 includes: Based on the speech signal, use time-domain analysis technology to decompose the speech signal and extract the main speech signal; For the main speech signal, use spectral analysis technology to calculate the interference frequency distribution and determine the interference frequency band; According to the interference frequency band, use notch filtering technology to suppress specific frequency interference to obtain a filtered speech signal; For the filtered speech signal, use adaptive filtering technology to adjust the filtering coefficient. If the signal-to-noise ratio is lower than the second preset threshold, iteratively update the filtering coefficient to obtain a speech signal with a high signal-to-noise ratio.

4. The method according to claim 1, wherein The step S103 includes: Extract time-domain features from the speech signal with a high signal-to-noise ratio, and use a feature extraction method combining Mel frequency cepstral coefficients and a convolutional neural network to obtain a time-frequency feature distribution; According to the time-frequency feature distribution, use principal component analysis technology for dimensionality reduction processing to obtain dimensionality-reduced speech features; For the dimension-reduced speech features, if the variance of the feature vector is lower than the third preset threshold, optimize it through regularization technology to obtain optimized speech features; Extract semantic information from the optimized speech features, and use a pre-trained language model for feature mapping to obtain semantic feature vectors; According to the semantic feature vectors, use a pre-trained model combined with fine-tuning of meeting domain corpus to generate a preliminary transcription text; For the preliminary transcription text, calculate the text semantic similarity through word embedding technology. If the text semantic similarity is lower than the fourth preset threshold, iteratively update the language model parameters until the similarity of the generated transcription text is greater than or equal to the preset threshold.

5. The method according to claim 1, characterized in that, The step S104 includes: Use dependency syntax analysis to parse the sentence structure in the transcription text to obtain a syntactic dependency tree; According to the syntactic dependency tree, extract the subject-predicate-object structure, and identify the subject, predicate, and object through syntactic role annotation technology to obtain the subject-predicate-object structure; If the subject or object in the subject-predicate-object structure contains a noun phrase, decompose the noun phrase through keyword extraction technology to obtain a set of meeting topic words; Obtain named entities from the syntactic dependency tree, and use named entity recognition technology to annotate personal names and organization names to obtain a list of participants; According to the subject-predicate-object structure and the set of meeting topic words, use semantic role annotation technology to identify the core verb and its associated arguments to obtain a set of resolution contents; For the set of resolution contents and the list of participants, generate a structured information set through information integration technology.

6. The method according to claim 5, characterized in that The step S105 includes: Obtain knowledge graph nodes from the structured information set, and use semantic analysis technology to calculate the semantic similarity between nodes to obtain a set of similarities; If the similarity exceeds the preset threshold, extract relevant node association data from the historical record to obtain a set of association records; According to the set of association records, use knowledge graph extension technology to generate extended nodes to obtain a set of extended nodes; Obtain node attributes from the set of extended nodes, and use information integration technology to generate extended information to obtain a set of extended information.

7. The method according to claim 1, wherein The step S106 includes: Obtain meeting data from the set of extended information. The meeting data includes meeting topics, participants, resolution contents, and historical records, and use data extraction technology to generate a set of extracted data; Generate a set of classified data based on the set of extracted data through information classification technology to obtain a set of classified data; Obtain classified data from the set of classified data, and use integrity verification technology to verify whether the classified data meets the preset integrity conditions. If the verification result shows that the classified data meets the integrity conditions, generate an initial memorandum document through a structured template; Obtain a memorandum document from the initial memorandum document, use formatting technology to generate a formatted memorandum, and use association analysis technology to judge the relevance of the formatted memorandum to the historical record to obtain a relevance analysis result; According to the relevance analysis result, obtain associated data from the historical record, and use information integration technology to incorporate the associated data into the formatted memorandum to obtain an initial memorandum document.

8. The method according to claim 1, wherein The step S107 includes: Obtain editing operations from the initial memorandum document, extract operation data using operation capture technology, and obtain an operation data set; Extract operation data from the operation data set, and generate an operation feature set using data parsing technology; Obtain operation features from the operation feature set, update the node relationships of the knowledge graph using graph mapping technology, and obtain an updated node relationship set; Extract node relationships from the updated node relationship set, and use relationship verification technology to determine whether the node relationships meet the preset consistency conditions. If they meet, determine the updated knowledge graph; Obtain node relationships from the updated knowledge graph, and generate a memorandum document using document generation technology to obtain a memorandum document set.

Images