A conference recording method based on internet of things

Abstract

The application discloses a conference recording method based on Internet of Things, and relates to the technical field of conference recording.The application adopts a microphone array and a direction sensor to position a speaker, and obtains individual voice features in combination with a wearable microphone, matches the voice with the identity of the speaker, and distinguishes the voice streams of different speakers in combination with a turn-taking detection technology in the speaker identification process, and generates a speaking time axis.For voice transcription, background noise is removed through a noise reduction technology, and semantic compensation is introduced, text is corrected in combination with projection and whiteboard writing data collected by IoT devices, and the identification accuracy is improved;additional theme segmentation is carried out, a semantic vectorization method is adopted to convert the text into a high-dimensional vector, a similarity calculation and a clustering algorithm are utilized to determine the theme boundary, IoT position data and individual voice features are combined for dynamic adjustment, non-continuous discussions on the same theme are accurately classified, and cross speeches of different themes are prevented from being mistakenly classified as a whole.

Description

Technical Field

[0001] This invention relates to the field of meeting recording technology, and in particular to a meeting recording method based on the Internet of Things. Background Technology

[0002] In formal meeting settings such as corporate, academic, and government meetings, meeting minutes are extremely important. Traditional meeting minutes methods include handwritten notes, typing, or voice transcription. However, these methods are difficult to balance completeness and readability in scenarios with multiple people and high-density discussions. Manual recording relies on the recorder's understanding and speed, and is prone to missing or missing key information. While voice transcription can preserve complete content, it contains too much redundant information and the text is messy and difficult to use directly.

[0003] To improve the structuring of records, automated methods based on topic segmentation have been introduced in recent years. These methods use speech recognition to convert meeting content into text and divide it by topic to facilitate summary generation. However, these methods assume that meeting discussions proceed sequentially, which is not the case in real meetings.

[0004] In scenarios such as project reviews, participants often engage in multiple rounds of discussions on the same topic, and these discussions are scattered across different stages of the meeting. Traditional time-series methods tend to fragment these discussions into multiple independent topics, leading to reduced information coherence and affecting logical consistency. Furthermore, cross-discussion situations are even more complex, with multiple participants potentially speaking on different topics at the same time. Traditional methods, relying on a single audio stream, struggle to accurately identify the true topic of each speaker, frequently resulting in misclassification and impacting the accuracy of meeting minutes. Therefore, there is an urgent need for a meeting recording method based on the Internet of Things (IoT) to address these issues. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] This invention provides an IoT-based meeting recording method to address the problems of traditional methods, such as difficulty in handling cross-conversations, fragmented topics, serious misclassification, missing or redundant information, which affect the quality of meeting records.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] This invention provides a meeting recording method based on the Internet of Things, comprising:

[0009] Step S1: Collect conference audio data;

[0010] Step S2: Based on the speech data, speaker identification is performed, and the speech flow of different speakers is distinguished by turn detection technology. A speaking timeline is generated to separate overlapping speeches.

[0011] Step S3: Based on the speaking timeline generated in step S2, perform speech transcription to generate transcribed text;

[0012] Step S4: Perform topic segmentation on the transcribed text generated in step S3, and dynamically adjust it in combination with the speaker location data and individual voice features obtained in step S1.

[0013] Step S5: Optimize and merge the topic segmentation results generated in step S4, and integrate discontinuous discussions on the same topic to form a topic structure;

[0014] Step S6: Based on the topic structure, extract the summary, identify key conclusions by combining decision point detection, and perform structured processing on task allocation content to extract and generate task allocation information, forming a to-do list, and finally forming meeting minutes.

[0015] In a preferred embodiment of the IoT-based meeting recording method described in this invention, during step S1, the voice data acquisition process includes:

[0016] The speaker's location is detected using a microphone array and orientation sensor, and individual voice characteristics are obtained by combining wearable microphones for identity binding.

[0017] In a preferred embodiment of the IoT-based meeting recording method described in this invention, the orientation sensor includes an ultrasonic sensor and an infrared camera.

[0018] As a preferred embodiment of the IoT-based meeting recording method described in this invention, the step of detecting the speaker's position using a microphone array and a direction sensor, and acquiring individual voice features using a wearable microphone for identity binding, is as follows:

[0019] By setting up a microphone array with multiple fixed positions, and utilizing signal time difference and beamforming techniques, the direction of speech sources in the environment is estimated.

[0020] By combining ultrasonic and infrared camera data to obtain the speaker's angle and position coordinates, their orientation relative to the microphone array can be calculated.

[0021] By using data fusion technology, the acoustic information of the voice source is matched with the positional information from the orientation sensor to determine the speaker's location.

[0022] High signal-to-noise ratio speech data is collected using wearable microphones to extract individual features, including the speaker's timbre, speech texture, speech rate, and pronunciation habits.

[0023] Calculate the similarity of voice features between environmental microphone data and wearable microphone data, bind individual identities, and record identity information.

[0024] As a preferred embodiment of the IoT-based meeting recording method of the present invention, the steps of identifying speakers based on voice data, distinguishing the voice streams of different speakers by combining turn-taking detection technology, generating a speaking timeline, and separating overlapping speeches are as follows:

[0025] The original speech signal x(t) is denoised and its features are enhanced. The denoising formula is as follows:

[0026] x′(t)=x(t)-n(t),

[0027] Where x'(t) represents the denoised speech signal, x(t) represents the original speech signal, and n(t) represents the estimated background noise.

[0028] Feature extraction is performed using Mel-frequency cepstral coefficients (MFCCs), and the extraction formula is as follows:

[0029] f i =MFCC(x′) i ),

[0030] Among them, f i Let x' represent the feature vector of the i-th frame. i This represents the denoised i-th frame of speech data, where MFCC(·) represents the Mel-frequency cepstral coefficient calculation function.

[0031] Clustering and classification are performed using a Gaussian Mixture Model (GMM) or a deep learning model. The classification formula is as follows:

[0032]

[0033] Where P(s|f) represents the probability that the speech feature belongs to a certain speaker, M represents the number of Gaussian distributions, and w j Let j be the weights of the j-th Gaussian distribution. The mean is μ j The covariance is Σ j Gaussian distribution,

[0034] Speech activity is calculated using short-time energy E, and the formula is as follows:

[0035]

[0036] Among them, E t Let x be the short-time energy at time t, N be the window size, and x be the short-time energy at time t. i The speech sample values ​​within the time window,

[0037] By threshold θ E Perform turn-based segmentation:

[0038] If E t >θE Then VAD(t) = 1.

[0039] If E t ≤θ E Then VAD(t) = 0.

[0040] Where VAD(t) is a binary function for speech activity detection, where 1 indicates the presence of speech and 0 indicates silence.

[0041] Multiple speech segments are separated using the Blind Source Separation (BSS) method, and the separation method is represented as follows:

[0042] Y = WX,

[0043] Where X is the observation signal matrix, W is the separation matrix, and Y is the separated independent speech signal.

[0044] In a preferred embodiment of the IoT-based meeting recording method of the present invention, in the text transcription process of step S3, noise reduction processing is used to remove background noise;

[0045] It also employs semantic compensation combined with IoT devices to collect projection and whiteboard writing data for text correction.

[0046] In a preferred embodiment of the IoT-based meeting recording method of the present invention, the dynamic adjustment in step S4 includes:

[0047] If individual speech characteristics change and location data changes are detected, topic boundaries are automatically inserted;

[0048] If the projection switch in step S3 is detected, the theme conversion is automatically inferred.

[0049] As a preferred embodiment of the IoT-based meeting recording method of the present invention, the step of performing topic segmentation on the transcribed text generated in step S3 and dynamically adjusting it in combination with the speaker location data and individual voice features obtained in step S1 is as follows:

[0050] First, the transcribed text generated in step S3 is semantically vectorized, and then a word embedding method is used to convert the text into a high-dimensional vector v. i :

[0051] v i =φ(T i ),

[0052] Among them, v i Let T be the semantic vector of the i-th sentence. i Let be the text content of the i-th sentence, and φ(·) be the word embedding transformation function.

[0053] The semantic similarity between adjacent sentences is calculated using cosine similarity, and the formula is as follows:

[0054]

[0055] Among them, S i,j Let v be the semantic similarity between sentence i and sentence j. i and v j Let represent the semantic vectors of the two sentences respectively, and |·| be the Euclidean norm of the vector.

[0056] Topic boundaries are detected using clustering algorithms, and a topic boundary function is defined:

[0057] If S k,k+1 <θ S Then B k =1,

[0058] If S k,k+1 ≥θ S Then B k =0,

[0059] Among them, B k Mark the subject boundary, θ S This is a similarity threshold; if the similarity between two sentences is below the threshold, they are marked as a new topic boundary.

[0060] If a change in individual speech characteristics is detected, and the speaker's location data changes, topic boundaries are automatically inserted:

[0061] If (Δv) k >θ v )∧(Δp k >θ p ), then B' k =1, otherwise B' k =B k ,

[0062] Among them, B' k For the adjusted subject boundary, Δv k θ represents the individual speech feature variation value. v Δp is the threshold for changes in speech features. k θ represents the change in location data. p The threshold for position change.

[0063] If a change in the content of the projection device is detected, the topic boundary will be automatically inserted:

[0064] If C k -C k-1 ≠0, then B″ k =1, otherwise B″ k =B' k

[0065] Among them, B″ k For the final adjusted topic boundaries, C k This is the content identifier for the projection device at time k.

[0066] As a preferred embodiment of the IoT-based meeting recording method of the present invention, the step of optimizing and merging the topic segmentation results generated in step S4, and integrating discontinuous discussions on the same topic to form a topic structure, is as follows:

[0067] For the topic fragment generated in step S4 and Calculate the semantic similarity between segments:

[0068]

[0069] in, Theme and The similarity between them, v i ,v j For belonging to the theme and Sentence semantic vectors, Each represents a theme and The number of sentences within the sentence, where |·| is the Euclidean norm.

[0070] If the similarity between two topics is higher than the threshold θ M Then merge the two topics:

[0071] like but

[0072] like but

[0073] in, To merge tags for topics, θ M The similarity threshold for topic merging.

[0074] If the topic and If there is a long interval between unrelated contents, calculate the interval using the following formula:

[0075]

[0076] in, Indicates the topic and The shortest interval between, I m,n A set of sentence indexes between two topics.

[0077] like This allows topic merging:

[0078] like but otherwise

[0079] Where, θ G The maximum interval threshold,

[0080] After merging, the topics are renumbered and a tree structure is built:

[0081] H k =Tree(M),

[0082] Among them, H k Tree(·) is a function that performs hierarchical topic classification based on the merge matrix M to represent the reconstructed topic hierarchy.

[0083] As a preferred embodiment of the IoT-based meeting recording method described in this invention, the step of combining decision point detection to identify key conclusions, and performing structured processing on task allocation content to extract and generate task allocation information is as follows:

[0084] Based on the topic hierarchy generated in step S5, calculate the representative sentences for each topic.

[0085] The core topic sentences are extracted using TF-IDF or BERT encoding with a bidirectional transform, and the sentence with the highest importance score is selected as the summary sentence.

[0086] For each topic, select the preceding sentence with the highest importance score as the topic summary;

[0087] Decision-making sentences, such as those containing keywords like "decision," "solution," "goal," and "plan," are identified through keyword matching and dependency relationship analysis.

[0088] By combining syntactic analysis, sentences containing active decision-making verbs are selected and matched with the speaker's identity;

[0089] Identify sentences containing task assignment information, focusing on sentences containing task-related terms such as "responsible for," "execute," and "completion time."

[0090] By combining the information of the conference speakers, the task executors are analyzed and matched to the specific speaker identities.

[0091] Extract key task attributes, including: task executor, task content, task deadline, task priority, and task-related resources;

[0092] Tasks are formatted and stored according to their attributes, generating a structured list of to-do items: task ID, task description, responsible person, deadline, and task status;

[0093] Summarize the topic summaries, decision points, and task assignment information, and categorize them chronologically and by topic to form the final meeting minutes document.

[0094] Based on the speakers' identity information and timeline data, the speakers' viewpoints are compiled and presented in a structured manner.

[0095] The beneficial effects of this invention are as follows: This invention uses a microphone array and a direction sensor to locate the speaker, and combines a wearable microphone to obtain individual voice features. The voice is matched with the speaker's identity. During the speaker recognition process, turn detection technology is used to distinguish the voice streams of different speakers and generate a speaking timeline. For speech transcription, noise reduction technology is used to remove background noise, and semantic compensation is introduced. The text is corrected by combining projection and whiteboard writing data collected by IoT devices to improve recognition accuracy.

[0096] This invention performs additional topic segmentation, using semantic vectorization to convert text into high-dimensional vectors, and utilizes similarity calculation and clustering algorithms to determine topic boundaries. It also dynamically adjusts the data by combining IoT location data and individual voice features to accurately categorize discontinuous discussions on the same topic and prevent misclassification of overlapping discussions from different topics. Furthermore, it calculates semantic similarity between topics to integrate discontinuous discussions on the same topic, and uses a maximum interval threshold to ensure reasonable topic merging. Finally, it constructs a tree structure to enhance the hierarchical organization of topics.

[0097] This invention combines TF-IDF or BERT models to extract meeting summaries, uses keyword matching and grammatical analysis to detect decision-making sentences, preserves the core content of the meeting, and effectively parses task assignment content, extracting key elements such as task executors, content, and deadlines. Finally, it presents the meeting minutes in a structured to-do list format, making the meeting records more readable and traceable.

[0098] In summary, this invention not only optimizes the accuracy of traditional meeting minutes in multi-speaker discussions and improves the relevance of topics across timelines, but also effectively reduces misclassification and information loss, thereby significantly improving the quality, completeness, and usability of meeting minutes. Attached Figure Description

[0099] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0100] Figure 1 This is a flowchart illustrating the IoT-based meeting recording method of the present invention. Detailed Implementation

[0101] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0102] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0103] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0104] Example 1, referring to Figure 1 This embodiment provides a meeting recording method based on the Internet of Things, including the following steps:

[0105] Step S1: Collect conference audio data;

[0106] In step S1, during the voice data acquisition process:

[0107] The speaker's position is detected by using a microphone array and orientation sensor, and individual voice characteristics are obtained by combining wearable microphones for identity binding;

[0108] Orientation sensors include ultrasonic and infrared cameras;

[0109] The steps for identifying a speaker by using a microphone array and orientation sensors, and combining this with wearable microphones to acquire individual voice characteristics, are as follows:

[0110] By setting up a microphone array with multiple fixed positions, and utilizing signal time difference and beamforming techniques, the direction of speech sources in the environment is estimated.

[0111] By combining ultrasonic and infrared camera data to obtain the speaker's angle and position coordinates, their orientation relative to the microphone array can be calculated.

[0112] By using data fusion technology, the acoustic information of the voice source is matched with the positional information from the orientation sensor to determine the speaker's location.

[0113] High signal-to-noise ratio speech data is collected using wearable microphones to extract individual features, including the speaker's timbre, speech texture, speech rate, and pronunciation habits.

[0114] Calculate the similarity of voice features between environmental microphone data and wearable microphone data, bind individual identities, and record identity information;

[0115] Step S2: Based on the speech data, speaker identification is performed, and the speech flow of different speakers is distinguished by turn detection technology. A speaking timeline is generated to separate overlapping speeches.

[0116] The process involves speaker identification based on voice data, distinguishing the speech streams of different speakers using turn-taking detection technology, generating a speaking timeline, and separating overlapping speeches.

[0117] The original speech signal x(t) is denoised and its features are enhanced. The denoising formula is as follows:

[0118] x′(t)=x(t)-n(t),

[0119] Where x'(t) represents the denoised speech signal, x(t) represents the original speech signal, and n(t) represents the estimated background noise.

[0120] Feature extraction is performed using Mel-frequency cepstral coefficients (MFCCs), and the extraction formula is as follows:

[0121] f i =MFCC(x′) i ),

[0122] Among them, f i Let x' represent the feature vector of the i-th frame. i This represents the denoised i-th frame of speech data, where MFCC(·) represents the Mel-frequency cepstral coefficient calculation function.

[0123] Clustering and classification are performed using a Gaussian Mixture Model (GMM) or a deep learning model. The classification formula is as follows:

[0124]

[0125] Where P(s|f) represents the probability that the speech feature belongs to a certain speaker, M represents the number of Gaussian distributions, and w j Let j be the weights of the j-th Gaussian distribution. The mean is μ j The covariance is Σ j Gaussian distribution,

[0126] Speech activity is calculated using short-time energy E, and the formula is as follows:

[0127]

[0128] Among them, E t Let x be the short-time energy at time t, N be the window size, and x be the short-time energy at time t. i The speech sample values ​​within the time window,

[0129] By threshold θ E Perform turn-based segmentation:

[0130] If E t >θ E Then VAD(t) = 1.

[0131] If E t ≤θ E Then VAD(t) = 0.

[0132] Where VAD(t) is a binary function for speech activity detection, where 1 indicates the presence of speech and 0 indicates silence.

[0133] Multiple speech segments are separated using the Blind Source Separation (BSS) method, and the separation method is represented as follows:

[0134] Y = WX,

[0135] Where X is the observed signal matrix, W is the separation matrix, and Y is the separated independent speech signal.

[0136] Specifically, this method uses speech signal denoising, feature extraction, speaker clustering and recognition, turn detection and blind source separation techniques to distinguish the speech streams of different speakers and generate a speaking timeline.

[0137] Background noise estimation is used to denoise the signal and improve speech clarity. Then, MFCC is used to extract speaker features and GMM or deep learning model is used to classify and identify speakers. Turn detection is performed by short time energy calculation to detect speech activity and identify the time interval of each speaker's speech. Finally, blind source separation technology is used to separate cross-speech and accurately construct the speech timeline.

[0138] Step S3: Based on the speaking timeline generated in step S2, perform speech transcription to generate transcribed text;

[0139] In the text transcription process of step S3, noise reduction processing is used to remove background noise;

[0140] Semantic compensation is used in conjunction with data collected from projection and whiteboard writing by IoT devices to correct the text.

[0141] Step S4: Perform topic segmentation on the transcribed text generated in step S3, and dynamically adjust it in combination with the speaker location data and individual voice features obtained in step S1.

[0142] The dynamic adjustment in step S4 includes,

[0143] If individual speech characteristics change and location data changes are detected, topic boundaries are automatically inserted;

[0144] If the projection switch in step S3 is detected, the theme conversion is automatically inferred;

[0145] The steps for performing topic segmentation on the transcribed text generated in step S3, and dynamically adjusting it based on the speaker location data and individual speech features obtained in step S1, are as follows:

[0146] First, the transcribed text generated in step S3 is semantically vectorized, and then a word embedding method is used to convert the text into a high-dimensional vector v. i :

[0147] v i =φ(T i ),

[0148] Among them, v i Let T be the semantic vector of the i-th sentence. i Let be the text content of the i-th sentence, and φ(·) be the word embedding transformation function.

[0149] The semantic similarity between adjacent sentences is calculated using cosine similarity, and the formula is as follows:

[0150]

[0151] Among them, S i,j Let v be the semantic similarity between sentence i and sentence j. i and v j Let represent the semantic vectors of the two sentences respectively, and |·| be the Euclidean norm of the vector.

[0152] Topic boundaries are detected using clustering algorithms, and a topic boundary function is defined:

[0153] If S k,k+1 <θ S Then B k =1,

[0154] If S k,k+1 ≥θ S Then B k =0,

[0155] Among them, B k Mark the subject boundary, θ S This is a similarity threshold; if the similarity between two sentences is below the threshold, they are marked as a new topic boundary.

[0156] If a change in individual speech characteristics is detected, and the speaker's location data changes, topic boundaries are automatically inserted:

[0157] If (Δv) k >θ v )∧(Δp k >θ p Then B' k =1, otherwise B' k =B k ,

[0158] Among them, B' k For the adjusted subject boundary, Δv k θ represents the individual speech feature variation value. v Δp is the threshold for changes in speech features. k θ represents the change in location data. p The threshold for position change.

[0159] If a change in the content of the projection device is detected, the topic boundary will be automatically inserted:

[0160] If C k -C k-1 ≠0, then B″ k =1, otherwise B″ k =B' k

[0161] Among them, B″ k For the final adjusted topic boundaries, C k The content identifier for the projection device at time k;

[0162] Specifically, this method processes the text using semantic vectorization, calculates the semantic similarity between adjacent sentences, uses a threshold to determine topic boundaries, and dynamically adjusts the topic segmentation based on speaker location and individual voice feature changes to improve the accuracy of topic segmentation. In addition, it combines IoT devices to detect the content of the projection switching to further optimize topic segmentation.

[0163] Step S5: Optimize and merge the topic segmentation results generated in step S4, and integrate discontinuous discussions on the same topic to form a topic structure;

[0164] The steps to optimize and merge the topic segmentation results generated in step S4, integrating discontinuous discussions of the same topic to form a topic structure, are as follows:

[0165] For the topic fragment generated in step S4 and Calculate the semantic similarity between segments:

[0166]

[0167] in, Theme and The similarity between them, v i ,v j For belonging to the theme and Sentence semantic vectors, Each represents a theme and The number of sentences within the sentence, where |·| is the Euclidean norm.

[0168] If the similarity between two topics is higher than the threshold θ M Then merge the two topics:

[0169] like but

[0170] like but

[0171] in, To merge tags for topics, θ M The similarity threshold for topic merging.

[0172] If the topic and If there is a long interval between unrelated contents, calculate the interval using the following formula:

[0173]

[0174] in, Indicates the topic and The shortest interval between, I m,n A set of sentence indexes between two topics.

[0175] like This allows topic merging:

[0176] like but otherwise

[0177] Where, θ G The maximum interval threshold,

[0178] After merging, the topics are renumbered and a tree structure is built:

[0179] H k =Tree(M),

[0180] Among them, H k Tree(·) is a function that performs hierarchical topic classification based on the merge matrix M, representing the reconstructed topic hierarchy.

[0181] Specifically, in step S5, similar topics are determined by calculating the semantic similarity between topics, and topic merging is performed based on the semantic similarity threshold. At the same time, in order to ensure that topics of non-continuous discussion can be correctly merged, the shortest interval between topics is calculated and the merging determination is made in combination with the maximum interval threshold. Finally, the merged topic hierarchy is constructed through a tree structure, making the topic organization more structured.

[0182] Step S6: Based on the topic structure, extract the summary, identify key conclusions by combining decision point detection, and perform structured processing on task allocation content to extract and generate task allocation information, form a to-do list, and finally form meeting minutes.

[0183] The steps for combining decision point detection and identification of key conclusions, and for structuring task allocation content to extract and generate task allocation information are as follows:

[0184] Based on the topic hierarchy H generated in step S5 k Calculate the representative sentences for each topic.

[0185] The core topic sentences are extracted using TF-IDF or BERT encoding with a bidirectional transform, and the sentence with the highest importance score is selected as the summary sentence.

[0186] For each topic Select the top N sentences with the highest importance scores as the topic summary;

[0187] Decision-making sentences, such as those containing keywords like "decision," "solution," "goal," and "plan," are identified through keyword matching and dependency relationship analysis.

[0188] By combining syntactic analysis, sentences containing active decision-making verbs are selected and matched with the speaker's identity;

[0189] Identify sentences containing task assignment information, focusing on sentences containing task-related terms such as "responsible for," "execute," and "completion time."

[0190] By combining the information of the conference speakers, the task executors are analyzed and matched to the specific speaker identities.

[0191] Extract key task attributes, including: task executor, task content, task deadline, task priority, and task-related resources;

[0192] Tasks are formatted and stored according to their attributes, generating a structured list of to-do items: task ID, task description, responsible person, deadline, and task status;

[0193] Summarize the topic summaries, decision points, and task assignment information, and categorize them chronologically and by topic to form the final meeting minutes document.

[0194] Based on the speakers' identity information and timeline data, the speakers' viewpoints are compiled and presented in a structured manner;

[0195] Specifically, in step S6, based on the topic structure generated in step S5, key content is further extracted to form a meeting summary and task allocation information. Core sentences are selected using TF-IDF or deep learning methods to achieve topic summary extraction. At the same time, key decision content in the meeting is extracted using decision point detection technology, and task allocation information is parsed using grammatical analysis. After the task data is structured, a to-do list is generated. All information is summarized to form a complete meeting record, making the archiving and tracking of meeting content more efficient.

[0196] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A meeting recording method based on the Internet of Things, characterized in that: include, Step S1: Collect conference audio data; Step S2: Based on the speech data, speaker identification is performed, and the speech flow of different speakers is distinguished by turn detection technology. A speaking timeline is generated to separate overlapping speeches. Step S3: Based on the speaking timeline generated in step S2, perform speech transcription to generate transcribed text; Step S4: Perform topic segmentation on the transcribed text generated in step S3, and dynamically adjust it in combination with the speaker location data and individual voice features obtained in step S1. Step S5: Optimize and merge the topic segmentation results generated in step S4, and integrate discontinuous discussions on the same topic to form a topic structure; Step S6: Based on the topic structure, extract the summary, identify key conclusions by combining decision point detection, and perform structured processing on task allocation content to extract and generate task allocation information, form a to-do list, and finally form meeting minutes. The dynamic adjustment includes, If individual speech characteristics change and location data changes are detected, topic boundaries are automatically inserted; If the projection switch in step S3 is detected, the theme conversion is automatically inferred; Step S4 is as follows: First, the transcribed text generated in step S3 is semantically vectorized, and then word embedding is used to convert the text into a high-dimensional vector. : ， in, For the first The semantic vector of a sentence For the first The text content of the sentence, For word embedding transformation function, The semantic similarity between adjacent sentences is calculated using cosine similarity, and the formula is as follows: ， in, For the first Sentence and the first Semantic similarity of sentences and These represent the semantic vectors of the two sentences respectively. Let be the Euclidean norm of the vector. Topic boundaries are detected using clustering algorithms, and a topic boundary function is defined: like ,but , like ,but , in, Mark the subject boundary. This is a similarity threshold; if the similarity between two sentences is below the threshold, they are marked as a new topic boundary. If a change in individual speech characteristics is detected, and the speaker's location data changes, topic boundaries are automatically inserted: like ,but ,otherwise , in, For the adjusted topic boundaries, For individual speech feature changes, Threshold for changes in speech features This represents the change in location data. The position change threshold, If a change in the content of the projection device is detected, the topic boundary will be automatically inserted: like ,but ,otherwise , in, For the final adjusted theme boundaries, For time Content labeling for projection equipment.

2. The meeting recording method based on the Internet of Things as described in claim 1, characterized in that: In step S1, during the voice data acquisition process: The speaker's location is detected using a microphone array and orientation sensor, and individual voice characteristics are obtained by combining wearable microphones for identity binding.

3. The meeting recording method based on the Internet of Things as described in claim 2, characterized in that: The orientation sensor includes ultrasonic and infrared cameras.

4. The meeting recording method based on the Internet of Things as described in claim 3, characterized in that: The steps of using a microphone array and orientation sensor to detect the speaker's position, and combining this with wearable microphones to acquire individual voice features for identity binding are as follows: By setting up a microphone array with multiple fixed positions, and utilizing signal time difference and beamforming techniques, the direction of speech sources in the environment is estimated. By combining ultrasonic and infrared camera data to obtain the speaker's angle and position coordinates, their orientation relative to the microphone array can be calculated. By using data fusion technology, the acoustic information of the voice source is matched with the positional information from the orientation sensor to determine the speaker's location. High signal-to-noise ratio speech data is collected using wearable microphones to extract individual features, including the speaker's timbre, speech texture, speech rate, and pronunciation habits. Calculate the similarity of voice features between environmental microphone data and wearable microphone data, bind individual identities, and record identity information.

5. A meeting recording method based on the Internet of Things as described in claim 4, characterized in that: The steps of speaker identification based on speech data, distinguishing the speech streams of different speakers by combining turn-taking detection technology, generating a speaking timeline, and separating overlapping speeches are as follows: For the original speech signal Denoising and feature enhancement are performed. The denoising formula is: ， in, This represents the denoised speech signal. Represents the original speech signal. This represents the estimated background noise. Feature extraction is performed using Mel-frequency cepstral coefficients (MFCCs), and the extraction formula is as follows: ， in, Indicates the first The feature vector of the frame, Indicates the denoised first... Frame audio data, This represents the function for calculating Mel frequency cepstral coefficients. Clustering and classification are performed using a Gaussian Mixture Model (GMM) or a deep learning model. The classification formula is as follows: ， in, This represents the probability that a speech feature belongs to a particular speaker. This represents the number of Gaussian distributions. For the first The weights of a Gaussian distribution The mean is covariance is Gaussian distribution, Utilizing short-time energy The formula for calculating speech activity is as follows: ， in, For time Short-term energy at any given moment For window size, The speech sample values ​​within the time window, By threshold Perform turn-by-turn segmentation: like ,but , like ,but , in, This is a binary function for speech activity detection, where 1 indicates speech is present and 0 indicates silence. Multiple speech segments are separated using the Blind Source Separation (BSS) method, and the separation method is represented as follows: ， in, For the observed signal matrix, For the separation matrix, These are the separated, independent speech signals.

6. The meeting recording method based on the Internet of Things as described in claim 5, characterized in that: In the text transcription process of step S3, noise reduction processing is used to remove background noise; It also employs semantic compensation combined with IoT devices to collect projection and whiteboard writing data for text correction.

7. A meeting recording method based on the Internet of Things as described in claim 6, characterized in that: The step of optimizing and merging the topic segmentation results generated in step S4, integrating discontinuous discussions of the same topic, and forming a topic structure is as follows: For the topic fragment generated in step S4 and Calculate the semantic similarity between segments: ， in, Theme and Similarity between them For belonging to the theme and Sentence semantic vectors, Each represents a theme and The number of sentences within the sentence For the Euclidean norm, If the similarity between two topics is higher than a threshold Then merge the two topics: like ,but , like ,but , in, Merge tags for the topic, The similarity threshold for topic merging. If the topic and If there is a long interval between unrelated contents, calculate the interval using the following formula: ， in, Indicate topic and The shortest interval between them A set of sentence indexes between two topics. like This allows topic merging: like ,but ,otherwise , in, The maximum interval threshold, After merging, the topics are renumbered and a tree structure is built: ， in, For the restructured topic hierarchy, For merging matrix A function for hierarchical topic categorization.

8. A meeting recording method based on the Internet of Things as described in claim 7, characterized in that: The steps of combining decision point detection and identification of key conclusions, structuring task allocation content, and extracting and generating task allocation information are as follows: Based on the topic hierarchy generated in step S5 Calculate the representative sentences for each topic. The core topic sentences are extracted using TF-IDF or BERT encoding with a bidirectional transform, and the sentence with the highest importance score is selected as the summary sentence. For each topic Select the top ones with the highest importance scores One sentence as a topic summary; Decision-making sentences are identified through keyword matching and dependency relationship analysis. By combining syntactic analysis, sentences containing active decision-making verbs are selected and matched with the speaker's identity; Identify sentences containing task assignment information. By combining the information of the conference speakers, the task executors are analyzed and matched to the specific speaker identities. Extract key task attributes, including: task executor, task content, task deadline, task priority, and task-related resources; Tasks are formatted and stored according to their attributes, generating a structured list of to-do items: task ID, task description, responsible person, deadline, and task status; Summarize the topic summaries, decision points, and task assignment information, and categorize them chronologically and by topic to form the final meeting minutes document. Based on the speakers' identity information and timeline data, the speakers' viewpoints are compiled and presented in a structured manner.

Images