Artificial intelligence assisted cross-language automatic note taking and term marking system

Abstract

The invention discloses an artificial intelligence-assisted cross-language automatic note taking and term tagging system, and belongs to the technical field of language tagging. The system firstly obtains and processes audio data, generates a preliminary text through cross-language speech recognition, recognizes term candidate words in the text, and performs pronunciation association matching with a term knowledge base; and performing language recognition and context analysis on the candidate words and the contexts thereof, performing comprehensive pronunciation matching, language recognition and context score calculation to obtain a comprehensive score, thereby accurately selecting a cross-language term which most conforms to the current context as the final candidate word, and obtaining the annotation information of the final candidate word. And integrating the final candidate word and the annotation information of the final candidate word into the calibration note text, and outputting an automatic note with cross-language term annotations. According to the method, complex audios including language mixing and the like can be automatically processed, the high-quality calibration note text is generated, and the information recording efficiency and accuracy in a cross-language communication scene are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of language annotation, and particularly to an artificial intelligence-assisted cross-language automatic note and term annotation system. Background Art

[0002] In the field of automated note generation, generating text from audio through speech recognition technology has received extensive attention and application. However, in the face of complex audio environments, such as the presence of background noise, multiple speakers, especially when dealing with speech inputs containing different languages or even mixed languages, accurately performing speech-to-text conversion itself is challenging. On this basis, automatically identifying and annotating specific information in the transcribed text, such as technical terms, proper nouns, or cross-language concepts, is even more difficult in the prior art. Current automated term annotation methods often rely on simple text matching, statistical features, or rules based on limited context, and it is difficult to effectively handle the pronunciation variations of terms, the meaning differences in different contexts, and the complexity in cross-language scenarios. Especially when terms appear in non-target languages or mixed languages in the original speech, existing methods are prone to misrecognition, missed annotation, or providing inaccurate annotation information, seriously affecting the accuracy and practical value of automatic notes. Summary of the Invention

[0003] To address the above technical problems, an artificial intelligence-assisted cross-language automatic note-taking and term annotation system of the present invention includes an audio processing and cross-language speech recognition module, which is used to receive audio data, perform preprocessing, and perform cross-language speech recognition on the preprocessed audio data to output preliminary text; a text proofreading and term candidate recognition module, which is used to receive the preliminary text, perform reciprocating matching and proofreading to output calibrated note text, and recognize term candidate words from the preliminary text or the calibrated note text; a cross-language term matching analysis and annotation decision-making module, which is used to receive the term candidate words and the calibrated note text, obtain the pronunciation representation of the term candidate words, compare them with the cross-language pronunciation representations of known terms in the term knowledge base and pronunciation model base to generate a preliminary cross-language term matching list, perform language recognition on the term candidate words or their contexts, perform context analysis on each known term in the preliminary cross-language term matching list and calculate the context matching score, calculate the comprehensive score by integrating the pronunciation association matching result, language recognition result and context matching score, select the final candidate word that best matches the current context, and obtain the annotation information of the final candidate word from the term knowledge base and pronunciation model base; an automatic note generation and annotation output module, which is used to receive the calibrated note text and the final candidate word and its annotation information, integrate the final candidate word and its annotation information into the calibrated note text, and generate an automatic note with cross-language term annotation. The term knowledge base and pronunciation model base are used to store terms, meanings in different languages and corresponding cross-language pronunciation representations, and provide query support for the cross-language term matching analysis and annotation decision-making module.

[0004] Further, the automatic note-taking and term annotation method includes the following steps:

[0005] Step S1: Obtain and process audio data, and perform cross-language speech recognition to obtain preliminary text.

[0006] Step S2: Perform reciprocating matching and proofreading on the preliminary text to generate calibrated note text, and recognize term candidate words.

[0007] Step S3: When term candidate words are recognized, perform cross-language feature judgment, pronunciation association matching and context analysis, and comprehensively score to determine the cross-language terms to be annotated and their information.

[0008] Step S4: Integrate the final candidate word and its annotation information into the note text, and output the note text with annotations.

[0009] Further, the step S1 includes,

[0010] Step S101: Obtain a multi-channel or single-channel audio data stream to be processed through an audio input device. The audio data stream may contain the voices of different speakers, mixed languages, and environmental noise.

[0011] Step S102: Perform enhanced preprocessing and acoustic feature extraction on the audio data. The preprocessing at least includes multi-channel signal processing, voice activity detection, noise reduction, acoustic environment adaptation, gain adjustment, and frame segmentation. The acoustic feature extraction is used to obtain a sequence of feature vectors suitable for speech recognition.

[0012] Step S103: Use a cross-language speech recognition model based on a deep learning model to perform speech-to-text operations on the sequence of acoustic feature vectors. The cross-language speech recognition model can process speech inputs containing mixed languages and output the confidence information of the speech recognition result and / or the language recognition confidence information of the text segment during the recognition process; output a preliminary text stream or text segment.

[0013] Further, step S2 includes:

[0014] Step S201: Perform chunking on the preliminary text output in step S1 according to a preset time length or semantic boundary to form text segments to be proofread and term-recognized.

[0015] Step S202: Adopt a context-dependent reciprocating matching mode, and use the speech recognition result corresponding to the subsequent audio received and / or the subsequent text context information to perform operations such as re-scoring, re-searching, or post-processing based on a language model on the currently and previously transcribed text segments, and dynamically correct the text content in combination with the confidence information of the speech recognition result to generate a more accurate calibrated note text.

[0016] Step S203: Scan the preliminary text or the calibrated note text to identify words or multi-word phrases that meet the preset features as the term candidate words.

[0017] Further, step S3 includes:

[0018] Step S301: Extract the current text context where the term candidate words are located.

[0019] Step S302: Obtain the pronunciation representation of the term candidate words, compare it with the pronunciation representations of known cross-language terms in the term knowledge base, perform pronunciation-related word center matching, and generate a preliminary cross-language term matching list.

[0020] Step S303: Based on the language recognition result of the term candidate or its context, and the center matching result of the pronunciation correlation word, preliminarily judge the cross - language feature of the term candidate, and analyze the compliance of each known term in the preliminary cross - language term matching list with the context, and calculate the context matching score.

[0021] Step S304: Synthesize the center matching result of the pronunciation correlation word, the language recognition result and the context matching score, calculate the comprehensive score of each known term in the preliminary cross - language term matching list, and select the known term with the highest score and higher than the preset threshold as the final candidate word according to the comprehensive score. At the same time, obtain the annotation information of the final candidate word from the term knowledge base.

[0022] Further, the step S4 includes:

[0023] Step S401: According to the characteristics or the comprehensive score of the final candidate word obtained in step S3, determine the annotation level or display priority of the final candidate word; and integrate the final candidate word and its annotation information into the position where the corresponding word appears in the calibrated note text in the preset integration format corresponding to the determined annotation level or display priority. The preset integration format at least includes inserting specific marks in the calibrated note text, changing the word style, adding hover tips, generating sidebar notes, creating expandable details.

[0024] Step S402: Output the finally generated automatic note text with cross - language term annotations. The automatic note text includes the calibrated note text and the structured annotation data associated with the final candidate word to support subsequent reference, editing or interactive display. And the output format of the automatic note text at least includes rich text format, HTML format, Markdown format.

[0025] Further, in step S304, when calculating the comprehensive score of each known term in the preliminary cross - language term matching list, the following formula is used:

[0026] , where, represents the term candidate, represents the current text context, represents the th known term in the preliminary cross - language term matching list, and the known term is from the term knowledge base; represents the pronunciation representation of the term candidate and the known term The pronunciation similarity score between cross - language pronunciation representations, where the pronunciation representations are obtained based on acoustic feature extraction of the speech of the candidate term, phoneme sequence prediction, or generating pronunciation vectors, and the cross - language pronunciation representations are the standard pronunciation representations of the known term in different languages, and the pronunciation similarity score is obtained by calculating the distance or similarity measure between the pronunciation representations, and this score is greater than 0; representing the known term and the current text context between the context matching scores, where the context matching scores are calculated by analyzing the semantic relevance between the known term and the context and the semantic relevance analysis can be implemented based on word vectors, language models, topic models, and text matching algorithms, and this score value is greater than 0; representing the system's judgment that the candidate term belongs to the main language of the known term of the language recognition confidence score, where the confidence score is obtained by performing language recognition analysis on the candidate term itself or its context where this score value is greater than 0 and less than or equal to 1; representing the known term with a preset main language label in the term knowledge base; representing a weight coefficient used to adjust the contribution ratio of each score item in the comprehensive score; representing a small constant greater than zero used to avoid calculation anomalies when taking the logarithm of the score items.

[0027] Furthermore, the weight coefficient is determined according to a preset strategy, and the strategy is used to balance the relative importance of pronunciation similarity, context matching, and language recognition in the comprehensive evaluation to optimize the recognition accuracy of cross - language terms.

[0028] The beneficial effects of the present invention compared with the prior art are as follows: (1) The present invention provides an artificial intelligence - assisted method that can automatically process complex audio containing language mixing and generate high - quality calibrated note text. (2) By integrating pronunciation, context, and language recognition information, the present invention can accurately identify and label cross - language terms, significantly improving the information recording efficiency and accuracy in cross - language communication scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is an exemplary step - by - step flowchart of the automatic note - taking and term annotation method of the present invention.

[0030] Figure 2This is an exemplary flowchart of the steps for processing audio data in the present invention.

[0031] Figure 3 This is an exemplary flowchart of the steps for performing reciprocating verification in the present invention.

[0032] Figure 4 This is an exemplary flowchart of the steps for determining candidate terms in the present invention.

[0033] Figure 5 This is an exemplary flowchart of the steps for integrating text results in the present invention. Detailed implementation manners

[0034] This application provides an artificial intelligence-assisted cross-language automatic note-taking and term annotation system, which aims to automatically generate notes with accurate cross-language term annotations by processing audio data from different sources, significantly improving the information recording efficiency and quality in scenarios such as cross-language communication, learning, and meeting minutes.

[0035] In one embodiment, the system may include an audio processing and cross-language speech recognition module, which is used to receive audio data, perform preprocessing, and perform cross-language speech recognition on the preprocessed audio data to output preliminary text; a text proofreading and term candidate recognition module, which is used to receive the preliminary text, perform reciprocating matching and proofreading to output calibrated note text, and identify candidate terms from the preliminary text or the calibrated note text; a cross-language term matching analysis and annotation decision module, which is used to receive the candidate terms and the calibrated note text, obtain the pronunciation representation of the candidate terms, compare it with the cross-language pronunciation representations of known terms in the term knowledge base and the pronunciation model base to generate a preliminary cross-language term matching list, perform language recognition on the candidate terms or their contexts, perform context analysis on each known term in the preliminary cross-language term matching list and calculate the context matching score, calculate the comprehensive score by integrating the pronunciation association matching result, the language recognition result, and the context matching score, select the final candidate term that best matches the current context, and obtain the annotation information of the final candidate term from the term knowledge base and the pronunciation model base; an automatic note generation and annotation output module, which is used to receive the calibrated note text and the final candidate term and its annotation information, integrate the final candidate term and its annotation information into the calibrated note text, and generate automatic notes with cross-language term annotations. The term knowledge base and the pronunciation model base are used to store terms, meanings, and corresponding cross-language pronunciation representations in different languages, and provide query support for the cross-language term matching analysis and annotation decision module.

[0036] As Figure 1 shown, this is an exemplary flowchart of the steps of the automatic note-taking and term annotation method in this embodiment, including the following steps:

[0037] Step S1: Obtain and process audio data, and perform cross - language speech recognition to obtain preliminary text.

[0038] Step S2: Perform reciprocating matching and proofreading on the preliminary text to generate a calibrated note text, and identify term candidate words.

[0039] Step S3: When term candidate words are identified, perform cross - language feature judgment, pronunciation association matching, and context analysis, and comprehensively score to determine the cross - language terms to be annotated and their information.

[0040] Step S4: Integrate the final candidate words and their annotation information into the note text, and output the note text with annotations.

[0041] As Figure 2 shown is an exemplary step - by - step flowchart for processing audio data in step S1 of this embodiment, including

[0042] Step S101: Through an audio input device, obtain a multi - channel or single - channel audio data stream to be processed. The audio data stream may contain the voices of different speakers, mixed languages, and environmental noise.

[0043] In one embodiment, the technical solution of step S101 is elaborated: The system receives the original audio data through various types of audio input devices. Exemplarily, these devices may include built - in microphones, external microphones, microphone arrays, voice recorders, conference systems, or audio streams received through the network. The obtained audio data stream can be single - channel or multi - channel, and may contain the voices of different speakers, a mixture of voices in multiple languages, and environmental background noise, such as the hum in a meeting room, keyboard typing sounds, or other interference sounds.

[0044] Step S102: Perform enhanced pre - processing and acoustic feature extraction on the audio data. The pre - processing at least includes multi - channel signal processing, voice activity detection, noise reduction, acoustic environment adaptation, gain adjustment, and frame segmentation. The acoustic feature extraction is used to obtain a sequence of feature vectors suitable for speech recognition.

[0045] In one embodiment, the technical solution of step S102 is elaborated: The received original audio data is first subjected to enhanced pre - processing. The pre - processing can at least include: If the audio is multi - channel, perform multi - channel signal processing. Exemplarily, such as beamforming to enhance the speech in a specific direction; perform voice activity detection to identify the segments with speech in the audio; apply a noise reduction algorithm to reduce background noise; perform acoustic environment adaptation adjustment to adapt to different recording environments; perform gain adjustment to standardize the loudness of the audio signal. The pre - processed audio is segmented into short - time frames, and acoustic features suitable for speech recognition are extracted from each frame.

[0046] Step S103: Use a cross - language speech recognition model based on a deep - learning model to perform speech - to - text operation on the acoustic feature vector sequence. The cross - language speech recognition model can process speech inputs containing mixed languages and output confidence information of the speech recognition result and / or language recognition confidence information of text segments during the recognition process; output a preliminary text stream or text segments.

[0047] In one embodiment, the technical solution of step S103 is elaborated: The extracted acoustic feature vector sequence is input into a cross - language speech recognition model based on a deep - learning architecture. This model is trained to process speech inputs containing different languages or even mixed languages. Exemplarily, the model can adopt connectionist temporal classification, attention mechanism, sequence - to - sequence model, or a Transformer - based architecture. When performing speech - to - text operation, the model outputs a preliminary text stream or outputs text results in segments. Additionally, to improve the accuracy of subsequent processing, the model can output the recognition confidence score of each word or text segment during the recognition process, and perform language recognition on each text segment and output its corresponding language confidence information. For example, determine whether the segment is Chinese, English, or other languages, and the probability of recognition.

[0048] As Figure 3 shown, an exemplary step - flow diagram for reciprocating proofreading in step S2 of this embodiment includes:

[0049] Step S201: Chunk the preliminary text output in step S1 according to a preset time length or semantic boundary to form text segments to be proofread and term - recognized.

[0050] In one embodiment, the preliminary text stream output from step S1 is segmented into more easily processed text segments. Exemplarily, the basis for chunking can be a fixed time length, detected end - of - sentence punctuation marks, or other semantic boundaries identified through natural language processing techniques.

[0051] Step S202: Adopt a context - dependent reciprocating matching mode, and use the speech recognition result corresponding to the received subsequent audio and / or subsequent text context information to perform operations such as re - scoring, re - searching, or language - model - based post - processing on the currently and previously transcribed text segments, and dynamically correct the text content in combination with the confidence information of the speech recognition result to generate a more accurate calibrated note text.

[0052] In one embodiment, the system employs an iterative or reciprocal proofreading mechanism to improve the accuracy of the preliminary text. Exemplarily, using the recognition results of subsequently received audio segments and longer text context information, the system can re-evaluate the processed text segments. This may include re-scoring the acoustic or language model based on a broader context, performing a re-search in the recognition network to find a better word sequence, or applying post-processing steps based on a large language model for grammar, spelling, and semantic corrections. Combining with the speech recognition confidence information output in step S103, the system can dynamically identify and correct potential errors, thereby generating a more accurate calibrated note text.

[0053] Step S203: Scan the preliminary text or the calibrated note text, and identify words or multi-word phrases that meet the preset features as candidate terms.

[0054] Such as Figure 4 As shown in the following is an exemplary step flowchart for determining candidate terms in step S3 of this embodiment, including:

[0055] Step S301: Extract the current text context where the candidate term is located.

[0056] In one embodiment, for each candidate term identified in step S203, the system extracts the surrounding text information as its current context. Exemplarily, this context can be a complete sentence, the current paragraph containing the candidate term, or a fixed number of words or sentences before and after the candidate term. Extracting the context is for performing context relevance analysis in subsequent steps.

[0057] Step S302: Obtain the pronunciation representation of the candidate term, compare it with the pronunciation representations of known cross-language terms in the term knowledge base, perform pronunciation-related term center matching, and generate a preliminary cross-language term matching list.

[0058] In one embodiment, the system obtains the pronunciation representation of the candidate term. This can be by performing acoustic analysis on the segment corresponding to the candidate term in the original audio, extracting acoustic features or phoneme sequences, or generating its representation in a certain standard phonetic notation through a text-to-phoneme model. Then, compare this pronunciation representation with the standard pronunciation representations of known cross-language terms stored in the term knowledge base and the pronunciation model library. By calculating the pronunciation similarity, perform pronunciation-related matching to generate a preliminary cross-language term matching list, which contains the known terms with pronunciations similar to the candidate term and their similarity scores.

[0059] Step S303: Based on the language recognition results of the term candidate or its context, as well as the center matching results of the pronunciation correlation words, preliminarily judge the cross - language features of the term candidate, and analyze the compliance of each known term in the preliminary cross - language term matching list with the context, and calculate the context matching score.

[0060] Step S304: Integrate the center matching results of the pronunciation correlation words, the language recognition results, and the context matching score, calculate the comprehensive scores of each known term in the preliminary cross - language term matching list, and select the known term with the highest score and higher than the preset threshold as the final candidate word according to the comprehensive scores. At the same time, obtain the annotation information of the final candidate word from the term knowledge base.

[0061] In step S304, when calculating the comprehensive scores of each known term in the preliminary cross - language term matching list, the following formula is used:

[0062] , where, represents the term candidate, represents the current text context, represents the th known term in the preliminary cross - language term matching list, and the known term is from the term knowledge base; represents the pronunciation similarity score between the pronunciation representation of the term candidate and the cross - language pronunciation representation of the known term . The pronunciation representation is obtained by extracting acoustic features, predicting phoneme sequences or generating pronunciation vectors from the speech of the term candidate. The cross - language pronunciation representation is the standard pronunciation representation of the known term in different languages. The pronunciation similarity score is obtained by calculating the distance or similarity measure between the pronunciation representations, and this score is greater than 0; represents the context matching score between the known term and the current text context . The context matching score is calculated by analyzing the semantic relevance between the known term and the context . The semantic relevance analysis can be implemented based on word vectors, language models, topic models, and text matching algorithms. This score value is greater than 0; represents the language recognition confidence score that the system judges the term candidate belongs to the main language of the known term . The confidence score is obtained by performing language recognition analysis on the term candidate itself or its context . This score value is greater than 0 and less than or equal to 1; represents the preset main language label of the known term in the term knowledge base. represents a weight coefficient, which is used to adjust the contribution ratio of each scoring item in the comprehensive score; represents a small constant greater than zero, which is used to avoid calculation anomalies when taking the logarithm of the scoring item.

[0063] In one embodiment, the technical solution of step S304 is elaborated: the system comprehensively calculates the pronunciation association matching result of step S302, the language recognition result of step S303, and the context matching score of step S303 to obtain the comprehensive scores of each known term in the preliminary matching list. Exemplarily, the calculation of the comprehensive score can use the above formula, where the weight coefficient can be adjusted according to the actual application scenario and expected preferences. For example, in a scenario with high requirements for pronunciation accuracy, increase the weight, and in a scenario with strong context association, increase the weight. The system selects the known term with the highest score as the most likely final candidate word according to the calculated comprehensive score, but only when the highest score exceeds a preset threshold, it is considered that an effective cross-language term matching has been found. Once the final candidate word is determined, the system retrieves the detailed annotation information associated with it from the term knowledge base and the pronunciation model library. Exemplarily, it includes its translation in the target language, complete definition, usage examples, related domain knowledge, or any other auxiliary information that helps understanding and usage.

[0064] The weight coefficient is determined according to a preset strategy, and the strategy is used to balance the relative importance of pronunciation similarity, context matching, and language recognition in the comprehensive evaluation to optimize cross-language

[0065] such as Figure 5 shown is an exemplary step flow chart for integrating text results in step S4 of this embodiment, including:

[0066] Step S401, according to the characteristics or comprehensive score of the final candidate word obtained in step S3, determine the annotation level or display priority of the final candidate word; and integrate the final candidate word and its annotation information into the position where the corresponding word appears in the calibration note text in a preset integration format corresponding to the determined annotation level or display priority. The preset integration format at least includes inserting specific marks in the calibration note text, changing the word style, adding hover tips, generating sidebar notes, creating expandable details.

[0067] In one embodiment, the technical solution of step S401 is elaborated: The system determines the annotation level or display priority of the term in the final note based on certain characteristics of the last candidate word determined in step S304 or its comprehensive score. Exemplarily, terms with high scores and strong importance may be given a "first-level annotation", while terms with lower scores or secondary importance are given a "second-level annotation". Then, the system integrates the last candidate word and its annotation information obtained from the knowledge base into the position where the word actually appears in the calibrated note text according to a preset integration format corresponding to the determined annotation level / priority. The preset integration format may at least include: inserting specific markers in the text; changing the visual style of the word to attract the user's attention; adding a hover tip to quickly display its brief information when the user hovers the mouse over the annotated word; generating an independent sidebar annotation area to centrally display the detailed annotation information of the term; or creating a clickable link or icon to allow the user to expand and view the detailed information of the term.

[0068] Step S402, output the finally generated automatic note text with cross-language term annotations. The automatic note text includes the calibrated note text and the structured annotation data associated with the last candidate word to support subsequent reference, editing, or interactive display, and the output format of the automatic note text at least includes Rich Text Format, HTML format, and Markdown format.

[0069] In one embodiment, the technical solution of step S402 is elaborated: The system finally generates and outputs an automatic note text with cross-language term annotations. This text not only includes the content of the calibrated note text generated in step S202, but also includes the structured annotation data associated with the last candidate word determined in step S304. Exemplarily, this structured annotation data can be stored inline with the text content or associated with the text file as an independent metadata file. This structured annotation is designed to support the subsequent reference, editing, and interactive display functions of the note. The output format of the automatic note text may at least include: Rich Text Format, which supports retaining the text style and basic structure; HTML format, which is convenient for display in a web browser and supports complex interactive functions; Markdown format, which is convenient for lightweight editing and supports embedding annotations or links through specific syntax; or other formats that support embedding structured data.

[0070] The above content is only an example and illustration of the present invention. Those skilled in the art of this technology can make various modifications or supplements to the described specific embodiments or use similar methods to replace them. As long as they do not deviate from the scope defined by the invention, they should fall within the protection scope of the present invention.

Claims

1. An artificial intelligence-assisted cross-language automatic note-taking and term annotation system, characterized in that: Including an audio processing and cross - language speech recognition module, which is used to receive audio data, perform pre - processing, and perform cross - language speech recognition on the pre - processed audio data to output a preliminary text; a text proofreading and term candidate recognition module, which is used to receive the preliminary text, perform reciprocal matching and proofreading to output a calibrated note text, and recognize term candidate words from the preliminary text or the calibrated note text; A cross - language term matching analysis and annotation decision - making module, which is used to receive the term candidate words and the calibrated note text, obtain the pronunciation representation of the term candidate words, compare with the cross - language pronunciation representations of known terms in the term knowledge base and pronunciation model library to generate a preliminary cross - language term matching list, perform language recognition on the term candidate words or their contexts, perform context analysis on each known term in the preliminary cross - language term matching list and calculate the context matching score, calculate a comprehensive score by integrating the pronunciation - related matching result, language recognition result and context matching score, select the final candidate word that best fits the current context, and obtain the annotation information of the final candidate word from the term knowledge base and pronunciation model library; An automatic note generation and annotation output module, which is used to receive the calibrated note text and the final candidate word and its annotation information, integrate the final candidate word and its annotation information into the calibrated note text to generate an automatic note with cross - language term annotation; A term knowledge base and pronunciation model library, which is used to store terms, meanings in different languages and corresponding cross - language pronunciation representations, and provide query support for the cross - language term matching analysis and annotation decision - making module.

2. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 1, characterized in that: The automatic note and term annotation method includes the following steps: Step S1, obtain and process audio data, and perform cross - language speech recognition to obtain a preliminary text; Step S2, perform reciprocal matching and proofreading on the preliminary text to generate a calibrated note text, and recognize term candidate words; Step S3, when term candidate words are recognized, perform cross - language feature judgment, pronunciation - related matching and context analysis, and comprehensively score to determine the cross - language term to be annotated and its information; Step S4, integrate the final candidate word and its annotation information into the note text, and output a note text with annotation.

3. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 2, characterized in that: The said Step S1 includes, Step S101, through an audio input device, obtain a multi - channel or single - channel audio data stream to be processed, and the audio data stream may contain voices of different speakers, mixed languages and environmental noises; Step S102, perform enhanced pre - processing and acoustic feature extraction on the audio data, and the pre - processing at least includes multi - channel signal processing, voice activity detection, noise reduction, acoustic environment adaptation, gain adjustment, frame segmentation, and the acoustic feature extraction is used to obtain a feature vector sequence suitable for speech recognition; Step S103, use a cross - language speech recognition model based on a deep learning model to perform a speech - to - text operation on the acoustic feature vector sequence, and the cross - language speech recognition model can process speech inputs containing language mixtures and output confidence information of the speech recognition result and / or language recognition confidence information of text segments during the recognition process; Output the preliminary text stream or text fragment.

4. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 2, characterized in that: The step S2 includes: Step S201: Chunk the preliminary text output in step S1 according to a preset time length or semantic boundary to form text fragments to be proofread and term-identified. Step S202: Adopt a context-dependent reciprocating matching mode, and use the speech recognition result corresponding to the received subsequent audio and / or subsequent text context information to perform operations such as re-scoring, re-searching, or language model-based post-processing on the currently and previously transcribed text fragments, and combine the confidence information of the speech recognition result to dynamically correct the text content to generate a more accurate calibrated note text. Step S203: Scan the preliminary text or the calibrated note text to identify words or multi-word phrases that meet the preset features as the term candidate words.

5. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 2, characterized in that: The step S3 includes: Step S301: Extract the current text context where the term candidate word is located. Step S302: Obtain the pronunciation representation of the term candidate word, compare it with the pronunciation representations of known cross-language terms in the term knowledge base, perform pronunciation-related word center matching, and generate a preliminary cross-language term matching list. Step S303: Based on the language recognition result of the term candidate word or its context, and the pronunciation-related word center matching result, preliminarily judge the cross-language feature of the term candidate word, and analyze the degree of conformity between each known term in the preliminary cross-language term matching list and the context, and calculate the context matching score. Step S304: Combine the pronunciation-related word center matching result, the language recognition result, and the context matching score, calculate the comprehensive score of each known term in the preliminary cross-language term matching list, and select the known term with the highest score and higher than the preset threshold as the final candidate word according to the comprehensive score. At the same time, obtain the annotation information of the final candidate word from the term knowledge base.

6. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 2, characterized in that: The step S4 includes: Step S401: Determine the annotation level or display priority of the final candidate word according to the characteristics or the comprehensive score of the final candidate word obtained in step S3. And integrate the final candidate word and its annotation information into the position where the corresponding word appears in the calibrated note text in a preset integration format corresponding to the determined annotation level or display priority. The preset integration format at least includes inserting a specific mark in the calibrated note text, changing the word style, adding a hover prompt, generating a sidebar note, creating an expandable detail. Step S402: Output the finally generated automatic note text with cross-language term annotations. The automatic note text includes the calibrated note text and the structured annotation data associated with the final candidate word to support subsequent viewing, editing, or interactive display. And the output format of the automatic note text at least includes rich text format, HTML format, Markdown format.

7. The artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 5, characterized in that: In step S304, to calculate the comprehensive score of each known term in the preliminary cross-language term matching list, the following formula is used: , where represents the candidate term, represents the current text context, represents the -th known term in the preliminary cross - language term matching list, and the known term is from the term knowledge base; represents the pronunciation similarity score between the pronunciation representation of the candidate term and the cross - language pronunciation representation of the known term . The pronunciation representation is obtained by extracting acoustic features, predicting phoneme sequences or generating pronunciation vectors from the speech of the candidate term. The cross - language pronunciation representation is the standard pronunciation representation of the known term in different languages. The pronunciation similarity score is obtained by calculating the distance or similarity measure between the pronunciation representations, and this score is greater than 0; represents the context matching score between the known term and the current text context . The context matching score is calculated by analyzing the semantic relevance between the known term and the context . The semantic relevance analysis can be implemented based on word vectors, language models, topic models, and text matching algorithms. This score value is greater than 0; represents the language recognition confidence score that the system determines whether the candidate term belongs to the main language of the known term . The confidence score is obtained by performing language recognition analysis on the candidate term itself or its context . This score value is greater than 0 and less than or equal to 1; represents the preset main language label of the known term in the term knowledge base; represents a weight coefficient used to adjust the contribution ratio of each scoring item in the comprehensive score; represents a small positive constant used to avoid calculation anomalies when taking the logarithm of the scoring items.

8. An artificial intelligence-assisted cross-language automatic note-taking and term annotation system according to claim 7, characterized in that: The weight coefficient is determined according to a preset strategy for balancing the relative importance of the pronunciation similarity, context matching, and the language recognition in the comprehensive evaluation to optimize the recognition accuracy of cross-language terms.

Images