Method for generating floc in business conference scene based on artificial intelligence

Abstract

The invention discloses a method for generating flocks in a business conference scene based on artificial intelligence, and particularly relates to the technical field of data processing. Conference content is analyzed by using natural language processing and computer vision technologies, high-value floc fragments are screened out, a multi-modal consistency detection model is adopted, voice features of a speaker are compared with facial expression change features, forged content generated by Deepfake or GANs is recognized, video authenticity is ensured, and the video quality is improved. The method further combines a long and short term memory network model to track the sound and picture synchronization condition, detect wrong editing or tampered content, set authority control, intelligently edit and optimize the floc segments passing authenticity verification, export the optimized floc video to an enterprise internal platform or social media, and support user feedback and secondary editing optimization. The authenticity, readability and propagation value of the video are ensured, and the credibility and influence of the business conference content are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for generating highlights in a business meeting scenario based on artificial intelligence. Background Art

[0002] The generation of highlights in business conference scenarios based on artificial intelligence refers to the use of AI technology to intelligently extract, edit and automatically generate content for key clips, highlights or interesting interactions during the meeting. These highlights can include the speaker's wonderful speeches, audience interactions, data visualization displays, and even AI-generated subtitles, soundtracks or stylized videos for meeting review, publicity or social media sharing. This technology usually combines natural language processing (NLP), computer vision and video editing algorithms to intelligently identify and filter out the most valuable clips by analyzing conference video, audio and text data. For example, AI can automatically extract high-frequency discussion topics, emotional moments or the process of forming key decisions, and integrate them into short highlights videos to make the meeting content more spreadable and reviewable.

[0003] The prior art has the following deficiencies: In the process of AI automatically editing and optimizing footage, some algorithms (such as Deepfake or GANs) may generate false or misleading content. For example, AI may cause a speaker's voice to not match his or her facial expression due to audio and video synchronization errors, or even generate false facial expressions or gestures, making the video content look like it has been tampered with, affecting the credibility of the meeting. In addition, if AI incorrectly edits or tampers with the content of the meeting, it may cause executives or decision makers to make wrong judgments based on false information, leading to wrong strategic decisions. Summary of the invention

[0004] The purpose of the present invention is to provide a method for generating highlights in a business meeting scenario based on artificial intelligence to address the shortcomings of the background technology.

[0005] In order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for generating highlights in a business meeting scenario based on artificial intelligence, comprising the following steps: S1: Obtain video, audio and text data of a business meeting, and pre-process the data, including noise removal, speech recognition and text transcription; S2: Use natural language processing and computer vision technology to identify frequently discussed topics, emotional moments, and key decision-making processes in meetings, and screen out potential highlights; S3: Based on the multimodal consistency detection model, the selected highlights are verified. By comparing the speaker's voice features with the changes in his or her facial expressions, it is detected whether there is forged content generated by Deepfake or GANs, and the authenticity of the highlights is determined; S4: Edit and optimize the authenticity-verified footage, including automatic subtitle generation, music matching, and visual effects enhancement; S5: The authenticity of the optimized highlights is further verified through the long short-term memory network model. By tracking the time series relationship between video frames and voice, incorrect editing or tampering of videos is filtered out, and permission control is set; S6: Export the final generated behind-the-scenes video to the company's internal platform, social media or storage system to support user feedback and secondary editing optimization.

[0006] Preferably, in S3, the extracted speech features are analyzed to generate a speech recognition anomaly index, and the method for obtaining the speech recognition anomaly index is: Define the input text: : Manual transcription of text, :ASR speech recognition output text; calculate Levenshtein distance , Levenshtein distance is defined as the minimum number of edit operations required to transform one string into another, and the expression is: Where: D(i,j) represents the The first i characters to The minimum edit distance of the first j characters of is an indicator function, if and If they are different, the value is 1; otherwise, the value is 0; the first item indicates deleting a character, the second item indicates inserting a character, and the third item indicates replacing a character; calculate the normalized edit distance , the expression is: ;in: and are the number of words in the manually transcribed text and the ASR recognized text, respectively; The normalized value is always between 0 and 1. The speech recognition anomaly index SRAI reflects the degree of deviation of ASR recognition. The expression is: ; Where: λ is the smoothing parameter.

[0007] Preferably, an abnormal threshold is set. If SRAI < 0.2, the ASR recognition quality is good and the error is acceptable; if 0.2 ≤ SRAI < 0.5, there are some recognition errors and further evaluation is required; if SRAI ≥ 0.5, the recognition quality is poor, including serious misrecognition or ASR model abnormality.

[0008] Preferably, in S3, the facial expression variation index is generated after analyzing the extracted facial expression variation features, and the facial expression variation index is obtained by: Set the conference video to be divided into multiple highlights segments, each of which contains consecutive video frames. To extract facial expression features, input: V: input video clip, containing N frames; : The facial expression feature vector detected in the tth frame; each frame The facial expression features are expressed as: ;in: Represents the 2D coordinates of facial key points, Deep learning features representing facial muscle movements; In continuous video frames, calculate the expression change rate of adjacent frames As the instantaneous expression variability, the expression is: ; Calculate the facial expression variability index, the expression is: ; N is the total number of frames in the video clip, and FEVI is the facial expression variability index.

[0009] Preferably, if FEVI<0.2, the expression changes are normal and there is no abnormality; if 0.2≤FEVI<0.5, the expression changes are abnormal and further analysis is required; if FEVI≥0.5, the expression changes are highly abnormal, which is Deepfake or unnatural facial expression synthesis.

[0010] Preferably, in S3, the speech recognition anomaly index and the facial expression variation index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model. The machine learning model uses each group of comprehensive feature vectors to predict the authenticity value label of the screened highlights as the prediction target, and takes minimizing the sum of prediction errors for the authenticity value labels of all screened highlights as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The authenticity value of the screened highlights is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0011] Preferably, the authenticity value of the screened footage clips obtained is compared with a authenticity value reference threshold pre-set based on historical data. If the authenticity value of the screened footage clips obtained is greater than or equal to the pre-set authenticity value reference threshold, it means that there is no forged content generated by Deepfake or GANs, the authenticity of the footage clip is high, and the corresponding footage clip is marked as a low-risk clip; if the authenticity value of the screened footage clips obtained is less than the pre-set authenticity value reference threshold, it means that there is forged content generated by Deepfake or GANs, the authenticity of the footage clip is low, and the corresponding footage clip is marked as a high-risk clip; the low-risk clips automatically pass the verification and are stored in the database; the high-risk clips are submitted for manual review to further confirm the authenticity.

[0012] Preferably, in S5, the authenticity of the optimized highlights is further verified by a long short-term memory network model, specifically: The input sequence for the LSTM model is constructed by extracting the speech recognition anomaly index and facial expression variation index from the highlights; Use LSTM to process time series data and predict the video authenticity score at each time step. LSTM processes the input features of time step t, calculates the hidden state and authenticity score, and the model outputs the authenticity prediction value for each time step t. : Take the average authenticity score of all time steps t of the entire behind-the-scenes clip as the overall authenticity score of the behind-the-scenes clip, with a value range of [0,1].

[0013] Preferably, the permission level of the behind-the-scenes clip is determined based on the authenticity score output by the model, and an authenticity threshold is set; if the authenticity score output by the model is greater than or equal to the set authenticity threshold, it means that the video authenticity is high, the audio and video are synchronized, and there are no obvious signs of tampering. It is marked as a low-risk authenticity clip and allowed to be automatically published or stored in a database; if the authenticity score output by the model is less than the set authenticity threshold, it means that the video contains fake faces, voice tampering or editing errors, and it is marked as a high-risk non-authentic clip, which requires manual review.

[0014] In the above technical solution, the technical effects and advantages provided by the present invention are: 1. The present invention deeply verifies the authenticity of the highlights through multimodal consistency detection and long short-term memory network (LSTM), effectively solving the problems of audio and video asynchrony, Deepfake tampering and false information dissemination that may be caused by AI editing in the prior art. The authenticity of the conference content is quantified by the speech recognition anomaly index (SRAI) and the facial expression variation index (FEVI), combined with the polynomial regression model for prediction, and the time series relationship between the voice and video frames is tracked by LSTM to further detect editing errors or forged content. In addition, the system supports intelligent editing optimization, including automatic subtitle generation, music matching and visual effects enhancement, to improve the viewing experience and communication value of the video.

[0015] 2. While ensuring the authenticity of the footage, the present invention also improves the automation and security of content generation. By setting permission control, low-risk footage can be directly published or stored in the database, while high-risk footage requires manual review to ensure the credibility of the disseminated content. The final generated footage video can be exported to the company's internal platform or social media, and supports user feedback and secondary editing optimization, allowing companies to efficiently and securely manage and promote business meeting content, thereby enhancing brand influence and decision-making transparency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] For examples, see Figure 1 As shown, the method for generating highlights in a business meeting scenario based on artificial intelligence in this embodiment includes the following steps: S1: Obtain video, audio and text data of a business meeting, and pre-process the data, including noise removal, speech recognition and text transcription; S2: Use natural language processing and computer vision technology to identify frequently discussed topics, emotional moments, and key decision-making processes in meetings, and screen out potential highlights; S3: Based on the multimodal consistency detection model, the selected highlights are verified. By comparing the speaker's voice features with the changes in his or her facial expressions, it is detected whether there is forged content generated by Deepfake or GANs, and the authenticity of the highlights is determined; S4: Edit and optimize the authenticity-verified footage, including automatic subtitle generation, music matching, and visual effects enhancement; S5: The authenticity of the optimized highlights is further verified through the long short-term memory network model. By tracking the time series relationship between video frames and voice, incorrect editing or tampering of videos is filtered out, and permission control is set; S6: Export the final generated behind-the-scenes video to the company's internal platform, social media or storage system to support user feedback and secondary editing optimization.

[0020] In S1, before a business meeting begins, the system deploys multiple data acquisition devices, including high-definition cameras, omnidirectional microphone arrays, and meeting recording software to ensure that audio and video information during the meeting is fully captured.

[0021] Video capture: Use a high-definition camera to record the meeting scene, including the speaker's expression, gestures, and projected content. The camera supports autofocus and can track the speaker intelligently based on his or her position. Audio capture: Obtain high-quality voice data through an omnidirectional microphone array, and combine beamforming technology to enhance the speaker's voice and reduce the interference of environmental noise. Text data collection: If the meeting is equipped with a real-time subtitle or note-taking system, text data can be directly obtained; otherwise, the audio needs to be transcribed into text through speech recognition technology. Clean, optimize, and format the collected meeting data to improve the accuracy and stability of subsequent analysis. Use Adaptive Noise Cancellation (ANC) technology to filter background noise, such as keyboard tapping, air conditioning noise, or echoes from remote participants in the conference room. Use speech enhancement algorithms (Speech Enhancement), such as the Wavenet noise reduction model based on deep learning, to improve the clarity of human voices, so that the speech recognition system can transcribe text more accurately.

[0022] Use end-to-end speech recognition models (such as DeepSpeech or Whisper) to convert conference audio to text. Use speaker diarization technology based on x-vector or Deep Speaker Embeddings models to distinguish the voices of different speakers and annotate the speakers. Perform grammatical correction and semantic optimization on the transcribed text, and use BERT or GPT language models for automatic error correction to improve transcription quality.

[0023] Combined with natural language processing (NLP) technology, keyword extraction and topic analysis are performed on the transcribed text to identify high-frequency discussion topics in the meeting. Sentiment Analysis algorithm is used to evaluate the speaker's tone and identify emotional moments in the meeting for subsequent editing. Through semantic segmentation, long texts are split into logically complete segments to provide structured data for subsequent editing.

[0024] The pre-processed audio and video data, text transcription results, and related metadata (such as speaker identity, timestamp, etc.) are stored in a database for easy subsequent analysis and call. The data format uses JSON or XML for structured storage, making subsequent AI editing and highlight generation more efficient.

[0025] In S2, first, the system receives and processes the pre-processed video, audio and text data, and extracts key features for screening the highlights.

[0026] Text data: Extract keywords, topics, and sentiment features based on the conference speech transcription results.

[0027] Audio data: Analyze voice pitch, speech rate, and voice sentiment to identify moments of high emotional volatility.

[0028] Video data: Detect the emotional state of the speaker or audience, such as smiling, surprise, or applause, based on facial expression recognition and body movement analysis.

[0029] Use natural language processing technology to extract high-frequency discussion topics from meeting text data to identify the most valuable content segments.

[0030] Keyword extraction: Use algorithms such as TF-IDF and TextRank to extract core terms from the meeting, and combine pre-trained language models (such as BERT and GPT) for contextual understanding to identify the most important topics.

[0031] Topic modeling: Use Latent Dirichlet Allocation (LDA) or BERT topic clustering methods to semantically classify meeting content and extract the main discussion topics.

[0032] Co-occurrence analysis: Analyze the co-occurrence relationship between keywords and build a topic network to discover important discussion content related to the core topics of the meeting.

[0033] Combined with audio and video data, key moments related to the emotional fluctuations of the audience or speakers during the meeting are detected to screen out highlights with communication value.

[0034] Audio emotion analysis: Based on speech emotion recognition models such as CNN+BiLSTM or Wav2Vec, it analyzes the pitch, speaking speed, and energy changes of speech, and identifies excited, excited, or emphasized speech fragments.

[0035] Expression recognition: Use deep learning models such as FaceNet, OpenCV DNN, or EmotionNet to detect the emotions of speakers and audiences, such as smiling, surprised, and thinking seriously, to select content with strong interactivity.

[0036] Body movement analysis: Through pose estimation technology, such as OpenPose or MediaPipe, the speaker’s gesture changes, standing movements, etc. are detected to identify highly dynamic moments.

[0037] In meetings, key decisions are often the most valuable highlights, so the decision-making process needs to be comprehensively identified based on text, voice, and visual data.

[0038] Text decision recognition: Based on semantic analysis models such as BERT and RoBERTa, it detects sentences in meetings involving decision-making terms such as "decision", "approval", "vote", and "take action", and analyzes the decision-making process in combination with the context.

[0039] Speech emphasis detection: Using audio feature extraction models such as MFCC (Mel-Frequency Cepstral Coefficients) and VGGish, the stress, pauses, and intonation changes in speech are analyzed to determine the speaker's speech pattern in key decision-making stages.

[0040] Voting and interaction detection: Based on computer vision technology, it detects actions such as raising hands, clapping, and nodding in meetings. Combined with speech recognition results, it determines whether the decision is unanimously recognized and selects corresponding clips.

[0041] Score all identified potential highlights and select the most valuable content. Content value scoring: Calculate the comprehensive score of each segment based on the popularity of the discussion topic, emotional intensity, and decision-making importance. Visual analysis: Use data visualization technology to display the emotional fluctuation curve, keyword cloud, and video timeline of each highlight segment to assist manual adjustment and optimization. Final screening: Set thresholds to filter high-value segments and make personalized recommendations based on user needs, such as "the most interactive segment" and "core decision-making segment".

[0042] The generated highlights are stored in the enterprise database and archived in a structured format (JSON, XML, etc.) to support subsequent calls. They can be exported to standard video formats (MP4, MOV, etc.) and automatically add subtitles, soundtracks, transition effects and other optimization elements. They can be published to social media, internal enterprise platforms or meeting record systems with one click for quick dissemination and review.

[0043] In S3, after the candidate highlights are screened, multimodal features for authenticity verification are extracted from the video, audio, and text data.

[0044] Speech feature extraction, including: Acoustic feature extraction methods such as Mel-spectrogram and MFCC (Mel-frequency cepstral coefficient) are used to obtain the key features of the speaker's voice.

[0045] Use deep learning models such as Wav2Vec2.0 or Deep Speaker Embeddings to feature encode the speaker's voice identity to ensure that the voice source is authentic and reliable.

[0046] Combined with the BiLSTM or Transformer model, the speech rhythm, stress, and pause patterns are analyzed to ensure that the speech content is natural and there is no trace of speech splicing or GANs generation.

[0047] Facial expression change feature extraction includes: Use a face recognition model such as FaceNet, Dlib, or MediaPipe to extract the coordinates of the speaker's facial key points, including the dynamic changes of the mouth, eyes, and eyebrows.

[0048] Use the LipNet or SyncNet model to analyze the speaker's mouth movement trajectory, align it with the speech waveform, and determine whether there is a problem of lip shape and speech being out of sync.

[0049] Use CNN+LSTM or Vision Transformer model to perform time series analysis on facial expression changes and detect whether there are unnatural facial movements (such as abnormal blinking frequency, stiff facial expression, etc.).

[0050] After analyzing the extracted speech features, a speech recognition anomaly index is generated. The method for obtaining the speech recognition anomaly index is as follows: Define the input text: : Manually transcribed text (Ground Truth), : ASR speech recognition output text; calculate Levenshtein distance , Levenshtein distance is defined as the minimum number of edit operations required to transform one string into another, and the expression is: Where: D(i,j) represents the The first i characters to The minimum edit distance of the first j characters of is an indicator function, if and If they are different, the value is 1 (replace operation); otherwise, the value is 0 (no edit operation). The first item means deleting a character, the second item means inserting a character, and the third item means replacing a character; In order to standardize the edit distance to be applicable to texts of different lengths, the normalized edit distance is calculated , the expression is: ;in: and are the number of words (or characters) in the manually transcribed text and the ASR recognized text, respectively. The normalized value is always between 0 and 1. The speech recognition anomaly index SRAI reflects the degree of deviation of ASR recognition. The expression is: ; Where: λ is the smoothing parameter (usually 3-5, depending on the system's sensitivity to errors).

[0051] Exponential function A smaller NED corresponds to a lower anomaly index, while a larger NED makes the anomaly index approach 1.

[0052] Set the abnormal threshold. If SRAI < 0.2, the ASR recognition quality is good and the error is acceptable. If 0.2 ≤ SRAI < 0.5, there may be some recognition errors and further evaluation is required. If SRAI ≥ 0.5, the recognition quality is poor and may contain serious misrecognition or ASR model abnormalities.

[0053] After analyzing the extracted facial expression change features, the facial expression variation index is generated. The method for obtaining the facial expression variation index is as follows: The conference video is divided into multiple highlights segments, each of which contains consecutive video frames. (time t) to extract facial expression features. Input: V: input video clip, containing N frames; : The facial expression feature vector detected in the tth frame; Facial feature extraction method: Use FaceNet, OpenFace, Dlib or MediaPipe to extract 68 facial landmarks to form a key point vector ; Use ResNet+LSTM or Facial Action Coding System (FACS) to calculate expression embedding vectors ; Finally, each frame The facial expression features are expressed as: ;in: Represents the 2D coordinates of facial key points (such as eyes, mouth, eyebrows, etc.), Deep learning features representing facial muscle movements; In continuous video frames, calculate the expression change rate of adjacent frames As the instantaneous expression variability, the expression is: ; Calculate the facial expression variability index, the expression is: ; N is the total number of frames in the video clip, and FEVI is the facial expression variability index.

[0054] If FEVI<0.2, the expression changes are normal and there is no abnormality; if 0.2≤FEVI<0.5, there are certain abnormalities in the expression changes and further analysis is required; if FEVI≥0.5, the expression changes are highly abnormal and may be Deepfake or unnatural facial expression synthesis.

[0055] The speech recognition anomaly index and the facial expression variation index are converted into comprehensive feature vectors, which are used as inputs of the machine learning model. The machine learning model predicts the authenticity value labels of the screened highlights with each group of comprehensive feature vectors as the prediction target, and minimizes the sum of the prediction errors of the authenticity value labels of all the screened highlights as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The authenticity value of the screened highlights is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0056] The authenticity value of the selected highlights is obtained by obtaining the corresponding function expression from the comprehensive feature vector training data of the trained machine learning model: ; In the formula, is the output function of the model, SRAI is the speech recognition anomaly index, FEVI is the facial expression variation index, It is the authenticity value of the filtered footage.

[0057] The authenticity value of the obtained screened footage is compared with the authenticity value reference threshold pre-set according to historical data. If the authenticity value of the obtained screened footage is greater than or equal to the pre-set authenticity value reference threshold, it means that there is no forged content generated by Deepfake or GANs, the authenticity of the footage is high, and the corresponding footage is marked as a low-risk footage; if the authenticity value of the obtained screened footage is less than the pre-set authenticity value reference threshold, it means that there is forged content generated by Deepfake or GANs, the authenticity of the footage is low, and the corresponding footage is marked as a high-risk footage; low-risk footage is automatically verified and stored in the database; high-risk footage (such as detection of GAN generation traces or identity mismatch) is submitted for manual review to further confirm the authenticity.

[0058] The footage clips that pass the authenticity verification are stored in the enterprise database and marked as "verified". If the footage clips fail the authenticity test, they can be screened again, edited, or directly discarded to prevent the spread of misleading information. A detailed authenticity test log is generated to record the risk score and test results of each clip, which is convenient for subsequent tracing and optimization.

[0059] S4: Edit and optimize the authenticity-verified footage, including automatic subtitle generation, music matching, and visual effects enhancement.

[0060] To edit and optimize the authenticity-verified footage, you first need to automatically generate subtitles to improve the readability and dissemination of the content. Use speech recognition (ASR) technology to extract audio content, and combine it with natural language processing (NLP) for grammar correction and sentence segmentation to ensure that the subtitles are smooth and in context. In addition, time synchronization technology (such as DTW or VAD) is used to accurately align subtitles and speech to avoid subtitle delays or misalignment. At the same time, it supports multi-language translation and provides high-quality cross-language subtitles through models such as Transformer or BERT to meet the needs of international conferences.

[0061] In terms of music matching, the system will automatically recommend suitable background music based on the mood and rhythm of the meeting content. Through audio emotion analysis (Speech Emotion Recognition, SER), the speaker's tone, intonation and emotional type are detected, and music that matches the atmosphere of the meeting is selected. For example, in formal business decision-making scenarios, low-key and steady background music is matched, while in team interaction sessions, faster and more positive music can be selected. In addition, a dynamic volume adjustment algorithm is used to prevent background music from interfering with the content of the speech, while AI automatic noise reduction and audio equalization are supported to optimize the overall listening experience.

[0062] To enhance the visual appeal of the video, visual effects can also be enhanced, including automatic camera switching, key content highlighting, and infographic overlays. Based on computer vision technologies (such as YOLO, OpenPose, etc.), speakers are automatically detected and tracked, and the camera angle is intelligently adjusted to make the picture more dynamic. At the same time, augmented reality (AR) technology is used to overlay key information in the video, such as meeting points, real-time data analysis, or voting results, so that the audience can understand the content more intuitively. In addition, smooth transition animations, brand logos, and dynamic titles can be added to enhance the professionalism and brand recognition of the video.

[0063] S5: The authenticity of the optimized highlights is further verified through the long short-term memory network model. By tracking the time series relationship between video frames and speech, incorrect editing or tampering of videos is filtered out, and permission control is set.

[0064] The input sequence for the LSTM model is constructed by extracting the speech recognition anomaly index and facial expression variation index from the highlights; LSTM is used to process time series data and predict the video authenticity score at each time step. LSTM processes the input features of time step t, calculates the hidden state and authenticity score, and finally, the model outputs the authenticity prediction value of each time step t. : Take the average authenticity score of all time steps t of the entire behind-the-scenes clip as the overall authenticity score of the behind-the-scenes clip, with a value range of [0,1].

[0065] According to the authenticity score output by the model, the permission level of the behind-the-scenes clip is determined, and the authenticity threshold Tth is set (such as Tth=0.7): If the authenticity score output by the model is greater than or equal to the set authenticity threshold, it means that the video is highly authentic, the audio and video are synchronized, and there are no obvious signs of tampering. It is marked as a low-risk authenticity clip and allowed to be automatically published or stored in the database. If the authenticity score output by the model is less than the set authenticity threshold, it means that the video may contain fake faces, voice tampering, or editing errors. It is marked as a high-risk non-authentic clip and requires manual review. Access permission restrictions may be triggered, and only administrators or auditors are allowed to view it.

[0066] S6: Export the final generated behind-the-scenes video to the company's internal platform, social media or storage system to support user feedback and secondary editing optimization.

[0067] After authenticity verification and optimized editing, the resulting behind-the-scenes video can be automatically exported to the company's internal platform, social media or cloud storage system for quick access and sharing by different audiences. The company's internal platform can be used for archiving and employee training, while social media platforms (such as LinkedIn, YouTube, and Twitter) can help with branding and marketing. In addition, the system supports multiple video formats (MP4, MOV, AVI) and resolution adaptation to ensure smooth playback on different devices, while integrating a secure encryption mechanism to prevent unauthorized access and tampering.

[0068] To improve content quality and interactivity, the behind-the-scenes video also supports user feedback and secondary editing optimization. Users can provide opinions through likes, comments or rating systems, and AI can improve editing strategies based on feedback analysis, such as adjusting the order of clips, optimizing subtitles or enhancing visual effects. At the same time, it supports video editing permission management, allowing authorized users to perform secondary editing directly in the cloud, such as supplementing key content, replacing background music or adding corporate brand elements, to ensure that the final video meets the communication needs and achieves the best effect.

[0069] In this embodiment, the video, audio and text data of the business meeting are first obtained and preprocessed, including noise removal, speech recognition and text transcription, to ensure data quality. Subsequently, natural language processing and computer vision technology are used to analyze the content of the meeting, identify high-frequency discussion topics, emotional moments and the process of forming key decisions, and screen out potential highlights. To ensure authenticity, a multimodal consistency detection model is used to compare the speaker's voice features with facial expression change features to detect whether there is forged content generated by Deepfake or GANs. The highlights that have passed the authenticity verification will be edited and optimized, including automatic generation of subtitles, matching of appropriate soundtracks and enhancement of visual effects to enhance the viewing experience. On this basis, the long short-term memory network model is further used to track the time series relationship between video frames and speech, detect whether there is wrong editing or tampering of content, and set corresponding access rights. Finally, the optimized highlights video will be exported to the company's internal platform, social media or storage system, and support user feedback and secondary editing optimization to ensure content quality and communication effect.

[0070] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0071] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0072] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects associated with each other are in an "or" relationship, but it may also indicate an "and / or" relationship, which can be understood by referring to the context. A person of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this article can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0073] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for generating highlights in a business meeting scenario based on artificial intelligence, characterized in that: The following steps are involved: S1: Obtain video, audio and text data of a business meeting, and pre-process the data, including noise removal, speech recognition and text transcription; S2: Use natural language processing and computer vision technology to identify frequently discussed topics, emotional moments, and key decision-making processes in meetings, and screen out potential highlights; S3: Based on the multimodal consistency detection model, the selected highlights are verified. By comparing the speaker's voice features with the changes in his or her facial expressions, it is detected whether there is forged content generated by Deepfake or GANs, and the authenticity of the highlights is determined; S4: Edit and optimize the authenticity-verified footage, including automatic subtitle generation, music matching, and visual effects enhancement; S5: The authenticity of the optimized highlights is further verified through the long short-term memory network model. By tracking the time series relationship between video frames and voice, incorrect editing or tampering of videos is filtered out, and permission control is set; S6: Export the final generated behind-the-scenes video to the company's internal platform, social media or storage system to support user feedback and secondary editing optimization.

2. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 1 is characterized in that: In S3, the extracted speech features are analyzed to generate a speech recognition anomaly index, and the method for obtaining the speech recognition anomaly index is as follows: Define the input text: : Manual transcription of text, :ASR speech recognition output text; calculate Levenshtein distance , Levenshtein distance is defined as the minimum number of edit operations required to transform one string into another, and the expression is: Where: D(i,j) represents the The first i characters to The minimum edit distance of the first j characters of is an indicator function, if and If they are different, the value is 1; otherwise, the value is 0; the first item indicates deleting a character, the second item indicates inserting a character, and the third item indicates replacing a character; calculate the normalized edit distance , the expression is: ;in: and are the number of words in the manually transcribed text and the ASR recognized text, respectively; The normalized value is always between 0 and 1. The speech recognition anomaly index SRAI reflects the degree of deviation of ASR recognition. The expression is: ; Where: λ is the smoothing parameter.

3. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 2 is characterized in that: The abnormal threshold is set. If SRAI < 0.2, the ASR recognition quality is good and the error is acceptable. If 0.2 ≤ SRAI < 0.5, some recognition errors exist and further evaluation is required. If SRAI ≥ 0.5, the recognition quality is poor, including serious misrecognition or ASR model abnormality.

4. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 3 is characterized in that: In S3, the facial expression variation index is generated after analyzing the extracted facial expression variation features. The facial expression variation index is obtained by: Set the conference video to be divided into multiple highlights segments, each of which contains consecutive video frames. To extract facial expression features, input: V: input video clip, containing N frames; : The facial expression feature vector detected in the tth frame; each frame The facial expression features are expressed as: ;in: Represents the 2D coordinates of facial key points, Deep learning features representing facial muscle movements; In continuous video frames, calculate the expression change rate of adjacent frames As the instantaneous expression variability, the expression is: ; Calculate the facial expression variability index, the expression is: ; N is the total number of frames in the video clip, and FEVI is the facial expression variability index.

5. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 4 is characterized in that: If FEVI<0.2, the expression changes are normal and there is no abnormality; if 0.2≤FEVI<0.5, the expression changes are abnormal and further analysis is required; if FEVI≥0.5, the expression changes are highly abnormal, which is Deepfake or unnatural facial expression synthesis.

6. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 5 is characterized in that: In S3, the speech recognition anomaly index and the facial expression variation index are converted into comprehensive feature vectors, and the comprehensive feature vectors are used as inputs of the machine learning model. The machine learning model predicts the authenticity value labels of the screened highlights with each group of comprehensive feature vectors as the prediction target, and minimizes the sum of the prediction errors of the authenticity value labels of all the screened highlights as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The authenticity value of the screened highlights is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

7. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 6 is characterized in that: The authenticity value of the obtained screened footage is compared with the authenticity value reference threshold pre-set according to historical data. If the authenticity value of the obtained screened footage is greater than or equal to the pre-set authenticity value reference threshold, it means that there is no forged content generated by Deepfake or GANs, the authenticity of the footage is high, and the corresponding footage is marked as a low-risk footage; if the authenticity value of the obtained screened footage is less than the pre-set authenticity value reference threshold, it means that there is forged content generated by Deepfake or GANs, the authenticity of the footage is low, and the corresponding footage is marked as a high-risk footage; low-risk footage automatically passes the verification and is stored in the database; high-risk footage is submitted for manual review to further confirm the authenticity.

8. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 7 is characterized in that: In S5, the authenticity of the optimized highlights was further verified by the long short-term memory network model, specifically: The input sequence for the LSTM model is constructed by extracting the speech recognition anomaly index and facial expression variation index from the highlights; Use LSTM to process time series data and predict the video authenticity score at each time step. LSTM processes the input features of time step t, calculates the hidden state and authenticity score, and the model outputs the authenticity prediction value for each time step t. : Take the average authenticity score of all time steps t of the entire behind-the-scenes clip as the overall authenticity score of the behind-the-scenes clip, with a value range of [0,1].

9. The method for generating highlights in a business conference scenario based on artificial intelligence according to claim 8, characterized in that: Based on the authenticity score output by the model, the permission level of the behind-the-scenes clip is determined and the authenticity threshold is set; if the authenticity score output by the model is greater than or equal to the set authenticity threshold, it means that the video is highly authentic, the audio and video are synchronized, and there are no obvious signs of tampering. It will be marked as a low-risk authenticity clip and allowed to be automatically published or stored in the database; if the authenticity score output by the model is less than the set authenticity threshold, it means that the video contains fake faces, voice tampering or editing errors, and it will be marked as a high-risk non-authentic clip and require manual review.

Images