Meeting interaction method and system based on meta universe

Abstract

The invention discloses a meta universe-based conference interaction method, and relates to the technical field of virtual conferences, and the method comprises the steps: extracting the facial features of participants, comparing the facial features with a pre-established facial feature library, obtaining the unique identity of each participant, and recognizing the identities of the participants; after the identities of the participants are determined, position changes of key body joint points of the participants are detected in real time, and specific actions of the participants are judged by analyzing relative positions and movement tracks between the joint points; the method comprises the following steps: receiving the acoustic characteristics of the participant language, converting the voice signal of the participant into a text, and analyzing the semantics and emotion of the text in combination with natural language processing to obtain the language expression intention of the participant. According to the method and the device, the optimal interaction mode can be dynamically selected from the preset library according to different interaction intentions, and the audio and video parameters and the interface layout are adjusted in real time, so that intelligent interaction in a multi-user scene is realized, the interaction experience of a conference is optimized, and the interaction effect can also be improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual conference interaction technology, and in particular to a conference interaction method and system based on the metaverse. Background Art

[0002] With the development of metaverse technology, humanoid complexes can be constructed in the virtual world through computer graphics, motion capture, speech synthesis and other technologies, making metaverse conferences gradually become a new form of conference.

[0003] At present, in virtual conference interactions, although online conferences have been upgraded to provide participants with an immersive virtual environment, in virtual environments, especially in complex scenarios of multi-person meetings, the number, position, and posture of participants are constantly changing, and individual movements and postures are reduced in the conference scene, which increases the difficulty of identifying and tracking interactive movements in the conference. In addition, the audio quality in multi-person meetings is easily disturbed, thus affecting the interaction effect. Therefore, in complex scenarios of multi-person meetings, the current interaction method has poor interaction effect between users. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art and to propose a conference interaction method and system based on the metaverse.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A conference interaction method based on metaverse, comprising the following steps:

[0007] S1, using face recognition, extracting the facial features of the participants and comparing them with the pre-established face feature database to obtain the unique identity of each participant and identify the participants;

[0008] S2, after confirming the identity of the participant, detect the position changes of the participant's key body joints in real time, and judge the specific actions of the participant by analyzing the relative positions and motion trajectories between the joints;

[0009] S3, receiving the acoustic features of the participant's language, converting the participant's voice signal into text, and combining natural language processing to analyze the semantics and sentiment of the text to obtain the participant's language expression intention;

[0010] S4, comprehensively analyzing the identity, action and language intention of the participants, determining the interaction intention of each participant according to the preset interaction intention judgment rules, and dividing them into different interaction intention categories;

[0011] S5, for different interaction intent categories, dynamically selects the most matching interaction method from the pre-designed conference interaction method library, and adjusts the audio and video codec parameters and interaction interface layout in real time.

[0012] A conference interactive system based on the metaverse, applicable to the above method

[0013] Compared with the prior art, the present invention has the following advantages:

[0014] The present invention uses face recognition and skeleton tracking to identify the identity and actions of participants in real time, combines voice recognition and natural language processing to analyze the language intentions of participants, comprehensively judges the interaction intentions, dynamically selects the best interaction method from a preset library for different interaction intentions, and adjusts audio and video parameters and interface layout in real time to achieve intelligent interaction in multi-person scenarios, optimize the interactive experience of the meeting, and improve the effect of interaction.

[0015] In addition, in intelligent interaction, through different situations such as one-to-one, one-to-many or many-to-many, the relative positions of the interacting personnel can be intelligently adjusted, which can not only achieve rapid responses and feedback from the other party in face-to-face communication, making the communication more timely, efficient and targeted, but also achieve concentrated information delivery to many listeners with higher efficiency, so as to enrich the interactive experience and further improve the effect of interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a conference interaction method based on the metaverse proposed by the present invention. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0018] Reference Figure 1 , a conference interaction method based on metaverse, comprising the following steps:

[0019] S1, uses face recognition to extract the facial features of participants and compare them with the pre-established face feature library to obtain the unique identity of each participant and identify the participant.

[0020] Specifically, the method includes locating and marking the face area from the input image or video through image processing to obtain face area data; if the face area data exists, adjusting the detected face area to a standard posture and size through affine transformation to ensure face alignment and obtain an aligned face image; extracting a feature vector capable of representing the identity through the aligned face image using feature extraction, including Gabor wavelet, to obtain a face feature vector; determining whether the face feature vector is complete, and if complete, calculating the similarity between the face feature vector and a pre-established face feature library of persons with known identities to obtain a similarity score; and calculating the similarity score based on the similarity score. , by comparing the similarity with each feature vector in the face feature library, the most similar feature vector is determined to obtain the comparison result; from the comparison result, the identity identifier corresponding to the highest similarity is obtained to obtain the unique identity of the participant; whether the unique identity is valid is determined, and if valid, the identity is updated to the identity recognition record system to complete the identity recognition record; through the identity recognition record system, the entry and exit records and identity of the participants are analyzed to obtain the behavior analysis data of the participants; if the behavior analysis data is complete, a behavior report of the participants is generated, and the data is fed back to the relevant system to form a closed-loop management of the participant's identity recognition.

[0021] Exemplarily, in image processing technology, when an image or video frame is input, the Haar cascade classifier in the OpenCV library or the HOG+SVM detector in the Dlib library can be used to locate the face area. If the face area is detected, the coordinates and size data of the area are extracted. Next, the face area is adjusted to a standard posture and size by affine transformation, which is usually achieved by defining the standard coordinates of the eye position and then calculating the required transformation matrix for transformation, which can be achieved using the cvgetAffineTransform function in OpenCV. The aligned face images can be unified to 112x112 pixels in size for subsequent processing. In the feature extraction stage, the Gabor wavelet transform is applied to extract local features of the face through a multi-frequency and multi-directional filter group to generate a Gabor feature map. Then, these feature maps are reduced in dimension and concatenated into a fixed-length feature vector, for example, with a length of 128 dimensions. This feature vector is used to represent personal identity. If the feature vector is complete, that is, there are no missing values ​​in all dimensions, similarity calculation can be performed. Next, the feature vector is compared with a pre-established identity feature library. The similarity score between feature vectors is calculated using cosine similarity or Euclidean distance. Assuming that the highest similarity calculated is 95, which is the most similar to the corresponding feature vector, the identity identifier corresponding to the feature vector is the unique identity identifier of the participant. After obtaining the identity identifier, check its validity, such as whether it is within the validity period or whether it is a legitimate participant. If valid, this identity identifier is updated to the identity recognition record system, which records every entry and exit information of the participant in order to track their activity data. By analyzing the identity recognition records, the behavior data of the participants, such as attendance time and frequency, can be obtained. If the behavior analysis data is complete and correct, a detailed behavior report is generated and fed back to relevant systems, such as the security monitoring system, to achieve closed-loop management of participant identity recognition. Throughout the process, each step is processed automatically without manual intervention to ensure efficient and accurate identity recognition.

[0022] S2, after confirming the identity of the participant, detect the position changes of the participant's key body joints in real time, and judge the specific actions of the participant by analyzing the relative positions and motion trajectories between the joints.

[0023] Specifically, if the identity information of the participant has been obtained, the position changes of the key body joints of the participant are captured in real time through the skeleton tracking algorithm to obtain the initial set of joint coordinates; based on the initial set of joint coordinates and the time series, the position change of each joint is calculated to obtain the motion trajectory data of the joint; if the motion trajectory data of all relevant nodes are obtained, the relative position changes between the joints are analyzed to determine the relative motion angles of each joint; if the relative motion angles of the joints have been determined, they are compared with the predefined motion pattern library to determine the specific action category of the participant; by judging the specific action category, the starting time point of the action is marked, and the end time point of the action is determined according to the subsequent motion changes; if the starting and ending time points of the action have been determined, the time duration of the action is calculated according to the time series; after obtaining the time duration of the action, the number of all actions of the current participant is counted in combination with the previous action information; if the number of actions is counted, the action behavior report of the participant is formed through the updated statistical data and action information; according to the action behavior report, the activity status of the participant during the meeting is output to provide data support for subsequent analysis and decision-making.

[0024] Exemplarily, after obtaining the identity information of the participants, deep learning and computer vision technology can be used in the skeleton tracking algorithm. For example, the body movements of the participants are captured in real time by an RGBD camera, and the three-dimensional coordinates of the joints including the head, shoulder, elbow, knee, and foot are obtained to form an initial joint coordinate set. Assuming that the data is captured at time t0, the initial coordinates of the right shoulder are (5,6,0), and then updated to (6,6,1) at time t1. By comparing the coordinate changes of these two time points, the speed and direction of the joint can be calculated. Through the cumulative analysis of the time series data, the motion trajectory can be extracted, and the Kalman filter is used to smooth the trajectory to form stable motion trajectory data. If the motion trajectory data of all related nodes are obtained, the Euclidean distance formula is used to calculate the relative position change between each joint point, such as calculating the distance change between the left hand and the right hand to determine the angle of the arm opening. After determining the relative motion angle of the joint point, it can be compared with the predefined motion pattern library through a pattern recognition method based on a convolutional neural network. Assuming that a predefined motion pattern is waving, the pattern library contains the trained waving motion features, such as the amplitude and frequency of arm swinging. Through similarity calculation, the specific action category of the participant is determined to be waving. After the action category is identified, the starting time point of the action is marked as t2. When the arm is subsequently detected to fall back to a stationary state, the end time point of the waving action is determined to be t3. If the start and end time points of the action have been determined, the duration of the action is calculated by the time difference t3t2. For example, if t2 is 5 seconds and t3 is 8 seconds, the duration of the action is 3 seconds. After obtaining the duration of the action, combined with the previously identified action information, the database records and classification algorithms are used to count the number of all actions of the current participant. Assuming that the participant has completed 5 waving, 3 nodding, and 2 hand-raising actions in the meeting, a participant action behavior report is generated based on the updated statistical data and action information. The report records the number and duration of each action category in detail. Finally, based on the action behavior report, the activity status of the participant during the meeting is analyzed. For example, the activity analysis may show that the participant is active during the meeting and provide data support for meeting decision-making. By deeply exploring these data, we can provide important references for subsequent meeting arrangements, participants' focus and interactivity.

[0025] S3 receives the acoustic features of the participant's language, converts the participant's voice signal into text, and combines natural language processing to analyze the semantics and sentiment of the text to obtain the participant's language expression intention.

[0026] Specifically, it includes: obtaining the voice signals of the participants according to the voice signals and extracting the acoustic features therein; converting the voice signals into text by using the voice recognition model through the acoustic features to obtain the recognition results; analyzing the semantic information of the text through natural language processing technology to obtain semantic understanding; judging the emotional features contained in the text based on the semantic understanding to obtain the sentiment analysis results; determining the language expression intention of the participants through the sentiment analysis results combined with the semantic understanding; if the language expression intention is clear, selecting a suitable interaction method according to the expression intention; after obtaining the interaction method, further optimizing the interaction method selection in combination with the sentiment analysis results; judging whether it meets the business goals and improving the communication effect through the optimized interaction method; and outputting the interaction plan if the interaction method meets the requirements.

[0027] Exemplarily, when acquiring the voice signal of the participant, microphone array technology can be used to improve the signal-to-noise ratio of the target voice through a beamforming algorithm so as to more accurately capture the voice signal of the participant. Afterwards, the voice signal is converted into a feature vector using acoustic feature extraction methods such as MFCC (Mel Frequency Cepstral Coefficient) to provide basic data for the subsequent voice recognition process. In the voice recognition stage, a hybrid model of a deep neural network (DNN) combined with a hidden Markov model (HMM) is used to map the acoustic features to text. In this process, a certain confidence threshold (e.g., 80%) can be set to filter out low-confidence recognition results to improve recognition accuracy. After completing text recognition, the natural language processing stage is entered. Using pre-trained language models such as BERT, the syntactic structure and semantic information of the text are analyzed to extract key topics and intentions. Next, the text is sentimentally classified using sentiment analysis algorithms such as LSTM (Long Short-Term Memory Network) or sentiment dictionary methods to determine the emotional state of the participant, such as positive, neutral, or negative. Assuming that the confidence level of sentiment analysis is above 85%, the analysis result is considered reliable. Subsequently, based on the semantic understanding of the text and the sentiment analysis results, the language expression intention of the participants is inferred. For example, when it is recognized that the user expresses an attitude of approval for a proposal, the system can choose a suitable interaction method, such as providing further detailed information or asking if there are other opinions. Through the results of sentiment analysis, the emotional tendencies of the participants are judged and the interaction method is optimized. If the participant's emotions are positive, positive interactive language is used. If the emotions are negative, the tone is adjusted appropriately to calm the emotions. Finally, evaluate whether the optimized interaction method is consistent with the business goals. Through A / B testing or user feedback system, the effect of the interaction plan is tested to ensure that it meets expectations. If the plan meets the needs, the final interaction plan is generated and applied to actual scenarios to improve the communication effect.

[0028] S4, comprehensively analyzes the identity, action and language intentions of the participants, determines the interaction intention of each participant according to the preset interaction intention judgment rules, and divides them into different interaction intention categories.

[0029] Specifically, the following steps are taken: by obtaining the identity information of the participants, including position, department, level and relationship with other participants, and combining it with database records to obtain preliminary judgment results on the identity attributes; based on the judgment results, cross-validation is performed with historical data to improve the accuracy of the identity attributes; by capturing and analyzing the body language, facial expressions, eye contact, sitting and standing postures of the participants, image processing technology is used to judge the action attributes; if a significant difference is found between the action and the identity attributes, the credibility of the action attributes needs to be re-evaluated; voice data is obtained based on voice recognition technology, and the language intent attributes are judged by analyzing the tone of voice, word choice and expression content of the participants; if the language intent is inconsistent with the body language, the credibility of the action attributes needs to be re-evaluated. If there is a contradiction between the identity attribute and the action attribute, it is necessary to adjust the weight of the language intention to ensure logical consistency; comprehensively analyze the three attributes of identity, action and language intention, apply the preset weights, and calculate the comprehensive score of each participant; if the comprehensive score exceeds a certain threshold, enter the next judgment logic; by comparing the comprehensive score with the thresholds of different interaction intention categories, judge the participant's preliminary interaction intention; if the preliminary interaction intention does not match the historical data, readjust the threshold setting to obtain a more accurate judgment based on the judgment result of the preliminary interaction intention, match it with the preset intention category to obtain the closest interaction intention category; if the match is insufficient, it is necessary to adjust the preset intention parameters and make a new judgment.

[0030] For example, when obtaining the identity information of the participants, an identity recognition algorithm can be designed, which combines machine learning models such as support vector machines (SVM) or decision trees to process data such as positions, departments, and levels. These data can be cross-validated with the historical records of the participants to form a preliminary judgment of the identity attributes. For the analysis of body language and facial expressions, convolutional neural networks (CNNs) in deep learning can be used to identify micro-expressions and posture changes. For example, if the system detects that a participant nods and smiles frequently during a meeting, it may be consistent with his identity attribute as a mid-level manager. If these actions are inconsistent with the identity attributes, such as an executive showing impatient body language, the system may need to recalibrate the parameters of the action recognition model to improve the reliability of the judgment. Using speech recognition technology, the system can analyze speech data through natural language processing (NLP) tools to perform detailed analysis of tone, vocabulary, and expression content. Suppose a participant uses words such as suggestions and promotions many times when speaking, with an upward tone, which may indicate that he has a positive attitude towards the project. However, if this language intention is inconsistent with its identity and action attributes (such as a lower position or indifferent body language), the system will maintain the consistency of judgment by adjusting the weight of the language intention. By comprehensively analyzing the identity, action and language intention attributes, the system can apply a weighted average algorithm to assign different weights to each attribute (for example, identity 40%, action 30%, language intention 30%) to calculate the comprehensive score of the participant. Assume that a participant's comprehensive score is 85, which exceeds the threshold of 80 for entering the next judgment logic. The system then compares this score with the thresholds of different interaction intention categories to preliminarily judge that its interaction intention is cooperation. If this preliminary intention does not match the historical data, for example, historically the participant has mostly shown a wait-and-see attitude, the system may need to adjust the threshold setting to improve accuracy. When confirming the stability of the interaction intention, the system can monitor it through real-time data. If the score of a participant fluctuates greatly in different sessions, the accuracy of the data processing algorithm needs to be evaluated. Finally, by analyzing the interaction intentions of all participants, the system generates an overall interaction pattern report. If the report finds that a participant's interaction intention conflicts with other participants, the system will conduct a conflict analysis, such as analyzing interpersonal relationships through a graph database, and then adjust the overall interaction strategy. If there are data anomalies in the final report, the system will trace back to the data source and use automated log analysis tools to recheck the data flow to ensure the accuracy and consistency of the report.

[0031] S5, for different interaction intent categories, dynamically selects the most matching interaction method from the pre-designed conference interaction method library, and adjusts the audio and video codec parameters and interaction interface layout in real time.

[0032] Specifically, it includes obtaining the audio and video input of the participants, analyzing the behavior and voice of the participants in real time through the intention recognition algorithm, and judging and obtaining the current interaction intention category; according to the current interaction intention category, obtaining the interaction method template that meets the category from the preset interaction method library, and obtaining the preliminary interaction method matching result; real-time monitoring of the network quality, comparing the network status with the preliminary interaction method matching result, and judging the adaptability of the audio and video parameters; if the network condition is not good, then adjusting the audio and video encoding method, bit rate, resolution and other parameters to adapt to the current network quality, and obtaining the optimized audio and video parameters; according to the optimized audio and video parameters, determining the appropriate interface layout plan, including elements such as the video window size and position, shared content area, and interactive personnel position.

[0033] For example, in a video conference, the audio and video input of the participants is first collected through a microphone and a camera, and a deep learning model such as a convolutional neural network (CNN) is used for behavior recognition. The voice content is analyzed in combination with a natural language processing algorithm (such as BERT) to identify the interaction intention category of the participants, such as asking questions, discussing, or sharing opinions. Then, according to the identified interaction intention category, a suitable interaction method template is selected from the preset interaction method library. For example, for discussion intentions, an interaction method based on group discussion can be selected. Next, by monitoring the current network quality indicators such as packet loss rate, delay, and bandwidth, an adaptive bitrate algorithm (ABR) is used to adjust the audio and video encoding method, bit rate, and resolution. For example, when the network conditions are poor, the video resolution is reduced to 720p to ensure smoothness. According to the optimized audio and video parameters, determine the appropriate interface layout scheme, use layout algorithms such as force-directed graph layout, dynamically adjust the size and position of the video window, and set the size of the shared content area to adapt to the current network environment and device performance. Further, after determining the interactive personnel and positions, according to the interactive method, if two people are interacting, adjust the positions of the two people to face each other, directly observe each other's face, body language and tone, better understand each other's emotions and intentions, and quickly get each other's response and feedback in face-to-face communication, so that the communication is more timely and efficient, and more targeted. If multiple people communicate together, a single person takes turns to speak, and a circular interaction is used, and the communication is more concentrated in the area. If the interactive method is a leader's speech, a podium-style interaction is used, which can deliver information to many listeners in a concentrated manner, with high efficiency. At the same time, if the device performance is low during the interaction process, the rendering complexity is appropriately reduced according to the CPU and GPU usage of the device, such as by reducing the frame rate to 15fps to ensure smooth operation of the interface. In addition, during the interaction, feedback data from participants will be collected in real time, and sentiment analysis algorithms (such as VADER) will be used to evaluate the interactive experience and determine the rationality of the interactive method. The frequency and intensity of the interaction will be adjusted based on the evaluation results. For example, when the number of participants increases, the frequency of interaction will be reduced to maintain order.

[0034] Specifically, during the meeting, by monitoring the meeting duration and participant feedback, the interaction mode and interface layout parameters can be dynamically adjusted, such as increasing the interaction interval in the middle and late stages of the meeting to reduce participant fatigue. In this way, the interaction mode of the meeting can be continuously optimized under different network conditions, device performance and meeting scale, ensuring a good experience for all participants.

[0035] Furthermore, during the interactive process of the conference, an adaptive network transmission strategy can be adopted to dynamically adjust the data transmission rate and retransmission mechanism by detecting the network status in real time, and combining redundant coding to maintain low-latency transmission of interactive data.

[0036] In order to synchronize the interactive data between multiple participants, distributed synchronization control is adopted, and a global clock and timestamp mechanism is set to synchronize the interactive data between different devices. Through prediction and interpolation, data inconsistencies caused by differences in device performance are compensated.

[0037] A conference interaction system based on the metaverse is applicable to the above method.

[0038] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A conference interaction method based on the metaverse, characterized in that: The following steps are involved: S1, using face recognition, extracting the facial features of the participants and comparing them with the pre-established face feature database to obtain the unique identity of each participant and identify the participants; S2, after confirming the identity of the participant, detect the position changes of the participant's key body joints in real time, and judge the specific actions of the participant by analyzing the relative positions and motion trajectories between the joints; S3, receiving the acoustic features of the participant's language, converting the participant's voice signal into text, and combining natural language processing to analyze the semantics and sentiment of the text to obtain the participant's language expression intention; S4, comprehensively analyzing the identity, action and language intention of the participants, determining the interaction intention of each participant according to the preset interaction intention judgment rules, and dividing them into different interaction intention categories; S5, for different interaction intent categories, dynamically selects the most matching interaction method from the pre-designed conference interaction method library, and adjusts the audio and video codec parameters and interaction interface layout in real time.

2. A conference interaction method based on the metaverse according to claim 1, characterized in that: It also includes, during the conference interaction process, the use of adaptive network transmission strategies, real-time detection of network status, dynamic adjustment of data transmission rate and retransmission mechanism, and combination of redundant coding to maintain low-latency transmission of interactive data.

3. According to the metaverse-based conference interaction method of claim 1, it is characterized in that: It also includes the use of distributed synchronization control for the synchronization of interactive data between multiple participants, setting up a global clock and timestamp mechanism, synchronizing the interactive data between different devices, and compensating for data inconsistencies caused by differences in device performance through prediction and interpolation.

4. A conference interaction method based on metaverse according to claim 1, characterized in that: S1 specifically includes: locating and marking the face area from the input image or video through image processing to obtain face area data; if the face area data exists, adjusting the detected face area to a standard posture and size through affine transformation to ensure face alignment and obtain an aligned face image; extracting feature vectors that can represent the identity through the aligned face image using feature extraction, including Gabor wavelet, to obtain a face feature vector; judging whether the face feature vector is complete, if complete, calculating the similarity between the face feature vector and a pre-established face feature library of persons with known identities to obtain a similarity score; judging the most similar feature vector based on the similarity score by comparing the similarity with each feature vector in the face feature library to obtain a comparison result; obtaining an identity identifier corresponding to the highest similarity from the comparison result to obtain a unique identity identifier of the participant; judging whether the unique identity identifier is valid, if valid, updating the identity identifier to the identity recognition record system to complete the identity recognition record.

5. The method for conference interaction based on metaverse according to claim 1, characterized in that: S2 specifically includes: if the identity information of the participant has been obtained, the position changes of the key body joints of the participant are captured in real time through the skeleton tracking algorithm to obtain the initial joint coordinate set; according to the initial joint coordinate set and the time series, the position change of each joint is calculated to obtain the motion trajectory data of the joint; if the motion trajectory data of all relevant nodes are obtained, the relative position changes between the joints are analyzed to determine the relative motion angles of each joint; if the relative motion angles of the joints have been determined, they are compared with the predefined motion pattern library to determine the specific action category of the participant; by determining the specific action category, the starting time point of the action is marked, and the end time point of the action is determined according to the subsequent motion changes; if the starting and ending time points of the action have been determined, the time duration of the action is calculated according to the time series; After obtaining the duration of the action, the number of all actions of the current participant is counted in combination with the previous action information; if the number of actions is counted, the action behavior report of the participant is formed through the updated statistical data and action information; based on the action behavior report, the activity status of the participant during the meeting is output to provide data support for subsequent analysis and decision-making.

6. A conference interaction method based on metaverse according to claim 1, characterized in that: S3 specifically includes obtaining a voice signal of a participant according to a voice signal and extracting acoustic features therein; Through acoustic features, a speech recognition model is used to convert speech signals into text to obtain recognition results; the semantic information of the text is analyzed through natural language processing technology to obtain semantic understanding; based on the semantic understanding, the emotional features contained in the text are judged to obtain the emotional analysis results; Through the results of sentiment analysis combined with semantic understanding, the language expression intention of the participants is determined; if the language expression intention is clear, the appropriate interaction method is selected according to the expression intention; after obtaining the interaction method, combined with the sentiment analysis results, the interaction method selection is further optimized; through the optimized interaction method, it is judged whether it meets the business goals and improves the communication effect; if the interaction method meets the requirements, the interaction plan is output.

7. A conference interaction method based on metaverse according to claim 1, characterized in that: S4 specifically includes: obtaining the identity information of the participants, including position, department, level and relationship with other participants, and combining it with database records to obtain preliminary judgment results on the identity attributes; based on the judgment results, cross-validating with historical data to improve the accuracy of the identity attributes; capturing and analyzing the body language, facial expressions, eye contact, sitting and standing postures of the participants, using image processing technology to judge the action attributes; if it is found that there is a significant difference between the action and the identity attributes, it is necessary to re-evaluate the credibility of the action attributes; obtaining voice data based on voice recognition technology, and judging the language intention attributes by analyzing the tone, word choice and expression content of the participants; if the language intention is inconsistent with the identity attributes and action attributes, it is necessary to adjust the weight of the language intention to ensure logical consistency; comprehensively analyzing the three attributes of identity, action and language intention, applying the preset weights, and calculating the comprehensive score of each participant; If the comprehensive score exceeds a certain threshold, the next step of judgment logic will be entered; By comparing the comprehensive score with the thresholds of different interaction intention categories, the initial interaction intention of the participants was determined; If the preliminary interaction intention does not match the historical data, the threshold setting is readjusted to obtain a more accurate judgment. According to the judgment result of the preliminary interaction intention, it is matched with the preset intention category to obtain the closest interaction intention category; If the match is not sufficient, you need to adjust the preset intent parameters and make a new judgment.

8. The method for conference interaction based on metaverse according to claim 1, characterized in that: S5 specifically includes obtaining the audio and video input of the participants, analyzing the behaviors and voices of the participants in real time through the intention recognition algorithm, and determining and obtaining the current interaction intention category; According to the current interaction intention category, an interaction mode template matching the category is obtained from a preset interaction mode library to obtain a preliminary interaction mode matching result; Monitor network quality in real time, compare network status with preliminary interaction mode matching results, and determine the adaptability of audio and video parameters; If the network condition is not good, then adjust the audio and video encoding method, bit rate, resolution and other parameters to adapt to the current network quality and obtain optimized audio and video parameters; based on the optimized audio and video parameters, determine the appropriate interface layout plan, including elements such as the video window size and position, shared content area, and interactive personnel position.

9. A conference interactive system based on the metaverse, characterized in that: Applicable to the method according to any one of claims 1 to 8.

Images