Face optimization conference image display method and system based on multiple video streams

Abstract

The invention provides a face optimization conference image display method and system based on multiple video streams, and the method comprises the steps: obtaining a plurality of original video streams, carrying out the preprocessing of all original video streams, and obtaining an optimized video stream; extracting a video frame of each optimized video stream, and performing face detection and human body posture analysis on the video frames to obtain marked face information and marked data of body orientation; performing feature coding on each piece of labeled data to obtain a plurality of high-dimensional feature vectors, calculating feature similarities among different optimized video streams based on the high-dimensional feature vectors, deleting repeated participant pictures of which the feature similarities are higher than a threshold value, and obtaining participant video pictures after duplicate removal; and carrying out image layout adjustment on the video pictures of the participants based on a dynamic layout algorithm, and outputting the conference picture after layout adjustment. The method can prevent the same participant from repeatedly appearing in a conference interface, reduces visual interference, improves the identity recognition accuracy, and remarkably improves the conference efficiency and experience.

Description

Technical Field

[0001] This invention relates to the field of video stream processing technology, and in particular to a method and system for optimizing conference image display based on multiple video streams. Background Technology

[0002] Existing video conferencing systems have significant technical shortcomings when dealing with multi-device / multi-angle access scenarios: Traditional facial recognition algorithms rely too heavily on facial feature point matching. When the same user joins the meeting through different devices or by significantly changing their body posture, factors such as camera angle, lighting conditions, and facial occlusion can cause feature extraction deviations. As a result, the system has difficulty establishing cross-device identity associations, causing multiple video streams of the same user to be misjudged as independent individuals, resulting in repeated display on the interface and seriously affecting the smoothness of the meeting and the user experience. Summary of the Invention

[0003] This invention provides a method and system for optimizing conference image display based on multiple video streams to solve the problems existing in related technologies. The technical solution is as follows:

[0004] In a first aspect, embodiments of the present invention provide a method for optimizing conference image display based on multiple video streams, including:

[0005] Multiple raw video streams are acquired, and all raw video streams are preprocessed to obtain an optimized video stream;

[0006] Extract video frames from each optimized video stream, perform face detection and human pose analysis on the video frames, and obtain labeled face information and body orientation label data;

[0007] Each labeled data is feature-encoded to obtain multiple high-dimensional feature vectors. Based on the high-dimensional feature vectors, the feature similarity between different optimized video streams is calculated. Duplicate participant images with feature similarity higher than the threshold are deleted to obtain deduplicated participant video images.

[0008] The image layout of the participants' video screens is adjusted based on a dynamic layout algorithm, and the adjusted meeting screen is output.

[0009] In one implementation, preprocessing all raw video streams includes:

[0010] All raw video streams are synchronized using timestamps to obtain time-synchronized video streams.

[0011] The time-synchronized video stream is optimized to obtain an optimized video stream; the preprocessing includes frame rate adjustment, resolution adjustment and noise filtering.

[0012] In one implementation, face detection and human pose analysis are performed on video frames to obtain labeled face information and body orientation annotation data, including:

[0013] A convolutional neural network is used to detect faces in each video frame, obtain face information, and annotate it in the video frame;

[0014] The positions of specified key points on the participant's body are determined by a pose estimation algorithm. A pose vector is generated based on the positions of the specified key points. The body orientation is determined based on the pose vector using a preset model and then labeled in the video frame.

[0015] In one implementation, it further includes:

[0016] If it is determined that a participant's body is not facing correctly, delete the participant's video feed and send a posture adjustment reminder message to the participant's terminal.

[0017] In one implementation, adjusting the image layout of participants' video feeds based on a dynamic layout algorithm includes:

[0018] Obtain the meeting role of each participant and determine the priority of each participant based on their meeting role;

[0019] Determine the current speaking status of each participant based on their video feed;

[0020] The image layout of the meeting screen is dynamically adjusted based on the current speaking status and the priority order of each participant.

[0021] In one implementation, it further includes:

[0022] Acquire eye-tracking data for each participant and determine the participant's gaze angle toward the center of the screen based on the eye-tracking data;

[0023] When a participant's line of sight exceeds a preset angle, the video image of the participant is corrected to obtain an augmented reality meeting image, and a posture adjustment reminder message is sent to the participant's terminal.

[0024] In one implementation, it further includes:

[0025] The background of the augmented reality meeting screen is optimized to obtain a blurred background image;

[0026] The blurred background image is compressed before being transmitted to the designated terminal device for final display.

[0027] Secondly, embodiments of the present invention provide a face-optimized conference image display system based on multiple video streams, comprising:

[0028] The video processing module acquires multiple raw video streams, preprocesses all raw video streams, and obtains an optimized video stream.

[0029] The image annotation module extracts video frames from each optimized video stream, performs face detection and human pose analysis on the video frames, and obtains annotated face information and body orientation annotation data.

[0030] The user deduplication module is used to encode the features of each labeled data to obtain multiple high-dimensional feature vectors. Based on the high-dimensional feature vectors, the feature similarity between different optimized video streams is calculated, and duplicate participant images with feature similarity higher than the threshold are deleted to obtain deduplicated participant video images.

[0031] The layout adjustment module is used to adjust the image layout of the participants' video screens based on a dynamic layout algorithm and output the adjusted meeting screen.

[0032] Thirdly, embodiments of the present invention provide an electronic device comprising a memory and a processor. The memory and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory. When the processor executes the instructions stored in the memory, it causes the processor to perform the method described in any of the above embodiments.

[0033] Fourthly, embodiments of the present invention provide a computer-readable storage medium that stores a computer program, wherein when the computer program is run on a computer, the methods in any of the embodiments described above are executed.

[0034] The advantages or beneficial effects of the above technical solutions include at least the following:

[0035] This invention preprocesses multiple original video streams, then performs face detection and human pose analysis on the video frames of the optimized video stream to obtain corresponding high-dimensional feature vectors. Based on these high-dimensional feature vectors, it calculates the feature similarity between different video streams to determine whether the same participant is displayed repeatedly. Duplicate participant images with feature similarity exceeding a threshold are deleted, and the remaining participant video images after deduplication are dynamically stitched together for display. This avoids the same participant appearing repeatedly on the meeting interface, reduces visual interference, improves the accuracy of identity recognition, and significantly enhances meeting efficiency and experience.

[0036] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0037] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in the invention and should not be construed as limiting the scope of the invention.

[0038] Figure 1 This is a flowchart illustrating the method for optimizing conference image display based on multiple video streams according to the present invention.

[0039] Figure 2 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0040] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0041] Example 1

[0042] Existing video conferencing systems generally employ a fixed grid layout, a design with several limitations in practical applications: First, due to the lack of an intelligent video quality assessment mechanism, significant differences in image clarity and lighting conditions among different users lead to inconsistent overall viewing experiences. Second, the system struggles to identify when the same user connects via multiple devices or cameras from multiple angles, resulting in the inefficient display of the same participant repeatedly on the interface. Third, the static grid arrangement cannot dynamically adjust based on participant activity or role, leading to a scattered visual focus and low space utilization. Furthermore, the system typically employs a full-stream video transmission strategy, which is prone to stuttering and latency issues in bandwidth-constrained environments. To address these pain points, there is an urgent need to introduce intelligent multi-stream video analysis technology to achieve a comprehensive improvement in the meeting experience through real-time quality assessment, user deduplication, and dynamic image optimization.

[0043] To address the aforementioned issues, this embodiment provides a method for optimizing conference image display based on multiple video streams, such as... Figure 1 As shown, the specific methods include:

[0044] Step S1: Obtain multiple raw video streams, preprocess all raw video streams to obtain optimized video streams.

[0045] This embodiment can acquire multiple raw video streams in real time from terminal devices such as PCs, smartphones, and conference room cameras. The raw video streams support different resolutions (such as 1080p / 720p) and frame rates (such as 30fps / 15fps). The video encoding format of the multiple raw video streams is unified (such as H.264 / H.265) to ensure compatibility.

[0046] During the acquisition of the raw video stream, a high-precision timestamp is embedded in each frame. All raw video streams are synchronized and calibrated using the timestamps. Multi-source video stream synchronization can be performed based on the NTP protocol or PTS (Presentation Time Stamp) to ensure time alignment, thereby obtaining a time-synchronized video stream.

[0047] Subsequently, the time-synchronized video stream undergoes preprocessing, including frame rate optimization, resolution optimization, and noise suppression. Frame rate optimization aims to balance smoothness and conserve computational resources. Resolution optimization (1080p→720p) reduces bandwidth usage and improves processing speed. Noise suppression employs Gaussian filtering or adaptive noise reduction algorithms to minimize interference from lighting changes and motion blur.

[0048] After the above preprocessing, multiple optimized video streams with synchronized timing are output. These optimized video streams are time-aligned and are lightweight and low-noise, which facilitates subsequent analysis.

[0049] Step S2: Extract video frames from each optimized video stream, perform face detection and human pose analysis on the video frames, and obtain labeled face information and body orientation data.

[0050] In this embodiment, a convolutional neural network is used to perform face detection on each video frame of the optimized video stream, determine the facial features, face orientation, and other facial information of the participants, and annotate them in the video frame.

[0051] If facial information indicates that a participant is in a side-facing or head-down posture, the participant's video feed will be deleted to ensure that the participant's face is clearly visible in the video.

[0052] Simultaneously, human pose analysis is performed on each video frame of the optimized video stream. This involves using a pose estimation algorithm (MediaPipe Pose or OpenPose) to detect 17-33 designated key points of the participant (such as head, shoulders, torso, elbows, hips, knees, etc.) and locating the position of each key point. The spatial orientation vector of the participant is calculated using the designated key points to obtain the pose vector. The direction the participant is facing is determined based on the pose vector, which helps to solve the problem of repeated viewpoints in multi-camera / multi-device scenarios.

[0053] The posture vector includes a shoulder line vector representing a spatial vector pointing from the left shoulder to the right shoulder, reflecting the horizontal orientation of the upper body. For example:

[0054] If the participants are facing forward, the vector direction is horizontal to the right (assuming the X-axis of the coordinate system is to the right).

[0055] If a participant turns to the side, the vector direction will tilt (forming an angle with the X-axis).

[0056] The posture vector also includes the spine angle, calculated by connecting the midpoint of the hip and the midpoint of the shoulder to determine the spine tilt angle (relative to the vertical axis).

[0057] Based on a preset model, the body orientation is determined according to the posture vector, thereby determining whether the participant's body is in an upright position, and this is marked in the video frame. At this time, the video frame records facial information and body orientation annotation data, so as to determine whether there are duplicate participants based on the annotation data later.

[0058] The preset model in this embodiment judges whether the body is upright based on the following three core conditions:

[0059] 1. Shoulder balance detection: This involves comparing the y-coordinate difference between the left shoulder (LS) and the right shoulder (RS) to obtain the height difference between the left and right shoulders. When the height difference is less than a set threshold (e.g., 20 pixels), it indicates that the body is not significantly tilted to one side.

[0060] 2. Trunk verticality test: The angle between the spine's central axis and the vertical direction is calculated by connecting the midpoint of the shoulder and the midpoint of the hip to form a spine vector. If the angle is within the allowable range, such as less than ±20 degrees, it indicates that the body is upright and not tilted.

[0061] 3. Consistency between face and torso directions: The angle between the face and torso directions should be less than 30 degrees. When the two directions are basically consistent, it indicates that the posture is natural and coordinated. This condition can eliminate misjudgment in special scenarios such as "turning the head to talk".

[0062] These three conditions together constitute the complete posture uprightness assessment system of the preset model. According to the analysis of the preset model, only when all three conditions are met can the body be judged to be in an upright state.

[0063] If the detection determines that a participant's body is not facing correctly, the video feed of the participant with the incorrect posture will be deleted, and a posture adjustment reminder message will be sent to the participant's terminal to remind them to adjust to the correct posture.

[0064] This embodiment removes video footage of participants with abnormal head postures (such as turning their heads to the side or looking down) and improper body postures. This can compensate for omissions caused by multi-angle interference and avoid situations where a person's face is facing the camera but their body is turned to the side, resulting in a strange overall image effect ("distorted posture").

[0065] Step S3: Perform feature encoding on each labeled data to obtain multiple high-dimensional feature vectors. Calculate the feature similarity between different optimized video streams based on the high-dimensional feature vectors. Delete duplicate participant images with feature similarity higher than the threshold to obtain deduplicated participant video images.

[0066] Since the same user joins the meeting from multiple devices, significant differences in body posture could lead to misidentification as different faces. Therefore, this embodiment performs feature encoding on the labeled data in the video frames to obtain a high-dimensional feature vector. This vector retains information features in the dimensions of the participant's face and body posture. Specifically:

[0067] The FaceNet / ArcFace model is used to extract a 512-dimensional face feature vector. The body / face orientation angle is converted into a unit direction vector (3-dimensional). The time difference and camera position are encoded into a 4-dimensional auxiliary vector. The face features, pose features and spatiotemporal features are concatenated to generate a multi-dimensional hybrid feature vector.

[0068] In this embodiment, the calculation of feature similarity includes:

[0069] Face similarity: Calculate the cosine similarity of face feature vectors (face threshold 0.6-0.8);

[0070] Pose consistency: Check the angle difference between the body / face orientation vectors (pose threshold < 30°).

[0071] Video frames at the same time point are extracted from different optimized video streams. The similarity of faces and poses in different video frames is compared. If the face similarity between different video frames is greater than a preset face threshold and the angle difference between the body / face orientation vectors is less than a preset pose threshold, then the participants shown in different video frames can be determined to be the same participant. If either the face similarity or pose consistency condition is not met, then the participant is retained as an independent participant.

[0072] When the same participant is identified, duplicate participants are deduplicated by automatically hiding duplicate images. The deduplication rules can be to prioritize video streams with higher resolution / frame rate, and when the quality is the same, select the image with the most standard frontal orientation.

[0073] Step S4: Adjust the image layout of the participants' video screens based on the dynamic layout algorithm, and output the adjusted meeting screen.

[0074] Each participant's identity is verified using facial recognition data. Based on this identity, each participant's meeting role is determined, and their priority is assigned accordingly. Meeting roles include moderator, speaker, guest speaker, and general participant.

[0075] The current speaking status of each participant is determined based on their video feed, and the image layout of the meeting screen is dynamically adjusted according to the current speaking status and the priority order of each participant.

[0076] For example:

[0077] If the host is not speaking, their screen will remain displayed in a small window in the corner; the current speaker's window will be enlarged and placed on the main screen; if the host is speaking, their video display will have dual priority (role + speaking), and the host's window will be placed in the center.

[0078] In addition, network status can be considered, i.e., real-time monitoring of the uplink bandwidth of each participant (such as WebRTC's REM report), calculation of key indicators, and determination of network status based on these indicators. For example, a packet loss rate >5% is considered a poor network status, latency >300ms is considered a poor network status, and jitter >50ms is considered a poor network status. Participants with poor network conditions should have their screens displayed in smaller sizes to reduce resource consumption.

[0079] Similarly, the video quality of different video streams is identified, and videos with poor quality are prioritized for reduction or hiding.

[0080] During the meeting, eye movement data (including pupil position, fixation point coordinates, etc.) of each participant can be obtained. Based on the eye movement data, the angle of the participant's gaze toward the center of the screen can be determined. Specifically, the angle θ between the gaze vector and the normal to the center of the screen is calculated through three-dimensional spatial geometry. When θ exceeds a preset threshold (such as ±30° in the horizontal direction or ±20° in the vertical direction), it is determined to be an attention deviance state.

[0081] When a participant's gaze angle exceeds a preset angle, a Generative Adversarial Network (GAN) is used to intelligently adjust and correct the participant's video feed, resulting in an augmented reality meeting image. Based on GAN, a virtual frontal image is generated using the current profile image and the target's frontal angle parameters as input. During training, the network learns the texture mapping rules of faces from multiple angles, automatically filling in occluded areas such as cheekbones and ears, while maintaining consistency in skin color and expression.

[0082] At the same time, a posture adjustment reminder message is sent to the terminal of the participant whose attention has wandered, prompting the participant to "face the screen".

[0083] In addition, image data can be compressed and optimized to obtain a low-bandwidth optimized augmented reality conference image. The initialized conference image can be used for final presentation on the terminal device. Specifically, background optimization (virtual background / blurred background) is used to obtain a blurred background image; H.265 / AV1 encoding is used to reduce bandwidth, and the blurred background image is compressed before being transmitted to the designated terminal device for final presentation.

[0084] Example 2

[0085] This embodiment provides a face-optimized conference image display system based on multiple video streams. The system executes the face-optimized conference image display method based on multiple video streams as described in Embodiment 1. The system includes:

[0086] The video processing module acquires multiple raw video streams, preprocesses all raw video streams, and obtains an optimized video stream.

[0087] The image annotation module extracts video frames from each optimized video stream, performs face detection and human pose analysis on the video frames, and obtains annotated face information and body orientation annotation data.

[0088] The user deduplication module is used to encode the features of each labeled data to obtain multiple high-dimensional feature vectors. Based on the high-dimensional feature vectors, the feature similarity between different optimized video streams is calculated, and duplicate participant images with feature similarity higher than the threshold are deleted to obtain deduplicated participant video images.

[0089] The layout adjustment module is used to adjust the image layout of the participants' video screens based on a dynamic layout algorithm and output the adjusted meeting screen.

[0090] This embodiment preprocesses multiple original video streams, then performs face detection and human pose analysis on the video frames of the optimized video stream obtained from the preprocessing to obtain corresponding high-dimensional feature vectors. Based on the high-dimensional feature vectors, the feature similarity between different video streams is calculated to determine whether the same participant is displayed repeatedly. The duplicate participant images with feature similarity higher than the threshold are deleted, and the remaining participant video images after deduplication are dynamically stitched together for display. This can avoid the same participant appearing repeatedly on the meeting interface, reduce visual interference, improve the accuracy of identity recognition, and significantly improve meeting efficiency and experience.

[0091] It should be noted that the functions of each module in the system of this embodiment can be found in the corresponding descriptions in the above methods, and will not be repeated here.

[0092] Example 3

[0093] This embodiment provides an electronic device. Figure 2 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 2 As shown, the electronic device includes a memory 100 and a processor 200. The memory 100 stores a computer program that can run on the processor 200. When the processor 200 executes the computer program, it implements the face optimization conference image display method based on multiple video streams in the above embodiments. The number of memories 100 and processors 200 can be one or more.

[0094] The electronic device also includes:

[0095] The communication interface 300 is used to communicate with external devices and perform data exchange and transmission.

[0096] If the memory 100, processor 200, and communication interface 300 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc.

[0097] Optionally, in a specific implementation, if the memory 100, processor 200, and communication interface 300 are integrated on a single chip, then the memory 100, processor 200, and communication interface 300 can communicate with each other through an internal interface.

[0098] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this invention.

[0099] This invention also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this invention.

[0100] This invention also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this invention.

[0101] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0102] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0103] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0104] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0105] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0106] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for optimizing face display in conferences based on multiple video streams, characterized in that, include: Multiple raw video streams are acquired, and all of the raw video streams are preprocessed to obtain an optimized video stream; Extract video frames from each of the optimized video streams, perform face detection and human pose analysis on the video frames, and obtain labeled face information and body orientation label data. Each of the labeled data is feature-encoded to obtain multiple high-dimensional feature vectors. Based on the high-dimensional feature vectors, the feature similarity between different optimized video streams is calculated. Duplicate participant images with feature similarity higher than a threshold are deleted to obtain deduplicated participant video images. The image layout of the participants' video screens is adjusted based on a dynamic layout algorithm, and the adjusted meeting screen is output.

2. The method for optimizing conference image display based on multiple video streams according to claim 1, characterized in that, The preprocessing of all the original video streams includes: All the original video streams are synchronized using timestamps to obtain time-synchronized video streams. The time-synchronized video stream is optimized to obtain the optimized video stream; wherein, the preprocessing includes frame rate adjustment, resolution adjustment and noise filtering.

3. The method for optimizing conference image display based on multiple video streams according to claim 1, characterized in that, The step of performing face detection and human pose analysis on the video frames to obtain labeled face information and body orientation annotation data includes: A convolutional neural network is used to perform face detection on each video frame to obtain face information and label it in the video frame; The positions of specified key points on the participant's body are determined by a pose estimation algorithm. A pose vector is generated based on the specified key point positions. The body orientation is determined based on the pose vector using a preset model and then marked in the video frame.

4. The method for optimizing conference image display based on multiple video streams according to claim 1, characterized in that, Also includes: If, based on the body orientation, it is determined that the participant's body orientation is incorrect, the participant's video feed will be deleted, and a posture adjustment reminder message will be sent to the participant's terminal.

5. The method for optimizing conference image display based on multiple video streams according to claim 1, characterized in that, The image layout adjustment of the participants' video screens based on the dynamic layout algorithm includes: Obtain the meeting role of each participant, and determine the priority of each participant based on the meeting role; Determine the current speaking status of each participant based on their video feed; The image layout of the meeting screen is dynamically adjusted based on the current speaking status and the priority order of each participant.

6. The method for optimizing conference image display based on multiple video streams according to claim 1, characterized in that, Also includes: Acquire eye movement data for each participant, and determine the participant's gaze angle toward the center of the screen based on the eye movement data; When a participant's line of sight exceeds a preset angle, the video image of the participant is corrected to obtain an augmented reality meeting image, and a posture adjustment reminder message is sent to the participant's terminal.

7. The method for optimizing conference image display based on multiple video streams according to claim 6, characterized in that, Also includes: The background of the augmented reality meeting screen is optimized to obtain a blurred background image; The blurred background image is compressed before being transmitted to a designated terminal device for final display.

8. A face-optimized conference image display system based on multiple video streams, characterized in that, include: The video processing module acquires multiple raw video streams and preprocesses all of the raw video streams to obtain an optimized video stream; The image annotation module extracts video frames from each of the optimized video streams, performs face detection and human pose analysis on the video frames, and obtains annotated face information and body orientation annotation data. The user deduplication module is used to encode the features of each of the labeled data to obtain multiple high-dimensional feature vectors, calculate the feature similarity between different optimized video streams based on the high-dimensional feature vectors, delete duplicate participant images with feature similarity higher than a threshold, and obtain deduplicated participant video images. The layout adjustment module is used to adjust the image layout of the participants' video screens based on a dynamic layout algorithm and output the adjusted meeting screen.

9. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the face-optimized conference image display method based on any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the face optimization conference image display method based on any one of claims 1 to 7.

Images