Large-screen video conference window adjusting method based on image processing

Abstract

The application discloses a large-screen video conference window adjusting method based on image processing and relates to the technical field of image processing. vid The window is subjected to image enhancement processing, speech adjustment, and a speaker detection algorithm is used to obtain a priority score S of each window pri The highest score is determined as the speaker, a conference record is kept, a video window is divided into a main window and a plurality of sub-windows through a video window division algorithm, the current speaker can be obviously seen, a priority score S is obtained through a speaker detection algorithm pri The highest score is determined as the speaker, and the speaker is automatically set, manual adjustment is not needed, the current speaker can be intelligently and effectively determined, and the experience effect of the large-screen video conference is increased.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for adjusting large-screen video conferencing windows based on image processing. Background Technology

[0002] Video conferencing is a real-time visual connection between two or more remote parties that simulates a face-to-face meeting. Image stitching refers to merging multiple video streams or image sources in a video conference into a single large screen display. This can increase interaction and communication between participants, improve the efficiency and quality of the meeting, and image stitching technology can combine these video streams into a whole picture to provide a better viewing experience.

[0003] Currently, in video conferencing, all participants' screens are displayed simultaneously. When speaking, it is difficult for participants to notice the specific speaker, making the meeting process rather chaotic. When there are many participants speaking during the meeting, it is necessary to constantly and manually switch the video conferencing client's display screen, making the switching process complicated and unable to intelligently and effectively identify the current speaker. This results in a poor experience for large-screen video conferencing and reduces the quality of the meeting. Summary of the Invention

[0004] The purpose of this invention is to provide a method for adjusting the window of a large-screen video conferencing system based on image processing, which improves the quality of the meeting and solves the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for adjusting a large-screen video conferencing window based on image processing, comprising the following steps:

[0006] Before the meeting, a large-screen display system is set up, which includes a display screen. The system acquires image data of the participants in the video conference and projects it onto the video window of the display screen.

[0007] Screen partitioning uses a video window partitioning algorithm to divide the video window into a main window and several sub-windows. The main window is used to display the speakers participating in the video conference. The main window occupies a larger screen area than the sub-windows. The sum of the number of main windows and sub-windows equals the number of people participating in the video conference.

[0008] Image processing involves using video quality detection algorithms in video conferencing to detect personnel image data in the main and secondary windows, resulting in a window quality score Q. vid And set the window quality score Q. vid The minimum threshold is Y1, when the window quality score Q vid When the value is less than Y1, perform image enhancement processing on the window;

[0009] Speech adjustment, in the process of large-screen video conference, using the speaker detection algorithm to get the priority score S of each window pri , the highest score is determined as the speaker, and the personnel image data is replaced with the personnel image data on the main window, and the quality threshold of the priority score S pri is Y2, when the priority score S pri is less than Y2, the data in the video quality detection algorithm is adjusted;

[0010] Meeting records, using recording software to record the process of video conference, get the meeting data, after the large-screen video conference, store the meeting data and set the label, so as to check later.

[0011] Optionally, the large-screen display system comprises a display screen, a signal processing and transmission device, a control device and an audio device.

[0012] Optionally, the main window size calculation process in the video window division algorithm is as follows:

[0013] W m = R m × W, H m = R m × H

[0014] Where W m is the width of the main window, H m is the height of the main window, and m is the main window;

[0015] R m is the proportion of the main window occupying the video window, and the value range is between 0 and 1;

[0016] W is the total width of the video window, and H is the total height of the video window;

[0017] The size calculation process of the sub-window is as follows:

[0018]

[0019] Where W s is the width of the sub-window, H s is the height of the sub-window, and s is the sub-window;

[0020] N is the total number of video conference;

[0021] N-1 is the number of sub-windows;

[0022] is the number of columns of the sub-window;

[0023] The sub-window arrangement position calculation process is as follows:

[0024]

[0025] wherein X si is the horizontal coordinate of the i-th window;

[0026] Y si is the vertical coordinate of the i-th window;

[0027] is the number of rows of the sub-window;

[0028] The video window division algorithm can automatically generate a suitable conference window layout according to the number of participants in the video conference, so that the display screen video window layout is more neat, and the main window can highlight the speaker.

[0029] Optionally, the video quality detection algorithm in the image processing step is as follows:

[0030]

[0031] wherein Q vid (i) is the i-th window quality score, Q vid is the window quality score;

[0032] W vidi ×H vidi is the resolution of the i-th window, indicating the clarity of the personnel image data;

[0033] W vidi is the width of the i-th window video, W max is the maximum width of the video;

[0034] H vidi is the height of the i-th window video; H max is the maximum height of the video;

[0035] F vidi is the frame rate of the i-th window video, F mzx is the maximum frame rate of the video;

[0036] K r is the resolution weight, taking a value of 0.5;

[0037] K f is the frame rate weight, taking a value of 0.5;

[0038] The i-th window quality score Q vid (i) indicates the clarity and smoothness of the personnel image data, and the window quality score Q vid is set to a minimum threshold Y1, and when the i-th window quality score Q vid (i) is less than Y1, it indicates that the video in the window is blurred and stuck, and the overall quality of the video is low, at which time the image enhancement processing is performed on the video in the window.

[0039] Optionally, the speaker detection algorithm in the speech adjustment step is as follows:

[0040] S pri (i) = P spe (i) × Q vid (i)

[0041] where S pri (i) is the priority score of the ith window, S pri is the priority score;

[0042] S spe (i) is the speech probability of the ith window

[0043] Q vid (i) is the quality score of the ith window;

[0044] Specifically, S pri (i) is derived as follows:

[0045] S pri (i) = S spe,face (i) × β + S spe,audio (i) × γ

[0046] where S spe,face (i) is the ith window based on facial speech probability;

[0047] S spe,audio (i) is the ith window based on audio speech probability;

[0048] β is the facial speech probability weight, ranging from 0.5 to 1;

[0049] γ is the audio speech probability weight, ranging from 0.5 to 1;

[0050] Specifically:

[0051]

[0052] where S spe,face (i) is the ith window based on facial speech probability;

[0053] f mouth,i is the mouth opening frequency of the ith window;

[0054] f spr is the standard mouth opening frequency when speaking;

[0055] α mouth,i is the mouth opening amplitude;

[0056] α max is the maximum mouth opening amplitude;

[0057]

[0058] wherein S spe,audio (i) is the i-th window based audio speaking probability;

[0059] E signal,i is the i-th window audio energy value;

[0060] E th is the audio energy threshold value;

[0061] priority score S pri represents the level of window speaking probability, priority score S pri The highest window will be replaced with the main window, when priority score S pri The highest is the main window, then no change will occur, ensuring the stability of the video conference.

[0062] Optionally, the quality threshold value for setting priority score S pri is Y2, when the priority score S pri of a certain window is less than Y2, it indicates that the current window picture data quality is too low, at this time the window resolution weight K r is increased in the video quality detection algorithm, specifically:

[0063] K r = 0.5 + Z

[0064] When the window resolution weight K r , in order to ensure that K r + K f = 1, at this time K f correspondingly decreases, specifically:

[0065] K f = 0.5 - Z

[0066] Z is the adjustment amount, the value range is 0.1 to 0.4;

[0067] Since the speaker detection algorithm needs to detect the face of the personnel, if the window resolution is too low, it will affect the accuracy of face detection, at this time the priority score S pri of each window, when the priority score S pri is too low, it can affect the data in the video quality detection algorithm, increase the weight of resolution, so that the window quality score Q vid can adjust the specific situation of window quality in real time.

[0068] Optionally, the image enhancement processing includes noise removal, super-resolution reconstruction and frame rate interpolation.

[0069] Optionally, the conference data comprises video data and audio data, wherein the video data comprises pre-image processing data and post-image processing data.

[0070] Compared with the prior art, the present application has the following advantages:

[0071] Firstly, the present application divides the video window into a main window and several sub-windows by a video window division algorithm, the main window is used to display the speaker, and the several sub-windows are evenly distributed in other positions, which can efficiently utilize the screen space, avoid overlapping and waste, and can generate a suitable conference window layout according to the number of participants in the video conference, no matter how many video conference participants are, the interface remains neat and has no crowded feeling, so that the video conference participants can obviously see the current speaker, effectively improve the attention concentration, and improve the video conference effect.

[0072] Secondly, the present application detects the personnel image data of each window by a video quality detection algorithm, combines the resolution and frame rate of the window, obtains a window quality score Q vid , sets a minimum threshold of the window quality score Q vid , when the window quality score is less than the minimum threshold, performs image enhancement processing on the video in the window, ensures the video image accuracy in each window, makes the participant recognition higher, and then obtains a priority score S pri from the face and audio data in each window by a speaker detection algorithm, determines the speaker with the highest score, and automatically replaces the personnel image data in the main window with the personnel image data in the window by software setting, without manual adjustment, so as to intelligently and effectively determine the current speaker, increase the experience effect of large-screen video conference, and improve the conference quality.

[0073] Thirdly, the present application sets a quality threshold for the priority score S pri , when the priority score S pri is less than the quality threshold, it indicates that the window picture quality is too low to detect data from the window, since the speaker detection algorithm needs to detect the mouth opening and closing frequency of the personnel, if the window resolution is too low, it will affect the detection accuracy, at this time, the data in the video quality detection algorithm is adjusted, the weight of the resolution is increased, so that the window quality score Q vid can be adjusted in real time according to the specific situation of the window quality, improve the flexibility of the window quality score Q vid , and at the same time, the algorithms can influence each other, and improve the large-screen video conference effect together. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 The present application is a method flowchart. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of the present application.

[0076] Embodiments, please refer to Figure 1 The present embodiment provides a large-screen video conference window adjustment method based on image processing, comprising the following steps:

[0077] Preparation before the meeting, build a large-screen display system, which includes a display screen, obtain the image data of the personnel participating in the video conference, and project it on the video window of the display screen;

[0078] Screen division, divide the video window into a main window and several sub-windows using a video window division algorithm, the main window is used to display the speaker participating in the video conference, the display area of the main window is larger than that of the sub-windows, and the sum of the number of the main window and the sub-windows is equal to the number of the personnel participating in the video conference;

[0079] Image processing, using a video quality detection algorithm to detect the image data of the personnel in the main window and the sub-windows in the video conference, to obtain a window quality score Q vid , and set the minimum threshold value Y1, when the window quality score Q vid is less than Y1, perform image enhancement processing on the window; vid

[0080] Speech adjustment, in the process of large-screen video conference, using a speaker detection algorithm to obtain a priority score S pri of each window, the highest score is determined as the speaker, and the image data of the personnel is replaced with the image data of the personnel on the main window, and set the quality threshold value Y2 of the priority score S pri , when the priority score S pri is less than Y2, adjust the data in the video quality detection algorithm;

[0081] Meeting record, record the process of video conference using recording software to obtain meeting data, after the large-screen video conference ends, store the meeting data and set a label for subsequent review.

[0082] Specifically, the meeting data includes video data and audio data, wherein the video data includes pre-image processing data and post-image processing data, by dividing the video data into pre-image processing data and post-image processing data, it can be clearly known that the image enhancement effect is obvious when viewed subsequently, so as to decide whether to adjust and optimize.​

[0083] In this embodiment, the video window is divided into a main window and several sub-windows by a video window division algorithm, the main window is used to display the speaker, since the main window is larger than the sub-windows, the several sub-windows are evenly distributed in other positions, which can efficiently utilize the screen space, avoid overlapping and waste, and the number of sub-windows can be dynamically adjusted according to the number of participants in the video conference, so that the interface remains neat and does not feel crowded regardless of the number of participants, thereby enabling other video conference participants to clearly see the current speaker, effectively improving the concentration degree and improving the conference effect. vid , the window quality score Q vid represents the clarity and smoothness of the window, and the minimum threshold of the window quality score Q vid is Y1, when the window quality score Q vid is less than Y1, image enhancement processing is performed on the window, the video quality detection algorithm can automatically perform image enhancement processing on the window video with low quality, ensuring the accuracy of each window video while reducing the consumption of computer performance, improving the accuracy of subsequent recognition, making the conference process smoother, and enhancing the comfort of large-screen video conference.

[0084] Finally, the speaker detection algorithm obtains a priority score S pri according to the face and audio data in each window, the window with the highest score is determined as the speaker, and the personnel image data in the window is automatically replaced with the personnel image data in the main window through software setting, without manual adjustment, thereby intelligently and effectively determining the current speaker, increasing the experience effect of large-screen video conference, improving the conference quality, and setting the quality threshold of the priority score S pri as Y2, when the priority score S pri is less than Y2, it indicates that the window picture quality is too low to detect data from the window, at this time, the data in the video quality detection algorithm is adjusted to change the window quality score Q vid according to the actual situation, improving the flexibility of the method.

[0085] Further, the large-screen display system includes a display screen, a signal processing and transmission device, a control device, and an audio device.

[0086] Specifically, the display screen is an LED display screen, the LED display screen supports large-size display while ensuring high-resolution image quality, the LED display screen has good brightness and contrast, and can keep clear and visible under different lighting conditions, in a multi-person video conference, participants can clearly see the current speaker in a complex visual environment, and the signal processing device is used for processing audio and video signals collected from devices such as cameras and microphones, thereby improving the quality of the video conference.

[0087] The control device is the command center of the entire large-screen display system, and is responsible for coordinating the source of the video signal, adjusting the layout, switching the video window, and the like, the control device starts, pauses or adjusts the video signal output of different participants according to the needs of the system, automatically or manually switches the speaker window, and ensures the smooth progress of the conference.

[0088] Further, the main window size calculation process in the video window division algorithm is as follows:

[0089] W m = R m × W, H m = R m × H

[0090] Wherein W m is the width of the main window, H m is the height of the main window, and m is the main window;

[0091] R m is the proportion of the main window occupying the video window, and the value range is 0 to 1;

[0092] W is the total width of the video window, and H is the total height of the video window;

[0093] The calculation process of the size of the secondary window is as follows:

[0094]

[0095] Wherein W s is the width of the secondary window, H s is the height of the secondary window, and s is the secondary window;

[0096] N is the total number of participants in the video conference;

[0097] N-1 is the number of secondary windows;

[0098] is the number of columns of the secondary window;

[0099] The calculation process of the arrangement position of the secondary window is as follows:

[0100]

[0101] Wherein Xsi is the horizontal coordinate of the i-th window;

[0102] Y si is the vertical coordinate of the i-th window;

[0103] is the number of rows of the sub-window;

[0104] The video window division algorithm can divide the display screen video window into a main window and a plurality of sub-windows, and can generate a suitable conference window layout according to the number of participants in the video conference, and the number of sub-windows can be dynamically adjusted as the number of participants in the video conference changes. No matter how many participants in the video conference, the interface remains neat and has no crowded feeling, so that the participants in the video conference can clearly see the current speaker, effectively improve the attention concentration, and improve the effect of the video conference.

[0105] Further, the video quality detection algorithm process in the image processing step is as follows:

[0106]

[0107] wherein Q vid (i) is the i-th window quality score, Q vid is the window quality score;

[0108] W vidi ×H vidi is the resolution of the i-th window, indicating the clarity of the personnel image data;

[0109] W vidi is the width of the i-th window video, W max is the maximum width of the video;

[0110] H vidi is the height of the i-th window video; H max is the maximum height of the video;

[0111] F vidi is the frame rate of the i-th window video, F mzx is the maximum frame rate of the video;

[0112] K r is the resolution weight, taking a value of 0.5;

[0113] K f is the frame rate weight, taking a value of 0.5;

[0114] Specifically, the i-th window quality score Q vid (i) indicates the clarity and smoothness of the personnel image data, and the window quality score Q vid is set to Y1, and when the i-th window quality score Qvid (i) is less than Y1, indicating that the video in the window is blurred and stuttered, and the overall quality of the video is low. In this case, image enhancement processing is performed on the video in the window. The video quality detection algorithm can automatically perform image enhancement processing on the window with low video quality, ensuring the precision of the video image in each window, making the participant more recognizable, and improving the accuracy of the subsequent speaker detection algorithm.

[0115] Further, the speaker detection algorithm in the speech adjustment step is as follows:

[0116] S pri (i) = P spe (i) × Q vid (i)

[0117] where S pri (i) is the priority score of the i-th window, S pri is the priority score;

[0118] S spe (i) is the i-th window speech probability

[0119] Q vid (i) is the i-th window quality score;

[0120] Specifically, S pri (i) is derived as follows:

[0121] S pri (i) = S spe,face (i) × β + S spe,audio (i) × γ

[0122] where S spe,face (i) is the i-th window based on facial speech probability;

[0123] S spe,audio (i) is the i-th window based on audio speech probability;

[0124] β is the facial speech probability weight, ranging from 0.5 to 1;

[0125] γ is the audio speech probability weight, ranging from 0.5 to 1;

[0126] Specifically:

[0127]

[0128] where S spe,face (i) is the i-th window based on facial speech probability;

[0129] f mouth,i is the mouth opening frequency of the i-th window;

[0130] f spr is the standard opening and closing frequency of the mouth when speaking;

[0131] α mouth,i is the opening and closing amplitude of the mouth;

[0132] α max is the maximum opening and closing amplitude of the mouth;

[0133]

[0134] where S spe,audio (i) is the i-th window-based audio speech probability;

[0135] E signal,i is the i-th window audio energy value;

[0136] E th is the audio energy threshold value;

[0137] In particular, the priority score S pri indicates the level of window speech probability, and through the information of the facial speech probability and the audio speech probability, the system can more accurately determine whether the current window is in a speech state. For example, the audio energy value in the audio speech probability can effectively identify whether the sound exists, and the mouth opening and closing frequency further confirms whether the corresponding mouth movement or expression is consistent with the audio energy value. The two complement each other, and even in an environment with background noise, the method can improve the recognition accuracy through the complementary characteristics of multiple modalities, avoid mistakenly identifying non-speakers or other participants as speakers, and the priority score S pri The highest window will replace the main window, and when the priority score S pri is the highest, no change will occur, ensuring the stability of the video conference and eliminating the need for manual adjustment, thereby intelligently and effectively determining the current speaker, increasing the experience effect of large-screen video conferences, and improving the quality of the conference.

[0138] Further, the quality threshold of the priority score S pri is set as Y2, and when the priority score S pri of a window is less than Y2, it indicates that the quality of the current window picture data is too low, and at this time, the window resolution weight K r is increased in the video quality detection algorithm, and the specific value is:

[0139] K r = 0.5 + Z

[0140] When the window resolution weight K r , in order to ensure that K r + K f = 1, at this time Kf Correspondingly reduced, specifically:

[0141] K f = 0.5 - Z

[0142] Z is an adjustment amount, and the value range is 0.1 to 0.4;

[0143] Since the speaker detection algorithm needs to detect the mouth opening frequency of the personnel, if the window resolution is too low, it will affect the accuracy of the detection, at this time, the priority score S of each window is calculated pri When the priority score S pri is too low, it can affect the data in the video quality detection algorithm, increase the weight of the resolution, and make the window quality score Q vid be able to adjust the specific situation of the window quality in real time, improve the flexibility of the window quality score Q vid .

[0144] Further, the image enhancement processing includes noise removal, super-resolution reconstruction and frame rate interpolation.

[0145] Specifically, noise removal is used to remove noise components introduced by shooting equipment, environmental light, network transmission, etc. from the video frame, noise usually appears as random points, spots or loss of details in the image, which seriously affects the visual quality. In a video conference, noise removal can reduce the interference on the screen and obtain clearer and more natural images, improving the viewing experience of conference participants. Super-resolution reconstruction can increase the number of pixels and details of the image, avoid distortion, and more accurately capture the facial features of personnel. Frame rate interpolation is to increase the video frame rate by inserting intermediate frames. By inserting new frames through algorithms, the transition between original frames is smoothed, avoiding the phenomenon of stuttering or incoherence in the video playback process, and improving the smoothness of the video conference.

[0146] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A large-screen video conference window adjustment method based on image processing, characterized in that, Comprising the following steps: Step S1: Pre-meeting preparation, build a large screen display system, the large screen display system includes a display screen, obtain the image data of the personnel participating in the video conference, and project it on the video window of the display screen; Step S2: Screen division, divide the video window into a main window and several sub-windows using a video window division algorithm, and automatically adjust the size of the main window and the sub-windows according to the number of participants in the meeting, the main window is used to display the speaker participating in the video conference, the area of the main window on the display screen is larger than that of the sub-windows, and the sum of the number of the main window and the sub-windows is equal to the number of participants in the video conference; Step S3: image processing, using video quality detection algorithm to detect the personnel image data in the main window and the sub-window in the video conference, and fusing and analyzing the standardized personnel image data to obtain a window quality score Q vid , and setting the window quality score Q vid The lowest threshold is Y1, and when the window quality score Q vid is less than Y1, image enhancement processing is performed on the window. The video quality detection algorithm process in the image processing step is as follows: where Q vid (i) is the i-th window quality score, Q vid is the window quality score; W vidi ×H vidi is the resolution of the i-th window, indicating the definition of the personnel image data; W vidi is the width of the i-th window video, W max is the maximum width of the video; H vidi is the height of the i-th window video; H max is the maximum height of the video; F vidi is the ith window video frame rate, F mzx is the maximum video frame rate; K r is a resolution weight, taking a value of 0.5; K f is a frame rate weight, taking a value of 0.5; The i-th window quality score Q vid (i) represents the definition and fluency of the personnel image data, and the window quality score Q vid The minimum threshold is Y1, and when the i-th window quality score Q vid (i) is less than Y1, it indicates that the video in the window is blurred and stuttered, and the overall quality of the video is low, and image enhancement processing is performed on the video in the window. Step S4: speech adjustment, in the process of large screen video conference, using the speaker detection algorithm to get the priority score S of each window pri , and introducing the face speech probability factor in the speaker detection algorithm, the face speech probability factor is obtained by processing and analyzing the face data, the priority score S pri The highest is the speaker, and the personnel image data is replaced with the personnel image data on the main window, and the priority score S is set pri The quality threshold value is Y2, when the priority score S pri is less than Y2, adjust the data in the video quality detection algorithm; The speaker detection algorithm process in the speaking adjustment step is as follows: where S pri (i) is the priority score of the ith window, S pri is the priority score; S spe (i) is the probability of utterance of the i-th window Q vid (i) is the i-th window quality score; In particular, S pri (i) is derived as follows: where S spe,face (i) is the i-th window based face speaking probability; S spe,audio (i) is the i-th window based on audio speech probability; Beta is the face speaking probability weight, and the value range is 0.5 to 1; Gamma is the audio speaking probability weight, and the value range is 0.5 to 1; Specifically: where S spe,face (i) is the i-th window based face speaking probability; f mouth,i is the mouth opening frequency of the ith window; f spr is the standard mouth opening frequency when pronouncing α mouth,i is the mouth opening amplitude; a max is the maximum opening amplitude of the mouth; where S spe,audio (i) is the ith window based audio utterance probability; E signal,i is the i-th window audio energy value; E th is an audio energy threshold; Priority score S pri The priority score S represents the probability of the window to speak pri The highest window will be replaced with the main window when the priority score S pri When the highest is the main window, no change will occur, ensuring the stability of the video conference; Step S5: Meeting record, record the video conference process using recording software to obtain meeting data, and store the meeting data after the large screen video conference ends, and set a label for subsequent review.

2. The image processing based large screen video conference window adjustment method according to claim 1, characterized in that: The large screen display system includes a display screen, signal processing and transmission equipment, control equipment and audio equipment.

3. The image processing based large screen video conference window adjustment method according to claim 2, characterized in that: The main window size calculation process in the video window division algorithm is as follows: , where W m is the main window width, H m is the main window height, m is the main window; R m is the proportion of the main window occupying the video window, and the value range is between 0 and 1; W is the total width of the video window, and H is the total height of the video window; The sub-window size calculation process is as follows: , where W s is the sub-window width, H s is the sub-window height, s is the sub-window; N is the total number of participants in the video conference; N-1 is the number of sub-windows; is the number of columns of the sub-window; The sub-window arrangement position calculation process is as follows: where X si is the horizontal coordinate of the ith window; Y si is the ordinate of the i-th window; is the number of rows of the secondary window; Through the video window division algorithm, a suitable meeting window layout can be automatically generated according to the number of participants in the video conference, so that the display screen video window layout is more neat, and the main window can highlight the speaker.

4. The image processing based large screen video conference window adjustment method according to claim 3, characterized in that: Set priority rating S pri The quality threshold is Y2, and when the priority score S of a certain window is... pri If the value is less than Y2, it indicates that the current window's image data quality is too low. In this case, the resolution weight K of the window needs to be increased. r The specific weighting of video quality detection algorithms is as follows: When the window resolution weight K r , in order to ensure that K r + K f = 1, K f is reduced accordingly, specifically: Z is the adjustment amount, and the value range is 0.1 to 0.4; Since the speaker detection algorithm needs to detect the face of the person, if the window resolution is too low, it will affect the accuracy of face detection. At this time, the priority score S of each window pri When the priority score S pri is too low, it can affect the data in the video quality detection algorithm, increase the weight of the resolution, and make the window quality score Q vid can be adjusted in real time according to the specific situation of the window quality.

5. The image processing based large screen video conference window adjustment method according to claim 4, characterized in that: The image enhancement processing includes noise removal, super-resolution reconstruction and frame rate interpolation.

6. The image processing based large screen video conference window adjustment method according to claim 2, characterized in that: The meeting data includes video data and audio data, wherein the video data includes pre-image processing data and post-image processing data.

Images