An implementation system and method for free layout of distributed seats

Abstract

The present invention belongs to the technical field of seat management, and specifically relates to a system and method for realizing distributed seat free layout, aiming to solve the problems of low flexibility of pre-designed layout, low user interaction experience, and the problem that the pictures in the layout are tiled, cannot be superimposed and displayed, and have no sense of hierarchy. The present invention includes: each signal source is encoded into network streaming media through a corresponding encoding node and sent to the seat node through the network, and the restored signal source is output. The UI module selects the network streaming media to be displayed through the displayed UI interface, freely arranges them in the canvas area, and sends the layout list information to the video output module. According to the layout list information and the restored signal source, an image is synthesized and output to the display for display. The present invention can flexibly perform free layout, enhance the user interaction experience, can layout multiple video windows for superimposed display, and can adjust the superimposed Z-order pictures to have a sense of hierarchy.

Description

Technical Field

[0001] The present invention belongs to the technical field of seat management, and particularly relates to a system and method for realizing free layout of distributed seats. Background Art

[0002] In multi-channel video application software such as video surveillance, video conferencing, and distance education of distributed seats, various video layout methods are often seen. The video layout method presents a checkerboard cutting mode. By defining the number of columns and rows of the checkerboard (such as xDiv and yDiv), it is determined how many basic units the panel is cut into. These layouts are usually preset layouts such as single screen, 4 screens, 9 screens, 16 screens, 1 large and 5 small, etc. During use, just click the layout application.

[0003] The preset layout has the following defects in actual use:

[0004] 1. The flexibility of the pre-designed (checkerboard cutting) layout is not high. Usually, only layouts such as single screen, 4 screens, 9 screens, etc. can be selected, which cannot meet the user's custom needs;

[0005] 2. The user interaction experience is not high. It is impossible to draw a picture window with a mouse, drag and adjust the window size, position, and adjust the Z-order;

[0006] 3. The pictures in the layout are in a tiled manner, and the pictures cannot be superimposed for display, and the picture display has no sense of hierarchy. Summary of the Invention

[0007] To solve the above problems in the prior art, that is, the low flexibility of the pre-designed layout, the low user interaction experience, and the problems that the pictures in the layout are tiled, cannot be superimposed for display, and have no sense of hierarchy, in the first aspect of the present invention, a system for realizing free layout of distributed seats is proposed. The system includes: a signal source, an encoding node, a network, a seat node, and a display; the seat node includes a decoding module, a UI module, and a video output module;

[0008] Each signal source is encoded into network streaming media through the corresponding encoding node and sent to the seat node through the network;

[0009] The UI module receives the stream selection instruction input by the user through the displayed UI interface, selects the network streaming media to be displayed, freely arranges the pre-displayed windows of the selected network streaming media in the canvas area on the UI interface, and sends the layout list information to the video output module;

[0010] The decoding module decodes the selected network streaming media and outputs the restored signal source;

[0011] The video output module synthesizes an image based on the layout list information and the restored signal source, and outputs it to the display for display. The process is as follows:

[0012] The video output module receives the layout list information; the layout list is a list composed of all video window information of the free layout of the UI interface; the video window information includes X coordinate, Y coordinate, width, height, transparency, and Z order;

[0013] According to the X coordinate, Y coordinate, width, and height in the layout list information, determine the display position of the video window, and calculate the overlapping area O and non-overlapping area between video windows;

[0014] For each pixel in the overlapping area O, calculate the fused pixel value according to the transparency and Z order of multiple video windows at the pixel position respectively;

[0015] For the non-overlapping area, directly draw the pixel value of the corresponding video window according to the layout list information;

[0016] Combine the images of multiple video windows after pixel value processing and the restored signal source to synthesize an image and display it on the display.

[0017] In some preferred embodiments, the fused pixel value is:

[0018] ;

[0019] In the formula, Alpha_i is the transparency of the i-th video window image; R_i, G_i, B_i are the RGB pixel values of the i-th video window image respectively; O_R_(i - 1), O_G_(i - 1), O_B_(i - 1) are the RGB pixel values after the superposition of the (i - 1)-th video window image respectively; O_Alpha_(i - 1) represents the transparency value after the superposition of the (i - 1)-th video window image; O_R_i, O_G_i, O_B_i represent the RGB pixel values after the superposition of the i-th video window image respectively.

[0020] In some preferred embodiments, when freely laying out, the video windows are drawn according to the Z order, and frame extraction and frame interpolation processing are also included during the drawing of the images;

[0021] Obtain the number of frames in the output resolution of the display as the first number of frames;

[0022] According to the layout list information, obtain the number of frames of the input frames of each image drawn by the video window corresponding to the selected network streaming media as the second number of frames;

[0023] If the first frame number is lower than the second frame number, frame extraction processing is performed on the corresponding picture. The frame extraction processing is as follows: The timer triggers to obtain an input frame once every reciprocal time of the first frame number, and the obtained input frame is placed in the timer's frame-to-be-obtained area in a manner of covering at the reciprocal time interval of the second frame, and any unobtained input frame received at a non-triggering moment of the timer is discarded;

[0024] If the first frame number is higher than the second frame number, frame interpolation processing is performed on the corresponding picture;

[0025] The frame interpolation processing is as follows: The output frame of the corresponding picture is placed in the memory as a memory frame; the timer triggers to obtain a memory frame once every reciprocal time of the first frame number. If the memory frame is not updated, the previous frame is displayed; otherwise, the updated memory frame is displayed.

[0026] The second invention of the present invention proposes a method for implementing distributed seat free layout, including the following steps:

[0027] When a mouse press event is detected, the coordinates of the mouse in the canvas area are obtained as the original coordinates;

[0028] It is judged whether the original coordinates are within each video window. If not, a dynamic rectangle is generated by combining the coordinates after the mouse movement and the original coordinates; based on the dynamic rectangle, an A-color frame, a B-color frame, and the picture of the video window are drawn simultaneously. If so, the Z-order of the clicked video window is changed to the front, and the picture of the video window is displayed; both the A-color frame and the B-color frame are dynamic rectangles whose colors are set according to the resolution of the dynamic rectangle;

[0029] When the dynamic rectangle is a B-color frame and a mouse release event is detected, the drawing of the current video window's picture is completed;

[0030] After the drawing is completed, the color display of the A-color frame and the B-color frame is turned off, and the validity of the current video window is obtained; if the current video window is a valid form, the picture of the video window is displayed, and the form information is saved to the layout list as the layout list information; if the current video window is not a valid form, it directly ends.

[0031] In some preferred embodiments, the color of the dynamic rectangle is set according to the resolution of the dynamic rectangle. The method is as follows:

[0032] The resolution of the dynamic rectangle is detected and compared with a preset threshold M. If it is less than M, the color of the dynamic rectangle is set to color A;

[0033] If it is greater than or equal to M, the color of the dynamic rectangular frame is set to color B.

[0034] In some preferred embodiments, when there are multiple video windows, the multiple video windows include multiple overlapping video windows and multiple non-overlapping video windows;

[0035] The Z order of the multiple superimposed video windows can be adjusted by a mouse click event on the video window.

[0036] In some preferred implementations, re-layout may be performed after free layout, and the re-layout includes:

[0037] Adjusting the size of the window after free layout, including reconfiguring the width and height in the video window information saved in the UI interface;

[0038] Or, on the video window saved in the UI interface, the width and height of the video window are changed based on mouse events; the mouse events include click events, press events, and move events;

[0039] In some preferred embodiments, the rearrangement further comprises:

[0040] Adjust the free layout window position, including reconfiguring the X and Y coordinates in the saved video window information in the UI interface;

[0041] Or, on a video window saved in the UI interface, the position of the video window is changed based on mouse events; the mouse events include click events, move events, and mouse release events.

[0042] Beneficial effects of the present invention:

[0043] 1. On the preset canvas of distributed seats, you can flexibly and freely layout the windows. You can use the mouse to freely layout the windows, drag the window position, and adjust the window size, which increases the operability and practicality of the user interaction process and bids farewell to the monotony of the traditional chessboard-type layout;

[0044] 2. In the free layout preset canvas, multiple video windows can be laid out, and the screens are drawn in Z order according to the layout list information. When drawing the screen, frame extraction and frame insertion processing are also included. The input frames of each screen drawn by the video window are consistent with the number of frames in the output resolution of the display. They can be superimposed and displayed with a sense of hierarchy. When multiple windows are superimposed, the superimposed Z order of the free layout windows can be freely adjusted;

[0045] 3. The free layout window can provide color prompts when it exceeds the output resolution limit. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non - limiting embodiments read in conjunction with the accompanying drawings:

[0047] Figure 1 is a structural block diagram of an implementation system for a distributed seat free layout of the present invention;

[0048] Figure 2 is a free layout step diagram of an implementation method for a distributed seat free layout of the present invention;

[0049] Figure 3 is a Z - order adjustment step diagram of an implementation method for a distributed seat free layout of the present invention;

[0050] Figure 4 is a mouse press event diagram of an implementation method for a distributed seat free layout of the present invention;

[0051] Figure 5 is a mouse movement event diagram of an implementation method for a distributed seat free layout of the present invention;

[0052] Figure 6 is a mouse release event diagram of an implementation method for a distributed seat free layout of the present invention;

[0053] Figure 7 is a step diagram for adjusting the free layout window size after free layout and then re - layout of an implementation method for a distributed seat free layout of the present invention;

[0054] Figure 8 is a step diagram for adjusting the free layout window position after free layout and then re - layout of an implementation method for a distributed seat free layout of the present invention;

[0055] Figure 9 is a schematic diagram of the UI module saving the output layout list to the video output module of an implementation method for a distributed seat free layout of the present invention. Detailed implementation manners

[0056] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and not to limit the invention. Additionally, it should be noted that for the convenience of description, only parts related to the relevant invention are shown in the drawings.

[0057] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0058] For a clearer description of an implementation method for a distributed seat free layout of the present invention, the following is combined withFigures 1 to 9 Details of each step in the embodiments of the present invention are elaborated below.

[0059] An implementation system for free layout of distributed seats in the first embodiment of the present invention is shown in Figure 1 , the system includes a signal source, an encoding node, a network, a seat node, and a display. The seat node includes a decoding module, a UI module, and a video output module;

[0060] Each signal source is encoded into network streaming media through the corresponding encoding node and sent to the seat node through the network;

[0061] The UI module receives the stream selection instruction input by the user through the displayed UI interface, selects the network streaming media to be displayed, freely arranges the pre-display windows of the selected network streaming media in the canvas area on the UI interface, and sends the layout list information to the video output module;

[0062] The decoding module decodes the selected network streaming media and outputs the restored signal source;

[0063] The video output module synthesizes an image according to the layout list information and the restored signal source, and outputs it to the display for display;

[0064] In this embodiment, the encoding node: a device that encodes video signals input through common video interfaces such as HDMI and DVI into network streaming media signals;

[0065] The seat node: a device that decodes multiple network streaming media through UI interface interaction for stream selection and outputs the decoded images to the subsequent display;

[0066] The video output module synthesizes an image according to the layout list information and the restored signal source, and outputs it to the display for display. The process is as follows:

[0067] The video output module receives the layout list information; the layout list is a list composed of all video window information freely arranged on the UI interface; the video window information includes X coordinate, Y coordinate, width, height, transparency, and Z order;

[0068] Taking the superposition display of three video windows A, B, and C in the free layout as an example in this embodiment, the display positions of the video windows are determined according to the X coordinate, Y coordinate, width, and height in the layout list information, and the overlapping area O and non-overlapping area between the video windows are calculated;

[0069] For each pixel in the overlapping region O, calculate the fused pixel value according to the transparency and Z-order of multiple video windows at the position of this pixel respectively, and draw the pixel value of the corresponding video window according to the fused pixel value. In this embodiment, the transparencies of the images A, B, and C corresponding to the three video windows A, B, and C are Alpha_A, Alpha_B, and Alpha_C respectively (the value range is from 0 to 1, where 1 means completely opaque and 0 means completely transparent). The fused pixel value is as follows:

[0070] ;

[0071] In the formula, Alpha_i is the screen transparency of the i-th video window; R_i, G_i, and B_i are the RGB pixel values of the screen of the i-th video window respectively; O_R_(i - 1), O_G_(i - 1), and O_B_(i - 1) are the RGB pixel values after the superposition of the screens of the (i - 1)-th video window respectively; O_Alpha_(i - 1) represents the transparency value after the superposition of the screens of the (i - 1)-th video window, and is calculated in the descending order of the screens according to the Z-order (i represents the screen serial number of the video window, starting from 1 to 3, corresponding to the descending order of images C, B, and A); O_R_i, O_G_i, and O_B_i represent the RGB pixel values after the superposition of the screens of the i-th video window respectively;

[0072] For the non-overlapping region, directly draw the pixel value of the corresponding video window according to the layout list information;

[0073] Combine the screens of multiple video windows after pixel value processing and the restored signal source to synthesize an image and display it on the display;

[0074] When freely laying out, the video windows draw the screens according to the Z-order, and frame extraction and frame interpolation processing are also included during screen drawing;

[0075] Obtain the number of frames in the output resolution of the display as the first number of frames;

[0076] According to the layout list information, obtain the number of frames of the input frames of each screen drawn by the video window corresponding to the selected network streaming media as the second number of frames;

[0077] If the first number of frames is lower than the second number of frames, perform frame extraction processing on the corresponding screen. The frame extraction processing is as follows: The timer triggers to obtain an input frame once every reciprocal time of the first number of frames, and put the obtained input frame into the timer frame waiting to be obtained area in the way of covering at the reciprocal time interval of the second frame, and discard any unobtained input frames received at non-triggering moments of the timer;

[0078] If the first number of frames is higher than the second number of frames, perform frame interpolation on the corresponding picture;

[0079] The frame interpolation process is as follows: Put the output frame of the corresponding picture into the memory as a memory frame; The timer triggers to obtain a memory frame every interval of the reciprocal time of the first number of frames. If the memory frame is not updated, display the previous frame; otherwise, display the updated memory frame;

[0080] The following is a detailed description with examples: When the output resolution of the subsequent display is 1920*1080 - 60 frames, and the picture input of the free-layout internal window A is 25 frames (less than 60 frames), frame interpolation operation needs to be performed on the picture of window A to complete 25 frames to 60 frames; When the subsequent display outputs 30 frames and the picture input of window A is 60 frames, frame extraction operation needs to be performed on the picture of A to extract 60 pictures into 30 frames to ensure that the frame rates of the pictures input by all windows are unified with the output frame rate;

[0081] For frame extraction and interpolation, the output frame of the picture of window A is put into the memory to be submitted. When the picture input of window A is 25 frames and the output is 60 frames, start the timer to obtain a memory frame every 1s / 60 = 16666us. Only when the memory is updated with the picture, the output will be updated synchronously; otherwise, play the previous frame of the repeated picture; When the picture input of window A is 60 frames and the output is 30 frames, the timer obtains the input frame every 1s / 30 = 33333us, and discards the unobtained input frames, so as to achieve dynamic frame extraction and interpolation operations.

[0082] It should be noted that the implementation system of a distributed seat free layout provided in the above embodiments is only illustrated by dividing the above functional modules. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further split into multiple sub-modules to complete all or part of the functions described above. For the names of the modules and steps involved in the embodiments of the present invention, they are only used to distinguish each module or step, and are not regarded as an improper limitation of the present invention.

[0083] A method for implementing a distributed seat free layout according to the second embodiment of the present invention, see Figures 4 - 6 , including the following steps:

[0084] When a mouse press event is detected, obtain the coordinates of the mouse in the canvas area as the original coordinates;

[0085] Determine whether the original coordinates are within each video window. If not, generate a dynamic rectangular box by combining the coordinates after mouse movement and the original coordinates. Based on the dynamic rectangular box, draw the A-color box, the B-color box, and the video window's screen simultaneously. The A-color box and the B-color box are both dynamic rectangular boxes with colors set according to the resolution of the dynamic rectangular box. During the process of drawing the video window, the displacement of the coordinates during mouse movement relative to the original coordinates and the rectangular box color are displayed in real time. The rectangular box color is used to limit the display form resolution to exceed M. During the movement process, the switching between color A and color B is involved. When the dynamic rectangular box is the B-color box and a mouse release event is detected, the drawing of the current video window's screen is completed, achieving free layout. A highlighted white rectangular box will be displayed, and rectangular information including X coordinate, Y coordinate, width, height, and transparency information will be displayed inside the rectangular box at this time.

[0086] Specifically, detect the resolution of the dynamic rectangular box and compare it with the preset threshold M. If it is less than M, set the color of the dynamic rectangular box to color A; if it is greater than or equal to M, set the color of the dynamic rectangular box to color B. The operation steps of the video window free layout are as Figure 2 shown. When a mouse click event occurs on the free layout button in the UI interface, the UI interface generates a canvas area. Based on the mouse press event, obtain the original coordinates. Based on the original coordinates and the mouse movement event, a dynamic rectangular box appears for free drawing of the video window. Based on the mouse release event and the dynamic rectangular box being the B-color box, the free drawing of the video window is completed.

[0087] If so, change the Z-order of the clicked video window to the front and display the screen of the video window. Specifically: when there are multiple video windows, the multiple video windows include multiple superimposed video windows and multiple non-superimposed video windows; for the multiple superimposed video windows, the Z-order of each video window can be adjusted through the mouse click event on the video window. The operation steps are as Figure 3 shown. When there are multiple video windows in the free layout canvas area; when multiple windows are superimposed and displayed, for example, the B and C screens are superimposed on the A screen; after the mouse clicks on the A screen, the Z-order of the A screen becomes the highest, and the A screen is located above the B and C screens.

[0088] After the drawing is completed, turn off the color display of the A-color box and the B-color box, and obtain the validity of the current video window. If the current video window is a valid form, display the screen of the video window and save the form information to the layout list as the layout list information. The specific steps are as Figure 9Shown as follows: After drawing multiple windows in the preset canvas; click the Save Layout button and enter the new layout name X; that is, a new X layout will be displayed on the layout switching page; after clicking on the X layout, it is output to the video output module, and the video output module outputs the free layout scheme to the subsequent display; then select the video source for a single picture and decode the output to obtain the final video effect; if the current video window is not a valid form, directly end;

[0089] After the free layout, re-layout can be performed to adjust the size of the free layout window, see Figure 7 , including reconfiguring the width and height in the video window information saved in the UI interface;

[0090] Or on the video window saved in the UI interface, change the width and height of the video window based on mouse events; the mouse events include click events, press events, and move events;

[0091] In this embodiment, reconfigure the width and height in the video window information saved in the UI interface; specifically: when there are multiple video windows in the free layout preset canvas; after clicking on window A, the rectangular frame of window A changes from transparent to bright white; at this time, rectangular information will be displayed inside the rectangular frame, including X coordinate, Y coordinate, width, and height information; after clicking on the X coordinate and Y coordinate of the starting position and re-entering values; after clicking Enter, the size of window A is successfully set; among them, numerical adjustment supports input from the physical keyboard and the right mouse button soft keyboard. During the adjustment process, the size of window A cannot exceed the canvas. After entering a value that exceeds the canvas, the input value automatically becomes the maximum value, and the window is pressed to the outermost edge of the canvas;

[0092] In this embodiment, change the width and height of the video window based on mouse events on the video window saved in the UI interface; specifically, after clicking on window A, the rectangular frame of window A changes from transparent to bright white; drag the bright white border to adjust the size of the window;

[0093] After the free layout, re-layout can be performed to adjust the position of the free layout window, see Figure 8 , including reconfiguring the X coordinate and Y coordinate in the video window information saved in the UI interface;

[0094] Or on the video window saved in the UI interface, change the position of the video window based on mouse events; the mouse events include click events, move events, and mouse release events;

[0095] In this embodiment, reconfiguring the X coordinate and Y coordinate in the video window information saved in the UI interface specifically means that when there are multiple video windows in the free-layout preset canvas; after clicking on window A, the rectangular frame of window A changes from transparent to bright white; at this time, rectangular information including the X coordinate, Y coordinate, width, and height information will be displayed inside the rectangular frame; after clicking on the X coordinate and Y coordinate of the starting position and re-entering values; after clicking Enter, the position of window A is successfully set; among them, numerical adjustment supports input from the physical keyboard and the right-click soft keyboard of the mouse. During the adjustment process, the size of window A cannot exceed the canvas. After entering a value that exceeds the canvas, the input value automatically becomes the maximum value, and the window is pressed to the outermost edge of the canvas.

[0096] In this embodiment, changing the position of the video window based on a mouse event on the video window saved in the UI interface specifically means that when the mouse clicks on window A and does not release the button while moving the mouse; the window follows the movement of the mouse. During the movement process, the displayed image of window A must not exceed the size of the canvas at all times; after the mouse is released, window A will stop at the position where the mouse is released.

[0097] Although the various steps have been described in the above order in the above embodiments, those skilled in the art can understand that in order to achieve the effects of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are all within the protection scope of the present invention.

[0098] Terms such as "first", "second", etc. are used to distinguish similar objects, rather than to describe or represent a specific order or sequence.

[0099] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, so that a process, method, article, or device / equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to these processes, methods, articles, or devices / equipment.

[0100] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A method for implementing distributed seat free layout, based on a distributed seat free layout implementation system, the system comprising: Signal source, encoding node, network, seat node, display; The seat node includes a decoding module, a UI module, and a video output module; Each signal source is encoded into a network streaming media by a corresponding encoding node and sent to the seat node through the network; The UI module receives a stream selection instruction input by a user through a displayed UI interface, selects a network streaming media to be displayed, and freely lays out the pre-displayed windows of the selected network streaming media in a canvas area on the UI interface, and sends the layout list information to the video output module; The decoding module decodes the selected network streaming media and outputs a restored signal source; The video output module synthesizes an image according to the layout list information and the restored signal source, and outputs the image to the display for display, and the process is as follows: The video output module receives the layout list information; the layout list is a list of all video window information freely laid out on the UI interface; the video window information includes X coordinate, Y coordinate, width, height, transparency, and Z order; Determine the video window display position according to the X coordinate, Y coordinate, width, and height in the layout list information, and calculate the overlap area O and non-overlap area between the video windows; For each pixel in the overlapped area O, the fused pixel value is calculated according to the transparency and Z order of the multiple video windows at the pixel position; For the non-overlapping area, directly drawing the pixel value of the corresponding video window according to the layout list information; Combining the images of the plurality of video windows after pixel value processing and the restored signal source to synthesize an image, and displaying the image on the display; Characterized in that the method comprises the following steps: When a mouse press event is detected, the coordinates of the mouse in the canvas area are obtained as the original coordinates; Determine whether the original coordinates are located in each video window. If not, generate a dynamic rectangular frame by combining the coordinates after the mouse is moved and the original coordinates; based on the dynamic rectangular frame, draw the A color frame, the B color frame and the screen of the video window at the same time. If yes, change the Z order of the clicked video window to the front and display the screen of the video window; the A color frame and the B color frame are both dynamic rectangular frames whose colors are set according to the resolution of the dynamic rectangular frame; When the dynamic rectangular frame is a B color frame and a mouse release event is detected, the picture drawing of the current video window is completed; After drawing is completed, close the color display of the A color box and the B color box, and obtain the validity of the current video window; if the current video window is a valid window, display the picture of the video window, and save the window information to the layout list as layout list information; if the current video window is not a valid window, end directly.

2. A method for implementing distributed seat free layout according to claim 1, characterized in that: The color of the dynamic rectangular frame is set according to the resolution of the dynamic rectangular frame, and the method is as follows: Detecting the resolution of the dynamic rectangular frame and comparing the resolution with a preset threshold M, if the resolution is less than the threshold M, setting the color of the dynamic rectangular frame to color A; If it is greater than or equal to M, the color of the dynamic rectangular frame is set to color B.

3. A method for implementing distributed seat free layout according to claim 1, characterized in that: When there are multiple video windows, the multiple video windows include multiple overlapping video windows and multiple non-overlapping video windows; The Z order of the multiple superimposed video windows can be adjusted by a mouse click event on the video window.

4. The method for implementing distributed seat free layout according to claim 1, characterized in that: After free layout, re-layout can be performed, and the re-layout includes: Adjust the size of the window after free layout, including reconfiguring the width and height in the saved video window information in the UI interface; Or on a video window saved in the UI interface, the width and height of the video window are changed based on mouse events; the mouse events include click events, press events, and move events‌.

5. A method for implementing distributed seat free layout according to claim 4, characterized in that: The re-arrangement also includes: Adjust the free layout window position, including reconfiguring the X and Y coordinates in the saved video window information in the UI interface; Or, on a video window saved in the UI interface, the position of the video window is changed based on mouse events; the mouse events include click events, move events, and mouse release events.

6. A method for implementing distributed seat free layout according to claim 1, characterized in that: The fused pixel value is: ； In the formula, Alpha_i is the picture transparency of the i-th video window; R_i, G_i, B_i are the RGB pixel values ​​of the picture of the i-th video window respectively; O_R_(i-1), O_G_(i-1), O_B_(i-1) are the RGB pixel values ​​of the picture of the i-1-th video window after superposition; O_Alpha_(i-1) represents the transparency value of the picture of the i-1-th video window after superposition; O_R_i, O_G_i, O_B_i represent the RGB pixel values ​​of the picture of the i-th video window after superposition.

7. The method for implementing distributed seat free layout according to claim 1, characterized in that: In free layout, the video window draws the screen in Z order, and also includes frame extraction and frame insertion processing; Obtaining a frame number in an output resolution of the display as a first frame number; According to the layout list information, the frame number of input frames of each screen drawn by the video window corresponding to the selected network streaming media is obtained as the second frame number; If the first frame number is lower than the second frame number, a frame extraction process is performed on the corresponding picture, wherein the frame extraction process is as follows: a timer is triggered to obtain an input frame once at a countdown time interval of the first frame number, and the obtained input frame is placed in a timer to-be-obtained frame area in a manner of being covered by a countdown time interval of the second frame, and any unobtained input frame received at a non-triggering time of the timer is discarded; If the first frame number is higher than the second frame number, performing frame insertion processing on the corresponding picture; The interpolation processing is as follows: putting the output frame of the corresponding picture into the memory as a memory frame; triggering the timer to obtain a memory frame once every countdown time of the first frame number, if the memory frame is not updated, displaying the previous frame, otherwise, displaying the updated memory frame.

Images