Display control method and device for multi-screen equipment

Abstract

The invention relates to the technical field of multi-screen control, and discloses a display control method and device for multi-screen equipment, and the method comprises the steps: judging whether each client is a master user or not when meeting joining requests of a plurality of clients are received; if the client is a main user, preferentially receiving a conference joining request of the client, and accessing at least two cameras to the client; sorting the video streams of the clients joining the conference, preferentially sorting the video streams of the client of the main user to the head of a preset acquisition queue, and splicing the video streams of the client of the main user to obtain a panorama; and controlling the panorama of each master user client and the video stream of each non-master user client to be centrally displayed on the screen of the multi-screen equipment based on the sorted video stream of the acquisition queue. The method has the effects that the window display definition of the conference joining client is improved, and the watching experience of a user is better.

Description

Technical Field

[0001] This application relates to the technical field of multi-screen control, and in particular, to a display control method and device for multi-screen devices. Background Art

[0002] In the scenario of multi-person video conferencing calls, there are many situations where multiple clients join a meeting at the same time. In the prior art, for the multi-screen gallery layout, an N-equal distribution layout from left to right is adopted. When the number of participating clients is small, the windows of the participating clients will be concentrated on the first screen for display. Due to the light interference around the screen, the color and brightness performance of the displayed pictures of the windows of the participating clients are poor, resulting in a problem of poor display clarity of the windows of the participating clients. Summary of the Invention

[0003] In order to improve the display clarity of the windows of the participating clients in the scenario of multi-person video conferencing calls, this application provides a display control method and device for multi-screen devices.

[0004] In a first aspect, this application provides a display control method for a multi-screen device.

[0005] This application is achieved through the following technical solutions:

[0006] A display control method for a multi-screen device includes the following steps,

[0007] When receiving the joining requests of multiple clients, determine whether each client is a primary user;

[0008] If the client is a primary user, preferentially accept the joining request of the client, and the client accesses at least two cameras;

[0009] Sort the video streams of each participating client, preferentially sort the video stream of the primary user client to the head of a preset acquisition queue, and splice the video streams of the primary user client to obtain a panoramic view;

[0010] Based on the sorted video streams in the acquisition queue, control the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device.

[0011] This application can be further configured in a preferred example as follows: The step of controlling the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device based on the sorted video streams in the acquisition queue includes,

[0012] Obtain the total number of cameras accessed by any primary user client;

[0013] Divide each screen into n equal parts, where n is a positive integer;

[0014] Create n x n galleries for each screen according to the equal division situation, and sort each gallery by row;

[0015] Select the panorama of the first main user client to enter the meeting from the sorted video streams in the acquisition queue and insert it at the head of the first row of galleries;

[0016] Calculate the current offset value based on the total number of cameras accessed by the main user client, the total number of galleries, and the current video stream length value in the gallery;

[0017] Move the whole panorama in the first row of galleries by the current offset value;

[0018] Detect all galleries row by row to find the target queue with the minimum length. From the sorted video streams in the acquisition queue, sequentially select the panoramas corresponding to the remaining main user clients and insert them at the preset positions of the current video stream in the target queue, and repeatedly calculate the current offset value of the target queue and move the current video stream by the current offset value until the panoramas of each main user client are centered on the screen of the multi-screen device;

[0019] Wherein, when the panorama is inserted at the head of an empty target queue, move the inserted panorama by the current offset value of the gallery.

[0020] In a preferred example, the present application can be further configured that the step of sequentially selecting the panoramas corresponding to the remaining main user clients from the sorted video streams in the acquisition queue and inserting them at the preset positions of the current video stream in the target queue includes,

[0021] Configure a variable for each target queue and the initial value of the variable is 0. When a panorama is inserted into a target queue, the variable value of the target queue iteratively changes in the order of 1, 0;

[0022] When inserting the panorama corresponding to the main user client into the target queue, first detect the variable value of the target queue;

[0023] If the variable value of the target queue is 0, splice the panorama to the end position of the current video stream;

[0024] If the variable value of the target queue is 1, splice the panorama to the head position of the current video stream.

[0025] In a preferred example, the present application can be further configured that the step of controlling the panoramas of each main user client and the video streams of each non-main user client to be centered on the screen of the multi-screen device based on the sorted video streams in the acquisition queue further includes,

[0026] Detect all galleries row by row to find the target queue with the minimum length. From the sorted video streams in the acquisition queue, sequentially select the video streams corresponding to non-primary user clients and insert them into a preset position of the current video stream in the target queue. When a video stream is inserted at the head of an empty target queue, move the inserted video stream as a whole by the current offset value of the gallery.

[0027] After all the video streams corresponding to non-primary user clients are inserted, calculate the current offset value of each gallery, and move the current video stream of each gallery as a whole by the corresponding current offset value to complete the centering display control of the panoramic images of each primary user client and the video streams of each non-primary user client on the screen of the multi-screen device.

[0028] In a preferred example, the present application can be further configured that the step of sequentially selecting the video streams corresponding to non-primary user clients from the sorted video streams in the acquisition queue and inserting them into a preset position of the current video stream in the target queue includes that

[0029] when a video stream is inserted into the target queue, the variable value of the target queue iteratively changes in the order of 1 and 0.

[0030] When inserting the video stream corresponding to a non-primary user client into the target queue, first detect the variable value of the target queue.

[0031] If the variable value of the target queue is 0, splice the video stream to the end position of the current video stream.

[0032] If the variable value of the target queue is 1, splice the video stream to the head position of the current video stream.

[0033] In a preferred example, the present application can be further configured that when calculating the current offset value based on the total number of cameras accessed by the primary user client, the total number of galleries, and the length value of the current video stream in the gallery, the following formula is used: offset value = Round((m * n - queue_max_size) / n)

[0034] In the formula, Round() represents the rounding operation, m is the total number of cameras accessed by any primary user client, n is the total number of galleries, and queue_max_size is the length value of the current video stream in the gallery. <s

[0035] In a preferred example, the present application can be further configured that the step of determining whether each client is a primary user when receiving the joining requests of multiple clients includes that

[0036] detect the number of input video streams.

[0037] When the number of video streams is greater than or equal to 2, determine that the client is a primary user.

[0038] In a preferred example, the present application can be further configured as follows: The steps of sorting the video streams of each client joining the meeting, preferentially sorting the video stream of the primary user client to the head of a preset acquisition queue, and stitching the video streams of the primary user client to obtain a panoramic view include:

[0039] Obtain the device ID, user identity identifier, camera number, and corresponding video stream of each client joining the meeting;

[0040] Preferentially sort the video stream of the client whose user identity identifier is the same as the primary user identifier to the head of the acquisition queue;

[0041] According to the device ID and the joining order, sort and stitch the video streams of the same device ID that joined earlier in ascending order according to the camera number to obtain the panoramic view of each primary user client.

[0042] In a second aspect, the present application provides a display control device for a multi-screen device.

[0043] The present application is achieved through the following technical solutions:

[0044] A display control device for a multi-screen device includes:

[0045] A primary user identification module, configured to determine whether each client is a primary user when receiving joining requests from multiple clients;

[0046] A primary user joining module, configured to preferentially accept the joining request of the client when determining that the client is a primary user, and the client is connected to at least two cameras;

[0047] A panoramic view module, configured to sort the video streams of each client joining the meeting, preferentially sort the video stream of the primary user client to the head of a preset acquisition queue, and stitch the video streams of the primary user client to obtain a panoramic view;

[0048] A display control module, configured to control the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device based on the sorted video streams in the acquisition queue.

[0049] In a third aspect, the present application provides a computer device.

[0050] The present application is achieved through the following technical solutions:

[0051] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned display control methods for a multi-screen device are implemented.

[0052] In summary, compared with the prior art, the beneficial effects brought by the technical solution provided by the present application at least include:

[0053] By preferentially accepting the joining requests of the primary user clients and sorting the video streams of the joined clients, and preferentially sorting the video stream of the primary user client to the head of the preset acquisition queue, it is beneficial to realize that the video streams of the telepresence users are preferentially arranged in the middle position of the screen; by controlling the panoramic views of the primary user clients and the video streams of the non-primary user clients to be centered on the screen of the multi-screen device, compared with the traditional tiled layout method, the display screen of this solution spreads from the middle of the screen to both sides, which can make the display screen concentrated in the middle position of the screen, rather than being biased to the left or right side. Furthermore, there is no problem that the color and brightness performance of the window display screen of the joined client is poor due to the light interference around the screen, which improves the window display clarity of the joined client. At the same time, the overall centered display screen can be located in the concentrated viewing area of the user, making the user's viewing experience better, especially suitable for ultra-large screens. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the main process of a display control method for a multi-screen device provided by an exemplary embodiment of the present application.

[0055] Figure 2 It is a schematic diagram of an application scenario of a display control method for a multi-screen device provided by an exemplary embodiment of the present application.

[0056] Figure 3 It is a schematic diagram of the video stream arrangement of a display control method for a multi-screen device provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] This specific embodiment is only an explanation of the present application, and it is not a limitation of the present application. Those skilled in the art can make modifications without creative contributions to this embodiment according to needs after reading this specification, but as long as they are within the scope of the claims of the present application, they are protected by the patent law.

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present application belong to the scope of protection of the present application.

[0059] In addition, the term "and / or" in this text merely describes the associated relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this text generally represents an "or" relationship between the front and back associated objects unless otherwise specified.

[0060] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0061] Referring to Figure 1 , the embodiments of the present application provide a display control method for a multi-screen device, and the main steps of the method are described as follows.

[0062] S1: When receiving the joining requests of multiple clients, determine whether each client is the main user;

[0063] S2: If the client is the main user, preferentially accept the joining request of the client, and the client accesses at least two cameras;

[0064] S3: Sort the video streams of each joined client, preferentially sort the video stream of the main user client to the head of a preset acquisition queue, and splice the video stream of the main user client to obtain a panoramic view;

[0065] S4: Based on the sorted video streams in the acquisition queue, control the panoramic views of each main user client and the video streams of each non-main user client to be centered and displayed on the screen of the multi-screen device.

[0066] In one embodiment, the step of determining whether each client is the main user when receiving the joining requests of multiple clients includes

[0067] Detect the number of input video streams;

[0068] When the number of video streams is greater than or equal to 2, determine that the client is the main user.

[0069] In one embodiment, the step of sorting the video streams of each joined client, preferentially sorting the video stream of the main user client to the head of a preset acquisition queue, and splicing the video stream of the main user client to obtain a panoramic view includes

[0070] Obtain the device ID, user identity identifier, camera number, and corresponding video stream of each joined client;

[0071] Preferentially sort the video stream of the client whose user identity identifier is consistent with the main user identifier to the head of the acquisition queue;

[0072] According to the device ID and the order of joining the meeting, the video streams of the same device ID that joined the meeting earlier are sorted in ascending order and spliced according to the camera numbers to obtain the panoramic views of each primary user client.

[0073] Refer to Figure 2 , in the scenario of a multi-person conference call, multiple clients can join a meeting simultaneously, and a meeting can have multiple screens set, such as Screen 1, Screen 2, and Screen 3. Each client can access multiple cameras. Among them, a client that accesses two or more cameras is called a telepresence user, and when displaying the picture, the images captured by the cameras of this client need to be joined into a single panoramic view for overall display.

[0074] Specifically, before the client joins the meeting, the cameras are numbered in the order of placement first, such as camera1, camera2, camera3, ….

[0075] When joining the meeting, the primary user joins first, and the primary user will carry information when joining, such as userid = 1, index = camera1, parentid = 1. Among them, userid is the unique ID assigned by the meeting control server to this user (client), index is the camera number, representing the sequence number of the video stream, and parentid is the unique ID of the belonging user. By judging whether the parentid field is greater than 0, it is associated with the corresponding primary user. If the parentid is not greater than 0, it means that this user is not a primary user.

[0076] A telepresence user can carry multiple video streams to join the meeting, and each video stream has a sequence number and belongs to information. Such as index = camera1 + camera2 + camera3 + … + camera n, and the sequence numbers increase sequentially.

[0077] According to the number of cameras, the secondary users join the meeting in turn. The information carried by the secondary users when joining the meeting is, for example, userid = 2, index = camera2; userid = 3, index = camera3; ….

[0078] The logic for the client to join the meeting is that the primary user must join successfully first before the secondary users can join.

[0079] After the client joins the meeting, it can obtain the information queue of all the joining clients from the meeting control server, such as userid = 1, index = camera1; userid = 2, index = camera2, parentid = 1; userid = 3, index = camera3, parentid = 1; ….

[0080] Sort the video streams of the clients according to the information queues of the clients that have joined the meeting: First, move the video streams of the clients with non-empty parentid to the head of the preset acquisition queue. If the parentids are the same, sort them in ascending order according to the index to form a new queue where the video streams of multi-camera users (TelePresence users) are in the front row of the acquisition queue.

[0081] Finally, perform an equal distribution layout on the sorted video streams in the acquisition queue to control the panoramic views of each primary user client and the video streams of each non-primary user client to be centered on the screen of the multi-screen device.

[0082] The equal distribution layout for the panoramic views of the primary user clients includes:

[0083] Obtain the total number of cameras accessed by any primary user client;

[0084] Divide each screen into n equal parts, where n is a positive integer;

[0085] According to the equal division situation, create n x n galleries for each screen and sort the galleries by row;

[0086] Select the panoramic view of the first primary user client that joined the meeting from the sorted video streams in the acquisition queue and insert it at the head of the first row of galleries;

[0087] Based on the total number of cameras accessed by the primary user client, the total number of galleries, and the current video stream length value in the gallery, calculate the current offset value;

[0088] Move the panoramic view in the first row of galleries as a whole by the current offset value;

[0089] Detect all galleries by row to find the target queue with the minimum length. Select the panoramic views of the remaining primary user clients from the sorted video streams in the acquisition queue and insert them at the preset positions of the current video streams in the target queue in sequence. Repeat calculating the current offset value of the target queue and move the current video stream as a whole by the current offset value until the panoramic views of each primary user client are centered on the screen of the multi-screen device;

[0090] Among them, when the panoramic view is inserted at the head of an empty target queue, move the inserted panoramic view as a whole by the current offset value of the gallery.

[0091] In an embodiment, the step of selecting the panoramic views of the remaining primary user clients from the sorted video streams in the acquisition queue and inserting them at the preset positions of the current video streams in the target queue in sequence includes:

[0092] Configure a variable for each target queue and the initial value of the variable is 0. When a panoramic image is inserted into the target queue, the variable value of the target queue changes iteratively in the order of 1, 0;

[0093] When inserting the panoramic image corresponding to the primary user client into the target queue, first detect the variable value of the target queue;

[0094] If the variable value of the target queue is 0, splice the panoramic image to the end position of the current video stream;

[0095] If the variable value of the target queue is 1, splice the panoramic image to the head position of the current video stream.

[0096] In one embodiment, the step of controlling the panoramic images of each primary user client and the video streams of each non-primary user client to be centered on the screen of the multi-screen device for the video stream based on the sorting of the acquisition queue further includes,

[0097] Detect all galleries row by row to find the target queue with the minimum length. From the sorted video stream of the acquisition queue, sequentially select the video streams corresponding to non-primary user clients and insert them into the preset position of the current video stream of the target queue; wherein, when the video stream is inserted into the head of the empty target queue, move the inserted video stream as a whole by the current offset value of the gallery;

[0098] After all the video streams corresponding to non-primary user clients are inserted, calculate the current offset value of each gallery, and move the current video stream of each gallery as a whole by the corresponding current offset value to complete the control of centering the panoramic images of each primary user client and the video streams of each non-primary user client on the screen of the multi-screen device.

[0099] In one embodiment, the step of sequentially selecting the video streams corresponding to non-primary user clients from the sorted video stream of the acquisition queue and inserting them into the preset position of the current video stream of the target queue includes,

[0100] When a video stream is inserted into the target queue, the variable value of the target queue changes iteratively in the order of 1, 0;

[0101] When inserting the video stream corresponding to the non-primary user client into the target queue, first detect the variable value of the target queue;

[0102] If the variable value of the target queue is 0, splice the video stream to the end position of the current video stream;

[0103] If the variable value of the target queue is 1, splice the video stream to the head position of the current video stream.

[0104] In one embodiment, when calculating the current offset value based on the total number of cameras accessed by the primary user client, the total number of galleries, and the current video stream length value in the gallery, the following formula is used:

[0105] Offset value = Round((m * n - queue_max_size) / n)

[0106] In the formula, Round() represents the rounding operation, m is the total number of cameras accessed by any primary user client, n is the total number of galleries, and queue_max_size is the current video stream length value in the gallery.

[0107] For example, when there are 3 screens and 5 clients, and 1 client accesses 3 cameras, the information is as follows: userid = 1, index = camera1; userid = 2, parentid = 1, index = camera2;

[0108] userid = 3, parentid = 3, parented = 2, index = camera3.

[0109] Each camera is divided into two equal parts to create 2X2 queues. Each queue has a variable, and when a picture is inserted once, the variable value changes, such as cycling between 1 and 0. The starting value of the variable can be set to 0. When inserted once, the variable value becomes 1, and when inserted again, the variable value becomes 0. 0 means the next insertion needs to be from the left side of the queue, and 1 means the next insertion needs to be from the right side of the queue. If a panoramic picture is inserted, the variable value only changes once.

[0110] When arranging, start from the left side of the topmost queue and arrange downwards in turn until all queues are arranged with pictures. Each time when arranging, find the queue with the shortest length in the order from top to bottom.

[0111] When arranging, the order of the video pictures appearing is: arranged vertically from top to bottom, and horizontally spread from the middle to both sides. When spreading horizontally, it can be arranged from the left side first and then the right side, or from the right side first and then the left side. Here, the example of arranging from the left side first and then the right side is used for illustration.

[0112] First, splice the video streams of the clients of 3 cameras into a panoramic picture and arrange it at the head of the first row of galleries, that is, at the head of the first target queue, and move the whole panoramic picture by the current offset value to the center position of 3 screens.

[0113] Then allocate the video stream of the second client to the head of the second row of galleries, that is, the second target queue.

[0114] Reallocate the video stream of the third client to the right of the video stream of the second client in the second row of the gallery, that is, the end of the current video stream in the second target queue.

[0115] Immediately allocate the video stream of the fourth client to the second row of the gallery. Since it is necessary to insert it at the head position of the current video stream in the second target queue at this time, it is necessary to move the current video stream of the second row of the gallery as a whole by the current offset value to the center position of 3 screens, and then insert the video stream of the fourth client to the left of the video streams of the second and third clients in the second row of the gallery, that is, the head of the current video stream in the second target queue. Because the video stream was initially arranged starting from the left side, when it is necessary to insert a video stream from the left side again, the current video stream in the queue needs to be moved as a whole to the right to vacate the position where the video stream needs to be inserted this time, that is, 1 grid for non-primary user clients.

[0116] Finally, allocate the video stream of the fifth client to the right of the panorama of the first client in the first row of the gallery, that is, the end of the current video stream in the first target queue.

[0117] After all the video streams of non-primary user clients are inserted, perform a centralized centering adjustment of the current video streams in each queue again to complete the centering display control of the panoramas of each primary user client and the video streams of each non-primary user client on the screen of the multi-screen device.

[0118] If a primary user client joins during the layout, give priority to the layout, refresh the screen for arrangement, and recalculate the current offset value of the target queue where the panorama is inserted, and move the current video stream of the target queue as a whole by the current offset value.

[0119] Repeat until the panoramas of each primary user client are centered on the screen of the multi-screen device.

[0120] If a user leaves during the arrangement, the overall layout of the video stream remains unchanged, and the leaving position is set to display black.

[0121] The logic of the arrangement is to start arranging from the leftmost side of the first queue and go down in turn until all queues have pictures arranged.

[0122] Next, judge whether the queues are neat, that is, whether there is a regular rectangle; if not, first judge which queue has the shortest current length, insert a video stream to supplement it until the queues are neat, and then start supplementing from the left side of the queue (when supplementing from the left side, the original picture is moved as a whole by the width of the picture supplemented on the left; if the supplemented picture is a single picture, that is, move one grid to the right as a whole, and the previously supplemented picture is supplemented in the first queue on the rightmost side, and then the picture order is arranged downwards).

[0123] After the arrangement, it is still necessary to determine whether the queue is neat. Because there will be a situation where the leftmost first queue to be supplemented is a long picture with two pictures, and the second queue is a single-picture one. After the queue is supplemented, the whole is not a rectangle, and the queue needs to be filled. However, the filling direction is the left side. After the left side is filled, continue to arrange to the right side of the whole picture.

[0124] If there are m screens, each screen is divided into n equal parts, and there are x client pictures to be laid out, the overall layout steps are as follows:

[0125] Step 1: Create n queues according to the equal division situation and sort them by row.

[0126] Step 2: Search for the target queue with the smallest queue length from top to bottom and insert the picture (video stream). If the previous video stream in this column was inserted in the front of the queue, the next video stream will be inserted at the back of the queue. The default for the first insertion can be configured to be inserted in the front / back.

[0127] Step 3: Calculate the offset value = Round((m * n - queue_max_size) / n), and move the video streams in the queue to the middle position of each screen as a whole for centered display.

[0128] By adjusting the camera angle + cropping, adjusting the optical zoom + offset, a panoramic view can be obtained without using the digital zoom, which can make the displayed picture clearer.

[0129] When all clients are non-primary user clients, such as 7 non-primary user clients and 3 screens, each screen is divided into 2X2 equal parts, and the effect after equal distribution layout is as Figure 3 shown.

[0130] In summary, a display control method for a multi-screen device prioritizes accepting the joining requests of the primary user clients and sorts the video streams of the joined clients, preferentially sorting the video stream of the primary user client to the head of a preset capture queue, which is conducive to achieving that the video stream of the telepresence user is preferentially arranged in the middle position of the screen. By controlling the panoramic views of the primary user clients and the video streams of the non-primary user clients to be centered on the screen of the multi-screen device, compared with the traditional tiled layout method (when there are fewer video images, the windows will be concentrated on the first screen, which is not aesthetically pleasing and not very applicable in the field of video calls), this solution spreads the display images from the middle of the screen to both sides, enabling the display images to be concentrated in the middle position of the screen rather than on the left or right side, thereby reducing the problem that the color and brightness performance of the window display images of the joined clients are poor due to light interference around the screen, improving the window display clarity of the joined clients. At the same time, the overall centering of the display images can be located in the central area of the user's field of vision, making the user's viewing experience better, especially suitable for ultra-large screens.

[0131] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0132] The embodiments of the present application further provide a display control device for a multi-screen device, which corresponds one-to-one with the display control method for a multi-screen device in the above embodiments. The display control device for the multi-screen device includes:

[0133] A primary user identification module, configured to determine whether each client is a primary user when receiving joining requests from multiple clients;

[0134] A primary user joining module, configured to preferentially accept the joining request of the client when determining that the client is a primary user, and the client is connected to at least two cameras;

[0135] A panoramic view module, configured to sort the video streams of the joined clients, preferentially sort the video stream of the primary user client to the head of a preset capture queue, and splice the video stream of the primary user client to obtain a panoramic view;

[0136] A display control module, configured to control the panoramic views of the primary user clients and the video streams of the non-primary user clients to be centered on the screen of the multi-screen device based on the sorted video streams in the capture queue.

[0137] For the specific limitations on a display control device for a multi-screen device, reference can be made to the limitations on a display control method for a multi-screen device in the above text, which will not be elaborated here.

[0138] Each module in the display control device of the above multi-screen device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0139] In one embodiment, a computer device is provided, and the computer device can be a server. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements any one of the above display control methods for the multi-screen device.

[0140] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements any one of the above display control methods for the multi-screen device.

[0141] In one embodiment, a computer program product is provided, and the computer program product includes a computer program. When the computer program is executed by the processor, it implements any one of the above display control methods for the multi-screen device.

[0142] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0143] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A display control method for a multi-screen device, characterized in that, including the following steps, when receiving the membership requests of multiple clients, determining whether each client is a primary user; if the client is a primary user, preferentially accepting the membership request of the client, and the client accessing at least two cameras; sorting the video streams of the clients that have joined the meeting, preferentially sorting the video stream of the primary user client to the head of a preset acquisition queue, and stitching the video streams of the primary user client to obtain a panoramic view; based on the sorted video streams in the acquisition queue, controlling the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device.

2. The display control method of the multi-screen device according to claim 1, wherein The step of controlling the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device based on the sorted video streams in the acquisition queue includes, obtaining the total number of cameras accessed by any primary user client; equally dividing each screen into n parts, where n is a positive integer; creating n x n galleries for each screen according to the equal division situation; selecting the panoramic view of the first primary user client that joined the meeting from the sorted video streams in the acquisition queue and inserting it at the head of the first row of galleries; calculating the current offset value based on the total number of cameras accessed by the primary user client, the total number of galleries, and the current video stream length value in the gallery; moving the panoramic view in the first row of galleries as a whole by the current offset value; detecting all galleries row by row to find the target queue with the minimum length, and successively selecting the panoramic views corresponding to the remaining primary user clients from the sorted video streams in the acquisition queue and inserting them at the preset position of the current video stream in the target queue, and repeating the calculation of the current offset value of the target queue and moving the current video stream as a whole by the current offset value until the panoramic views of each primary user client are centered and displayed on the screen of the multi-screen device; wherein, when the panoramic view is inserted at the head of an empty target queue, moving the inserted panoramic view as a whole by the current offset value of the target queue.

3. The display control method of the multi-screen device according to claim 2, wherein The step of successively selecting the panoramic views corresponding to the remaining primary user clients from the sorted video streams in the acquisition queue and inserting them at the preset position of the current video stream in the target queue includes, configuring a variable for each target queue and the initial value of the variable is 0 for all. When a panoramic view is inserted into a target queue, the variable value of the target queue iteratively changes in the order of 1, 0; when inserting the panoramic view corresponding to the primary user client into the target queue, first detecting the variable value of the target queue; if the variable value of the target queue is 0, stitching the panoramic view to the end position of the current video stream; if the variable value of the target queue is 1, stitching the panoramic view to the head position of the current video stream.

4. The display control method of the multi-screen device according to claim 3, characterized in that, The step of controlling the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device based on the sorted video streams in the acquisition queue further includes, Detect all galleries row by row to find the target queue with the minimum length. From the sorted video streams in the acquisition queue, sequentially select the video streams corresponding to non-primary user clients and insert them into the preset position of the current video stream in the target queue. When a video stream is inserted at the head of an empty target queue, move the inserted video stream as a whole by the current offset value of the gallery. After all the video streams corresponding to non-primary user clients are inserted, calculate the current offset value of each gallery, and move the current video stream of each gallery by the corresponding current offset value to complete the centering display control of the panorama of each primary user client and the video streams of each non-primary user client on the screen of the multi-screen device.

5. The display control method of the multi-screen device according to claim 4, wherein The step of sequentially selecting the video streams corresponding to non-primary user clients from the sorted video streams in the acquisition queue and inserting them into the preset position of the current video stream in the target queue includes When a video stream is inserted into the target queue, the variable value of the target queue iteratively changes in the order of 1 and 0. When inserting the video stream corresponding to a non-primary user client into the target queue, first detect the variable value of the target queue. If the variable value of the target queue is 0, splice the video stream to the end position of the current video stream. If the variable value of the target queue is 1, splice the video stream to the head position of the current video stream.

6. The display control method of the multi-screen device according to claim 2, characterized in that, When calculating the current offset value based on the total number of cameras accessed by the primary user client, the total number of galleries, and the length value of the current video stream in the gallery, the following formula is used: Offset value = Round((m * n - queue_max_size) / n) In the formula, Round() represents the rounding operation, m is the total number of cameras accessed by any primary user client, n is the total number of galleries, and queue_max_size is the length value of the current video stream in the gallery.

7. The display control method of the multi-screen device according to claim 1, wherein The step of determining whether each client is a primary user when receiving the joining requests of multiple clients includes Detect the number of input video streams. When the number of video streams is greater than or equal to 2, determine that the client is a primary user.

8. The display control method of the multi-screen device according to claim 1, characterized in that, The step of sorting the video streams of each joined client, preferentially sorting the video stream of the primary user client to the head of the preset acquisition queue, and splicing the video streams of the primary user client to obtain a panorama includes Obtain the device ID, user identity identifier, camera number, and corresponding video stream of each joined client. Preferentially sort the video stream of the client whose user identity identifier is the same as the primary user identifier to the head of the acquisition queue. According to the device ID and the joining order, sort and splice the video streams of the same device ID that joined first in ascending order of the camera number to obtain the panorama of each primary user client.

9. A display control device for a multi-screen device, characterized in that, including A primary user identification module for determining whether each client is a primary user when receiving the joining requests of multiple clients. A primary user joining module for preferentially accepting the joining request of the client when it is determined that the client is a primary user, and the client accesses at least two cameras. A panoramic view module, which is used to sort the video streams of each client joining the meeting, preferentially sort the video stream of the primary user client to the head of a preset acquisition queue, and splice the video stream of the primary user client to obtain a panoramic view; A display control module, which is used to control the panoramic views of each primary user client and the video streams of each non-primary user client to be centered and displayed on the screen of the multi-screen device based on the sorted video streams in the acquisition queue.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

Images