A screen combination method and device

Abstract

An embodiment of the present application provides a screen combination method and device, which relate to the field of terminals and can automatically complete screen splicing to improve the user experience. The embodiment of the present application is applied to a screen splicing system, which includes at least two screens and a host. The at least two screens include a first screen and a second screen. The method includes: the first screen and the second screen form a first screen group, and the first screen and the second screen are communicatively connected; the host sends a first instruction to the first screen and a second instruction to the second screen; the first screen captures a first image according to the first instruction; the second screen captures a second image according to the second instruction; the orientation information of the first screen and the second screen is determined based on the first image and the second image.

Description

Technical Field

[0001] This application relates to the field of terminals, and in particular, to a screen combination method and apparatus. Background Art

[0002] With the widespread application of network technology, from the establishment of command and monitoring centers and network management centers to the holding of temporary meetings and technical lectures, a larger display effect is required. To achieve this effect, the screens of multiple screen-equipped devices can be spliced together to meet the requirement of a larger display effect.

[0003] When combining multiple screens, it is necessary to determine the position and orientation information of each screen. Currently, the orientation and position information of the screen can be input into the control host by manual input of the user. For example, in the scenario of combining multiple monitors connected to a computer host, the positions of the monitors can be manually marked to complete the setting of the screen position and orientation relationship of the combination of multiple monitors. As Figure 1A shown, in the setting interface, the monitors can be dragged and marked manually (by mouse or touch screen) to complete the orientation combination of multiple monitors.

[0004] In the above method, the setting of the position and orientation information in the screen combination relationship requires manual input by the user, and the combination process is complex and the steps are cumbersome, resulting in a poor user experience. Summary of the Invention

[0005] Embodiments of this application provide a screen combination method and apparatus, which can automatically complete screen splicing and improve the user experience.

[0006] In a first aspect, an embodiment of this application provides a screen combination method, which is applied to a screen splicing system. The screen splicing system includes at least two screens and a host. The at least two screens include a first screen and a second screen, and the method includes: the first screen and the second screen form a first screen group, and the first screen and the second screen are communicatively connected; the host sends a first instruction to the first screen and a second instruction to the second screen; the first screen captures a first image according to the first instruction; the second screen captures a second image according to the second instruction; and the orientation information of the first screen and the second screen is determined based on the first image and the second image.

[0007] Based on the method provided by the embodiment of this application, during the screen combination and splicing process, the relative orientation relationship between the two devices can be identified according to the images (photos) captured by the devices (the first screen or the second screen), without manual setting by the user, which can improve the user experience.

[0008] In a possible implementation, the host is integrated into the first screen or the second screen; or the host is independent of the first screen or the second screen. Exemplarily, the first screen or the second screen may be a television. When the host is independent of the first screen or the second screen, the host may be a device such as a set-top box or a router. When the host is integrated into the first screen or the second screen, the host can be regarded as a processing module in the first screen or the second screen.

[0009] In a possible implementation, determining the orientation information of the first screen and the second screen according to the first image and the second image includes: the first screen sends the first image to the second screen; the second screen sends the second image to the first screen; the first screen and the second screen respectively determine the orientation information of the first screen and the second screen according to the first image and the second image; the first screen sends the orientation information determined by the first screen to the host, and the second screen sends the orientation information determined by the second screen to the host; the host determines the orientation information of the first screen and the second screen according to the orientation information determined by the first screen and the orientation information determined by the second screen. In some cases, the orientation information between some devices is redundant, and these information can be not used, or the recognition result can be verified with reference to the redundant information.

[0010] In a possible implementation, determining the orientation information of the first screen and the second screen according to the first image and the second image includes: the first screen sends the first image to the host; the second screen sends the second image to the host; the host determines the orientation information of the first screen and the second screen according to the first image and the second image. That is, the host can identify the orientation relationship of each device in the screen group, and other devices (for example, the second screen) do not need to perform orientation recognition, which can save the power consumption of other devices.

[0011] In a possible implementation, before the first screen and the second screen form the first screen group, the method further includes: the first screen or the second screen sends the first short-range signal to each other at a preset frequency, and the first screen or the second screen determines the distance between the first screen and the second screen according to the received signal strength indication (RSSI) of the first short-range signal transmitted between the first screen and the second screen; when the distance between the first screen and the second screen is less than or equal to the maximum combination radius corresponding to the first screen and the second screen, the first screen and the second screen form the first screen group; wherein, the maximum combination radius corresponding to the first screen and the second screen is determined according to the sizes of the first screen and the second screen and the positions of the antennas. In this way, the first screen and the second screen can determine whether to perform screen combination according to the first short-range signal, that is, the screen combination can be automatically performed without complex operations by the user, which can improve the user experience.

[0012] In a possible implementation, before the first screen and the second screen form the first screen group, the method further includes: the first screen and / or the second screen display a first prompt message for prompting the user that a device is detected nearby and asking whether to perform screen splicing, and the first screen and / or the second screen obtain an instruction from the user, where the instruction from the user is used to confirm performing screen splicing. In this way, it is possible to determine whether to form a screen group according to the user's operation, thereby avoiding the error of automatically triggering screen combination.

[0013] In a possible implementation, determining the orientation information of the first screen and the second screen according to the first image and the second image includes: performing image matching on the first image and the second image according to an image matching algorithm to determine the overlapping area of the first image and the second image; determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image and the orientation in the second image. In this way, by determining the orientation of the overlapping area in different photos, the relative orientation relationship between the two devices (the first screen and the second screen) can be recognized without manual setting by the user, which can improve the user experience.

[0014] In a possible implementation, determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image includes: if the overlapping area is located in the lower half of the first image and in the upper half of the second image, it is determined that the first screen is above the second screen; if the overlapping area is located in the lower left corner of the first image and in the upper right corner of the second image, it is determined that the first screen is in the upper right of the second screen; if the overlapping area is located in the left half of the first image and in the right half of the second image, it is determined that the first screen is to the right of the second screen; if the overlapping area is located in the upper left corner of the first image and in the lower right corner of the second image, it is determined that the first screen is in the lower right of the second screen; if the overlapping area is located in the upper half of the first image and in the lower half of the second image, it is determined that the first screen is below the second screen; if the overlapping area is located in the upper right corner of the first image and in the lower left corner of the second image, it is determined that the first screen is in the lower left of the second screen; if the overlapping area is located in the right half of the first image and in the left half of the second image, it is determined that the first screen is to the left of the second screen; if the overlapping area is located in the lower right area of the first image and in the upper left area of the second image, it is determined that the first screen is in the upper left of the second screen. In this way, by determining the orientation of the overlapping area in different photos, the relative orientation relationship between the two devices (the first screen and the second screen) can be recognized without manual setting by the user, which can improve the user experience.

[0015] In a possible implementation, the image matching algorithm includes at least one of the Scale-Invariant Feature Transform (SIFT) algorithm, Speeded-Up Robust Features (SURF) algorithm, and Fast Nearest Neighbor Search (FLANN) algorithm. Of course, the image matching algorithm can be other algorithms, which are not limited in this application.

[0016] In a possible implementation, determining the orientation information of the first screen and the second screen based on the first image and the second image includes: if it is determined that the first image and the second image include a target object, determining the orientation of the first screen relative to the second screen according to the orientation of the target object in the first image and the second image; wherein, the target object includes any one of a human face, a human body movement, or a furniture item. In this way, by determining the orientation of the target object in different photos, the relative orientation relationship between the two devices (the first screen and the second screen) can be identified without manual setting by the user, which can improve the user experience.

[0017] In a possible implementation, determining the orientation of the first screen relative to the second screen according to the orientation of the target object in the first image and the second image includes: if the target object is located in the lower half of the first image and in the upper half of the second image, determining that the first screen is above the second screen; if the target object is located in the lower left corner of the first image and in the upper right corner of the second image, determining that the first screen is in the upper right of the second screen; if the target object is located in the left half of the first image and in the right half of the second image, determining that the first screen is to the right of the second screen; if the target object is located in the lower left corner of the first image and in the lower right corner of the second image, determining that the first screen is to the right of the second screen; if the target object is located in the upper left corner of the first image and in the upper right corner of the second image, determining that the first screen is to the right of the second screen; if the target object is located in the upper left corner of the first image and in the lower right corner of the second image, determining that the first screen is in the lower right of the second screen; if the target object is located in the upper half of the first image and in the lower half of the second image, determining that the first screen is below the second screen; if the target object is located in the upper right corner of the first image and in the lower left corner of the second image, determining that the first screen is in the lower left of the second screen; if the target object is located in the right half of the first image and in the left half of the second image, determining that the first screen is to the left of the second screen; if the target object is located in the lower right corner of the first image and in the lower left corner of the second image, determining that the first screen is to the left of the second screen; if the target object is located in the upper right corner of the first image and in the upper left corner of the second image, determining that the first screen is to the left of the second screen; if the target object is located in the lower right area of the first image and in the upper left area of the second image, determining that the first screen is in the upper left of the second screen. In this way, by determining the orientation of the target object in different photos, the relative orientation relationship between the two devices (the first screen and the second screen) can be identified without manual setting by the user, which can improve the user experience.

[0018] In a possible implementation, before the first screen captures a first image according to a first instruction and the second screen captures a second image according to a second instruction, the method further includes: the host sends layout information to the first screen and the second screen, where the layout information includes at least one combination mode; in response to an operation by the user to select one combination mode from the at least one combination mode, the host sends operation information to the first screen and the second screen, and the first screen and / or the second screen instructs the user to perform a first gesture or action at a first position and a second gesture or action at a second position according to the operation information; determining the orientation information of the first screen and the second screen based on the first image and the second image includes: if it is determined that the area of the first gesture or action in the first image is greater than or equal to a preset threshold, determining that the first screen is located at the first position; if it is determined that the area of the second gesture or action in the second image is greater than or equal to a preset threshold, determining that the second screen is located at the second position. This solution for determining the orientation of the device based on the user's gestures can improve the user's participation and interest during the screen splicing process.

[0019] In a possible implementation, the host is integrated into the first screen or the second screen, and the first screen and the second screen form a first screen group. The method further includes: the first screen or the second screen rates the resource status of the first screen and the second screen; where the resource status includes at least one of the processing capabilities of the central processing unit (CPU), the storage capabilities of the read-only memory (ROM), or the random access memory (RAM); if the rating of the first screen is higher than the rating of the second screen, the host is integrated into the first screen; if the rating of the second screen is higher than the rating of the first screen, the host is integrated into the second screen. When the host is integrated into the first screen, the first screen can be considered the main device, and when the host is integrated into the second screen, the second screen can be considered the main device.

[0020] In a possible implementation, the method further includes: the host determines the display information corresponding to the first screen and the second screen according to the orientation information of the first screen and the second screen; the host sends the display information corresponding to the first screen to the first screen; the first screen displays the corresponding display screen according to the display information corresponding to the first screen; the host sends the display information corresponding to the second screen to the second screen; after receiving the display information corresponding to the second screen, the second screen displays the corresponding display screen according to the display information corresponding to the second screen. In this way, the first screen and the second screen can display the corresponding display screens according to the display information determined by the host, and can meet the display effect of a larger screen.

[0021] In a possible implementation, the screen splicing system further includes a third screen, and the method further includes: the first screen and the third screen send second short-range signals to each other; the second screen and the third screen send third short-range signals to each other; determining the distance between the first screen and the third screen according to the RSSI of the second short-range signal; determining the distance between the second screen and the third screen according to the RSSI of the third short-range signal; when the distance between the first screen and the third screen is less than or equal to the maximum combination radius corresponding to the first screen and the third screen, forming a second screen group with the first screen, the second screen and the third screen; wherein, the maximum combination radius corresponding to the first screen and the third screen is determined according to the sizes of the first screen and the third screen and the positions of the antennas; or when the distance between the second screen and the third screen is less than or equal to the maximum combination radius corresponding to the second screen and the third screen, forming a second screen group with the first screen, the second screen and the third screen; wherein, the maximum combination radius corresponding to the second screen and the third screen is determined according to the sizes of the second screen and the third screen and the positions of the antennas. In this way, it can be determined whether to perform screen combination between the first screen and the third screen according to the second short-range signal; or it can be determined whether to perform screen combination between the second screen and the third screen according to the third short-range signal, without the need for complex operations by the user, and the user experience can be improved.

[0022] In a possible implementation, the method further includes: the first screen and / or the second screen display a second prompt message for prompting the user that a new device is detected and asking whether to perform screen splicing; the first screen and / or the second screen obtain an instruction from the user, and the instruction from the user is used to confirm performing screen splicing. In this way, it can be determined whether to form a screen group according to the operation of the user, thereby avoiding errors in automatically triggering screen combination.

[0023] In a possible implementation, if the first condition is satisfied, the method further includes: the first screen and / or the second screen detect whether the first condition is satisfied; if the first condition is satisfied, the first screen and / or the second screen remove the third screen from the second screen group. That is, the first screen or the second screen can automatically detect whether a screen (such as the third screen) is removed, and then prompt the user so that the user can always understand the situation of the screen group.

[0024] In a possible implementation, the first condition includes: the heartbeat connection between the third screen and the first screen is disconnected, or the heartbeat connection between the third screen and the second screen is disconnected; or the host receives an operation from the user to delete the third screen; or the distance between the first screen and the third screen is greater than the maximum combination radius corresponding to the first screen and the third screen; or the distance between the second screen and the third screen is greater than the maximum combination radius corresponding to the second screen and the third screen.

[0025] In a possible implementation, the method further includes: the host re-determines the display information corresponding to the first screen and the second screen according to the orientation information of the first screen and the second screen. That is, the host can adaptively adjust the display information of the screen group according to the changes of the devices in the screen group.

[0026] In a possible implementation, the method further includes: the host sends a third instruction to the third screen, a fourth instruction to the first screen, and a fifth instruction to the second screen; the third screen captures a third image according to the third instruction; the first screen captures a fourth image according to the fourth instruction; the second screen captures a fifth image according to the fifth instruction; the third screen sends the third image to the host; the second screen sends the fifth image to the host; after receiving the fourth image and the fifth image, the host determines the orientation information of the first screen, the second screen, and the third screen according to the third image, the fourth image, and the fifth image. That is, when a new device is added to the screen group, it can be considered that the screen group has been reorganized, and the devices in the screen group can be made to capture images again to re-determine the relative orientation relationship of the devices in the screen group.

[0027] In a possible implementation, the third screen is removed from the second screen group, and the method further includes: the host sends a sixth instruction to the first screen and a seventh instruction to the second screen; the first screen captures a sixth image according to the sixth instruction; the second screen captures a seventh image according to the seventh instruction; the first screen sends the sixth image to the host; the second screen sends the seventh image to the host; the host determines the orientation information of the first screen and the second screen according to the sixth image and the seventh image. That is, when a device is removed from the screen group, it can be considered that the screen group has been reorganized, and the devices in the screen group can be made to capture images again to re-determine the relative orientation relationship of the devices in the screen group.

[0028] In a second aspect, an embodiment of the present application provides a screen combination method applied to a screen splicing system. The screen splicing system includes at least two screens and a host. The at least two screens include a first screen and a second screen. The first screen and the second screen form a first screen group, and the first screen and the second screen are communicatively connected. The method includes: the host sends a first instruction to the first screen, and the first instruction is used to instruct the first screen to capture a first image; the host sends a second instruction to the second screen, and the second instruction is used to instruct the second screen to capture a second image; the host determines the orientation information of the first screen and the second screen according to the first image and the second image.

[0029] Based on the method provided by the embodiment of the present application, during the screen combination and splicing process, the relative orientation relationship between two devices can be recognized according to the images (photos) captured by the devices (the first screen or the second screen), without manual setting by the user, which can improve the user experience. Moreover, the embodiment of the present application can dynamically monitor the distance between devices, automatically recognize the combination intention between devices and start the screen assembly program, without manual setting by the user, which is more intelligent and convenient.

[0030] In a possible implementation, the host is integrated into the first screen or the second screen; or the host is independent of the first screen or the second screen.

[0031] In a possible implementation, the host determines the orientation information of the first screen and the second screen based on the first image and the second image, including: the host receives the first image from the first screen; the host receives the second image from the second screen; the host determines the orientation information of the first screen and the second screen based on the first image and the second image.

[0032] In a possible implementation, determining the orientation information of the first screen and the second screen based on the first image and the second image includes: the host performs image matching on the first image and the second image according to an image matching algorithm to determine the overlapping area of the first image and the second image; determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image and the orientation of the second image.

[0033] In a possible implementation, determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image includes: if the overlapping area is located in the lower half of the first image and in the upper half of the second image, it is determined that the first screen is above the second screen; if the overlapping area is located in the lower left corner of the first image and in the upper right corner of the second image, it is determined that the first screen is in the upper right of the second screen; if the overlapping area is located in the left half of the first image and in the right half of the second image, it is determined that the first screen is to the right of the second screen; if the overlapping area is located in the upper left corner of the first image and in the lower right corner of the second image, it is determined that the first screen is in the lower right of the second screen; if the overlapping area is located in the upper half of the first image and in the lower half of the second image, it is determined that the first screen is below the second screen; if the overlapping area is located in the upper right corner of the first image and in the lower left corner of the second image, it is determined that the first screen is in the lower left of the second screen; if the overlapping area is located in the right half of the first image and in the left half of the second image, it is determined that the first screen is to the left of the second screen; if the overlapping area is located in the lower right area of the first image and in the upper left area of the second image, it is determined that the first screen is in the upper left of the second screen.

[0034] In a possible implementation, the image matching algorithm includes at least one of the Scale-Invariant Feature Transform (SIFT) algorithm, the Speeded-Up Robust Features (SURF) algorithm, and the Fast Nearest Neighbor Search (FLANN) algorithm.

[0035] In a possible implementation, determining the orientation information of the first screen and the second screen based on the first image and the second image includes: if it is determined that the first image and the second image include a target object, determining the orientation of the first screen relative to the second screen according to the orientation of the target object in the first image and the second image.

[0036] In a possible implementation, before the first screen captures a first image according to a first instruction and the second screen captures a second image according to a second instruction, the method further includes: the host sends layout information to the first screen and the second screen, where the layout information includes at least one combination mode; in response to an operation in which the user selects one combination mode from the at least one combination mode, the host sends operation information to the first screen and the second screen, where the operation information is used to instruct the user to perform a first gesture or action at a first position and a second gesture or action at a second position; determining the orientation information of the first screen and the second screen based on the first image and the second image includes: if it is determined that the area of the first gesture or action included in the first image is greater than or equal to a preset threshold, determining that the first screen is located at the first position; if it is determined that the area of the second gesture or action included in the second image is greater than or equal to a preset threshold, determining that the second screen is located at the second position.

[0037] In a possible implementation, the method further includes: the host determines the display information corresponding to the first screen and the second screen respectively according to the orientation information of the first screen and the second screen; the host sends the display information corresponding to the first screen to the first screen; the host sends the display information corresponding to the second screen to the second screen.

[0038] The beneficial effects of the various implementations in the second aspect can refer to the beneficial effects of the corresponding implementations in the first aspect, which will not be elaborated here.

[0039] In a third aspect, an embodiment of the present application provides an electronic device, which may be the first screen or the second screen. The electronic device includes: a wireless communication module, a memory, and one or more processors; the wireless communication module, the memory are coupled to the processor; wherein, the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method described in the first aspect or the second aspect and any of its possible implementations.

[0040] In a fourth aspect, an embodiment of the present application provides a chip system, which includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected by lines. The above chip system can be applied to an electronic device (such as the first screen or the second screen) including a communication module and a memory. The interface circuit is used to receive a signal from the memory and send the received signal to the processor, and the signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device can execute the method described in any aspect and any of its possible implementations.

[0041] Fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions run on an electronic device (for example, the first screen or the second screen), the electronic device is caused to execute the method described in the first aspect and any possible implementation manner thereof.

[0042] Sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the method described in the first aspect or the second aspect and any possible implementation manner thereof.

[0043] Seventh aspect, an embodiment of the present application provides a software upgrade system, which includes a first screen, a second screen, and a host. The first screen, the second screen, and the host can execute the method described in the first aspect and any possible implementation manner thereof. Description of the Drawings

[0044] Figure 1A It is a schematic diagram of a display interface of a screen combination in the prior art;

[0045] Figure 1B It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0046] Figure 1C It is another schematic diagram of a system architecture provided by an embodiment of the present application;

[0047] Figure 2A It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0048] Figure 2B It is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0049] Figure 3A It is a schematic diagram of the connection of multiple devices provided by an embodiment of the present application;

[0050] Figure 3B It is another schematic diagram of the connection of multiple devices provided by an embodiment of the present application;

[0051] Figure 3C It is a schematic diagram of a display provided by an embodiment of the present application;

[0052] Figure 3D It is another schematic diagram of a display provided by an embodiment of the present application;

[0053] Figure 3E It is another schematic diagram of a display provided by an embodiment of the present application;

[0054] Figure 3F It is another schematic diagram of a display provided by an embodiment of the present application;

[0055] Figure 3G Another display schematic diagram provided by the embodiments of the present application;

[0056] Figure 3H Another display schematic diagram provided by the embodiments of the present application;

[0057] Figure 4 A signal interaction schematic diagram provided by the embodiments of the present application;

[0058] Figure 5 A schematic diagram for determining the antenna distance between two devices provided by the embodiments of the present application;

[0059] Figure 6 A schematic diagram of the overlapping area of the images captured by TV 101 and TV 102 provided by the embodiments of the present application;

[0060] Figure 7 Another schematic diagram of the overlapping area of the images captured by TV 101 and TV 102 provided by the embodiments of the present application;

[0061] Figure 8 A schematic diagram of the position of a human face in the images captured by TV 101 and TV 102 provided by the embodiments of the present application;

[0062] Figure 9 A schematic diagram of the sorting of devices provided by the embodiments of the present application;

[0063] Figure 10 Another signal interaction schematic diagram provided by the embodiments of the present application;

[0064] Figure 11A Another schematic diagram of the sorting of devices provided by the embodiments of the present application;

[0065] Figure 11B Another schematic diagram of the sorting of devices provided by the embodiments of the present application;

[0066] Figure 12 Another signal interaction schematic diagram provided by the embodiments of the present application;

[0067] Figure 13 Another display schematic diagram provided by the embodiments of the present application;

[0068] Figure 14 Another signal interaction schematic diagram provided by the embodiments of the present application;

[0069] Figure 15 A schematic diagram of the structure of the chip system provided by the embodiments of the present application. Detailed implementation manners

[0070] The present application provides a screen combination method. For a screen terminal device with a built-in camera, without the aid of any other specific sensors and manual input of orientation, it can automatically detect the screen combination scenario and automatically calculate the relative orientation of the screens based on the orientation of the overlapping area of the pictures captured by the camera, thereby completing the screen combination process and giving users a simple and intelligent screen splicing usage experience.

[0071] As Figure 1B shown, it is a schematic architecture diagram of a screen splicing system provided by an embodiment of the present application. As Figure 1B shown, the system may include: one or more electronic devices, for example, it may include router 100, TV 101, TV 102, TV 103, and TV 104. TV 101, TV 102, TV 103, and TV 104 can be connected to the same local area network based on router 100. Of course, the screen splicing system may also include more electronic devices, which are not limited in the present application.

[0072] As Figure 1C shown, it is a schematic architecture diagram of another screen splicing system provided by an embodiment of the present application. The system may include: one or more electronic devices, for example, it may include TV 101, TV 102, and TV 103. TV 101, TV 102, and TV 103 can be connected pairwise through short-range communication technologies (for example, WIFI direct connection technology, Bluetooth technology, etc.). Of course, the screen splicing system may also include more electronic devices, which are not limited in the present application.

[0073] As Figure 2A shown, TV 101, TV 102, TV 103, or TV 104 may be screen 110, and screen 110 may include: a processor 111, a memory 112, a wireless communication processing module 113, a power switch 114, a wired LAN communication processing module 115, an HDMI communication processing module 116, a universal serial bus (USB) communication processing module 117, a display screen 118, an audio module 119, a speaker 119A, a microphone 119B, and so on. Among them:

[0074] The processor 111 can be used to read and execute computer-readable instructions. In a specific implementation, the processor 111 may mainly include a controller, an arithmetic unit, and registers. Among them, the controller is mainly responsible for instruction decoding and sending a control signal for the operation corresponding to the instruction. The arithmetic unit is mainly responsible for storing the register operands and intermediate operation results temporarily stored during the instruction execution process, etc. In a specific implementation, the hardware architecture of the processor 111 may be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

[0075] In some embodiments, the processor 111 may be used to parse signals received by the wireless communication processing module 113 and / or the wired LAN communication processing module 115. The processor 111 may be used to perform corresponding processing operations according to the parsing results of the signals, such as responding to data requests, or controlling the display of the display screen 118 and / or the output of the audio module 119 according to the control request, and so on.

[0076] In some embodiments, the processor 111 may also be used to generate signals sent out by the wireless communication processing module 113 and / or the wired LAN communication processing module 115, such as Bluetooth broadcast signals, beacon signals, and so on.

[0077] The memory 112 is coupled to the processor 111 and is used to store various software programs and / or multiple sets of instructions. In a specific implementation, the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 112 may store an operating system, such as embedded operating systems like uCOS, VxWorks, RTLinux, etc. The memory 112 may also store a communication program, which may be used to communicate with other devices.

[0078] The wireless communication processing module 113 may include a Bluetooth (BT) communication processing module 113A and a WLAN communication processing module 113B.

[0079] In some embodiments, one or more of the Bluetooth (BT) communication processing module 113A and the WLAN communication processing module 113B may monitor signals emitted by other devices, such as detection requests, scan signals, etc., and may send response signals, such as detection responses, scan responses, etc., so that other devices can discover the screen 110 and establish a wireless communication connection with other devices to communicate with other devices through one or more wireless communication technologies such as Bluetooth or WLAN. The WLAN communication processing module 113B may include one or more WLAN communication solutions such as Wi-Fi direct, Wi-Fi LAN, or Wi-Fi softAP.

[0080] In other embodiments, one or more of the Bluetooth (BT) communication processing module 113A and the WLAN communication processing module 113B may also emit signals, such as broadcast Bluetooth signals, beacon signals, so that other devices can discover the screen 110 and establish a wireless communication connection with other devices to communicate with other devices through one or more wireless communication technologies such as Bluetooth or WLAN.

[0081] In some embodiments, the screen 110 can be connected to the Internet via WLAN wireless communication technology, thereby establishing a communication connection with servers on the Internet (such as channel identification servers, on-demand resource servers, etc.).

[0082] The wireless communication processing module 113 may further include an infrared communication processing module 113C. The infrared communication processing module 113C can communicate with other devices (such as remote controls) through infrared remote control technology.

[0083] The power switch 114 can be used to control the power supply to the display screen 118.

[0084] The wired LAN communication processing module 115 can be used to communicate with other devices in the same LAN via wired LAN, and can also be used to connect to the WAN via wired LAN to communicate with devices in the WAN.

[0085] The HDMI communication processing module 116 can be used to communicate with devices such as set-top boxes through the HDMI port. For example, the HDMI communication processing module 116 can receive media content sent by the set-top box through the HDMI port, etc.

[0086] The USB communication processing module 117 can be used to communicate with other devices through the USB interface.

[0087] The display screen 118 can be used to display images, videos, etc. The display screen 118 can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED) display screen, an active-matrix organic light-emitting diode (AMOLED) display screen, a flexible light-emitting diode (FLED) display screen, a quantum dot emitting diodes (QLED) display screen, etc.

[0088] The audio module 119 can be used to convert digital audio signals into analog audio signals for output, and can also be used to convert analog audio inputs into digital audio signals. The audio module 119 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 119 can be provided in the processor 111, or some functional modules of the audio module 119 can be provided in the processor 111. The audio module 119 can transmit audio signals to the wireless communication processing module 113 through a bus interface (such as a UART interface, etc.), to implement the function of playing audio signals through a Bluetooth speaker.

[0089] The speaker 119A can be used to convert the audio signals sent by the audio module 119 into sound signals.

[0090] In some embodiments, the screen 110 may further include a microphone 119B, also known as a "microphone" or "transmitter", for converting sound signals into electrical signals. When sending a voice control instruction, the user can speak through the mouth to input the sound signal into the microphone 119B.

[0091] The camera 120 can be used to capture still images or videos.

[0092] It can be understood that the above screen 110 may have more or fewer components than those Figure 2A shown, can combine two or more components, or can have different component configurations. Figure 2A The various components shown in

[0093] such as Figure 2B shown, the application package may include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, etc.

[0094] In the embodiments of the present application, the application layer may further include a screen splicing management service, which is used to manage the screen splicing (screen combination) between multiple devices. The screen splicing management service can be integrated in the system APP or a third-party APP, such as the Smart Life APP, the Smart Interconnection APP, the Settings application, etc., and the present application does not make a limitation.

[0095] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0096] such as Figure 2BAs shown, the application framework layer may include an Activity Manager, a Window Manager, a Content Provider, a View System, a Resource Manager, a Notification Manager, etc. The embodiments of the present application do not impose any restrictions on this.

[0097] Activity Manager: It is used to manage the life cycle of each application. Applications usually run in the operating system in the form of Activities. For each Activity, there will be a corresponding application record (ActivityRecord) in the Activity Manager, and this ActivityRecord records the status of the Activity of the application. The Activity Manager can use this ActivityRecord as an identifier to schedule the Activity process of the application.

[0098] WindowManagerService: It is used to manage the graphical user interface (GUI) resources used on the screen. Specifically, it can be used for: obtaining the display screen size, creating and destroying windows, showing and hiding windows, window layout, focus management, and input method and wallpaper management, etc.

[0099] The system libraries and kernel layer, etc. below the application framework layer can be referred to as the underlying system, and the underlying system includes an underlying display system for providing display services. For example, the underlying display system includes a display driver in the kernel layer and a surface manager in the system library, etc.

[0100] The Content Provider is used to store and obtain data, and make this data accessible to application programs. The data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The View System includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The View System can be used to build application programs. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures. The Resource Manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, etc. The Notification Manager enables application programs to display notification information in the status bar, can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the Notification Manager is used to inform that the download is completed, message reminders, etc. The Notification Manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of a background-running application program, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompting text information in the status bar, emitting a prompt sound, the terminal vibrating, the indicator light flashing, etc.

[0101] As Figure 2B shown, the Android Runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part is the functional functions that the Java language needs to call, and the other part is the core libraries of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0102] As Figure 2B shown, the system libraries can include multiple functional modules. For example: Surface Manager, Media Libraries, OpenGL ES, SGL, etc.

[0103] The Surface Manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0104] The Media Libraries support the playback and recording of multiple common audio and video formats, as well as static image files, etc. The Media Libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0105] OpenGL ES is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0106] SGL is a drawing engine for 2D drawing.

[0107] As Figure 2B shown, the kernel layer is the layer between hardware and software. The kernel layer contains at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0108] As Figure 3A shown, a connection schematic diagram of TVs 101, 103, 103, and 104 is provided. TVs 101, 103, 103, and 104 can respectively include hardware modules such as a processor, a display screen, a camera, and a communication unit. TVs 101, 103, 103, and 104 can be interconnected through the communication unit to communicate with each other.

[0109] As Figure 3BAs shown, a connection diagram of another TV 101, TV 103, TV 103, and TV 104 is provided. TV 101, TV 103, TV 103, and TV 104 may respectively include an application layer, an application framework layer, an Android runtime and system libraries, and a kernel layer. In the embodiments of the present application, TV 101, TV 103, TV 103, and TV 104 can automatically perform screen splicing through a screen splicing management service.

[0110] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more than two. In addition, in order to clearly describe the technical solutions in the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily limit being different.

[0111] For ease of understanding, the following specifically introduces the screen combination method provided in the embodiments of the present application with reference to the accompanying drawings.

[0112] As Figure 3C shown in (a) of [], the main interface 300 of TV 101 is shown. When TV 101 determines that the distance D1 between TV 101 and TV 102 is less than or equal to the maximum combination radius R1 corresponding to TV 101 and TV 102 (for the concept of the maximum combination radius, see the relevant description in step 406a below), as Figure 3C shown in (b) of [], TV 101 can pop up a dialog box 301 to prompt the user that a nearby device has been detected. The dialog box 301 may include a "Yes" button 302 and a "No" button 303 for the user to select whether to perform screen combination. As Figure 3C shown in (c) of [], in response to the user's operation of selecting the "Yes" button 302 (for example, selecting the button 302 through a remote control or a touch screen), as Figure 3C shown in (d) of [], the user can be prompted that screen combination is in progress.

[0113] Optionally, TV 101 can also prompt the user with the identifier or ID of the nearby device. For example, as Figure 3D shown, TV 101 can pop up a dialog box 304 to prompt the user that device 222xxx (222xxx is the ID of TV 102) has been detected nearby. The dialog box 304 may include a "Yes" button 302 and a "No" button 303 for the user to select whether to perform screen combination with the TV in the living room.

[0114] Assume that the display contents of TV 101 and TV 102 before splicing are as shown in (a) and (b) of Figure 3E respectively. After screen splicing, as shown in (c) of Figure 3E , TV 101 and TV 102 can jointly display the desktop of TV 101 (for example, TV 101 is the master device, and the process of determining the master device can refer to the relevant description in step 409 below), or, as shown in (d) of Figure 3E , TV 101 and TV 102 can respectively display the desktop of TV 101.

[0115] In some embodiments, if a new TV 103 is added to the screen group (for example, the first screen group) composed of TV 101 and TV 102 (that is, when TV 101 and TV 102 are spliced together, the new TV 103 needs to be further spliced with TV 101 and TV 103), exemplarily, as shown in (a) of Figure 3F , TV 103 can gradually approach the screen group composed of TV 101 and TV 102. Then, as shown in (b) of Figure 3F , a pop-up box 305 can be popped up on TV 101 and TV 102 to prompt the user that device 123xxx (123xxx is the ID of TV 103) is detected. The pop-up box 305 can include a "Yes" button 302 and a "No" button 303 for the user to select whether to add the device to the screen group. Optionally, TV 103 can also prompt the user with the identification or ID of nearby devices. For example, TV 103 can pop up a pop-up box 306 to prompt the user that device 111xxx (111xxx can be the ID of TV 101) is detected. The pop-up box 306 can include a "Yes" button 302 and a "No" button 303 for the user to select whether to perform screen combination. In response to the user's operation of clicking the "Yes" button 302, as shown in (c) of Figure 3F , TV 101, TV 102 and TV 103 can form a new screen group (for example, the second screen group).

[0116] In some embodiments, as shown in (a) of Figure 3G , the screen group composed of TV 101, TV 102 and TV 103 can jointly display the corresponding display content. If TV 103 needs to be removed from the screen group composed of TV 101, TV 102 and TV 103 (for example, when TV 101, TV 102 and TV 103 are spliced together, TV 103 is removed), exemplarily, as shown in (a) of Figure 3GAs shown in (b) thereof, the TVs 101 and 102 can pop up a dialog box 307 to prompt the user that the device 123xxx in the current screen group has been removed. The dialog box 307 can include an OK button 308. In response to the user clicking the OK button 308, the TV 101 determines that the information in the dialog box 307 is known to the user and can hide the dialog box 307. Alternatively, the dialog box 307 can automatically hide after a few seconds (e.g., 2 s) of appearance to avoid affecting the display content of the TVs 101 and 102.

[0117] In some other embodiments, as Figure 3H shown in (a) thereof, the screen group composed of the TVs 101, 102, and 103 can jointly display corresponding display content. If it is necessary to remove the TV 103 from the screen group composed of the TVs 101, 102, and 103 (e.g., when the TVs 101, 102, and 103 are spliced together and the TV 103 is removed), exemplarily, as Figure 3H shown in (b) thereof, the TVs 101, 102, and 103 can still maintain their previous display content, and the TVs 101 and 102 can pop up a dialog box 309 to prompt the user to confirm whether to remove the device 123xxx from the current screen group? The dialog box 309 can include an OK button 310 and a Cancel button 311. In response to the user clicking the OK button 310, as Figure 3H shown in (c) thereof, the TVs 101 and 102 can jointly display corresponding display content (the display content of the TVs 101 and 102 can be determined by the processor of the TV 101 (the main device)), and the TV 103 can separately display corresponding display content (the content displayed by the TV 103 can be determined by the processor of the TV 103). Additionally, if the user clicks the Cancel button 311, as Figure 3H shown in (b) thereof, the TVs 101, 102, and 103 can still maintain their previous display content.

[0118] As Figure 4 shown, taking the screen splicing of the TVs 101 and 102 as an example, the specific implementation process of the screen combination method provided by the embodiments of the present application is described, including the following processes:

[0119] 401. The TV 101 starts the screen splicing management service.

[0120] After the TV 101 is powered on, it can start the screen splicing management service. Among them, the screen splicing management service can be integrated into the system APP or third-party APP on the TV 101, such as the Smart Life APP, the Smart Connectivity APP, the Settings application, etc., which is not limited in the present application.

[0121] 402. The TV 102 starts the screen splicing management service.

[0122] After the TV 102 is powered on, the screen splicing management service can be started. The screen splicing management service can refer to the relevant description in step 401 and will not be elaborated here.

[0123] 403. The TV 101 and the TV 102 establish a network connection and share the device information of the TV 101 and the TV 102.

[0124] In one implementation, the TV 101 and the TV 102 can be connected to the same local area network to establish a network connection between the TV 101 and the TV 102.

[0125] After the TV 101 starts the screen splicing management service, the screen splicing management service of the TV 101 can complete the discovery of other nearby devices (such as the TV 102) installed with the screen splicing management service based on short-range communication technology (for example, the proximity discovery technology of Bluetooth / WIFI).

[0126] Similarly, after the TV 102 starts the screen splicing management service, the screen splicing management service can complete the discovery of other nearby screen devices (such as the TV 101) based on short-range communication technology (for example, the proximity discovery technology of Bluetooth / WIFI).

[0127] In some embodiments, the TV 101 and the TV 102 can directly discover and connect to each other through technologies such as Bluetooth / WIFI direct connection.

[0128] 404a. The TV 101 establishes a list of nearby devices.

[0129] The screen splicing management service of the TV 101 can interact with the screen splicing management services of other devices (such as the TV 102) connected to the TV 101 to obtain a list of nearby devices. Exemplarily, the list of nearby devices established by the TV 101 can be as shown in Table 1.

[0130] Table 1

[0131] Nearby Devices (Connected) MAC / ID TV 102 MAC 2 / ID2

[0132] Of course, the TV 101 can also be connected to more devices. For example, it can be connected to devices such as the TV 103 and the TV 104. In this way, the list of nearby devices established by the TV 101 can be as shown in Table 2.

[0133] Table 2

[0134] Nearby Devices (Connected) MAC / ID TV 102 MAC 2 / ID2 TV 103 MAC 3 / ID3 TV 104 MAC4 / ID4

[0135] Optionally, the TV 101 can also obtain information such as the name, size information (e.g., the length and width of the device), antenna information (the installation position of the antenna in the device, the type of the antenna, the accuracy, the size, etc.) of each device from the devices in the nearby device list.

[0136] 404b. The TV 102 creates a nearby device list.

[0137] Exemplarily, the nearby device list created by the TV 102 can be as shown in Table 3.

[0138] Table 3

[0139] Nearby Devices (Connected) MAC / ID TV 101 MAC 1 / ID1

[0140] Of course, the TV 102 can also be connected to more devices. For example, it can be connected to devices such as the TV 103 and the TV 104. In this way, the nearby device list created by the TV 102 can be as shown in Table 4.

[0141] Table 4

[0142] Nearby Devices (Connected) MAC / ID TV 101 MAC 1 / ID1 TV 103 MAC 3 / ID3 TV 104 MAC4 / ID4

[0143] Optionally, the TV 102 can also obtain information such as the name, size information (e.g., the length and width of the device), antenna information (the installation position of the antenna in the device, the type of the antenna, the accuracy, the size, etc.) of each device from the devices in the nearby device list.

[0144] 405. A short-range signal is sent between the TV 101 and the TV 102.

[0145] In a possible design, the TV 101 can measure the distance between the TV 101 and each device in the nearby device list through short-range communication technologies (such as Bluetooth / WIFI signal ranging technology). For example, the TV 101 can obtain the distance D1 between the two devices based on the received signal strength indication (RSSI) of the short-range signal sent by the TV 102.

[0146] The TV 102 can measure the distance between the TV 102 and each device in the nearby device list of the TV 102. For example, the TV 102 can obtain the distance D1 between the two devices based on the RSSI of the short-range signal sent by the TV 101. Alternatively, the TV 101 can notify the measured distance D1 to the TV 102.

[0147] 406a. The TV 101 determines that the distance D1 between the TV 101 and the TV 102 is less than or equal to the maximum combined radius R1 corresponding to the TV 101 and the TV 102.

[0148] The television 101 can measure the distance between the television 101 and each device in the list of nearby devices, and determine the magnitude of the distance between every two devices (two devices) and the maximum combined radius corresponding to the two devices.

[0149] For example, the television 101 can measure the distance D1 between the television 101 and the television 102, and determine the magnitude of D1 and the maximum combined radius R1 between the television 101 and the television 102.

[0150] Among them, the maximum combined radius corresponding to the television 101 and the television 102 can be determined according to the sizes of the two devices (the television 101 and the television 102) and the positions of the antennas. Among them, the antennas can be, for example, Bluetooth antennas, WIFI antennas, etc.

[0151] Exemplarily, as Figure 5 shown in (a) of [], assuming that the sizes of the two devices (for example, the television 101 and the television 102) are the same, the height is h, the width is w, w >= h, taking the upper left corner of the device as the (0, 0) coordinates of the coordinate system, the coordinates of the center point of the screen are (x, y), and the coordinates of the antenna can be (x1, y1). The distance recognition accuracy of the antenna can be a centimeter.

[0152] In the scenario where the two devices are combined horizontally, assuming that the antenna is located in the middle of the vertical direction of the device, that is, y1 = y, and x1 > x, then the combined radius corresponding to the two devices = w + 2 * (x1 - x) + 2 * a. Similarly, in the scenario where the two devices are combined vertically, assuming that the antenna is located in the middle of the horizontal direction of the device, that is, x1 = x, and y1 > y, then the combined radius corresponding to the two devices = h + 2 * (y1 - y) + 2 * a.

[0153] As Figure 5 shown in (b) of [], in the scenario where the two devices are combined diagonally, assuming that the antennas are located at the edges of the devices, for example, at the upper left and lower right respectively. At this time, the combined radius corresponding to the two devices = 2 * r + 2 * d + 2 * a; where d is the distance between the antenna and the center point of the device, r is the maximum distance from the center point of the device to the edge,

[0154] It can be understood that the combined radius corresponding to the two devices in the diagonal splicing scenario is the largest. In order to ensure the effectiveness of the combined radius as much as possible, the maximum combined radius corresponding to the two devices can be: R1 = 2 * r + 2 * d + 2 * a. That is, the maximum combined radius corresponding to the two devices can be determined according to the distance between the antennas of the television 101 and the television 102 in the diagonal splicing scenario.

[0155] The algorithm for the maximum combined radius corresponding to two devices of the same size is described above. If two devices of different sizes are combined, as shown in Table 5, assuming that the size parameters corresponding to Device 1 are r1 and d1, and the size parameters corresponding to Device 2 are r2 and d2, then the maximum combined radius R2 of Device 1 and Device 2 = (r1 + r2) + (d1 + d2) + 2 * a. Among them, r1 is the maximum distance from the center point of Device 1 to the edge, d1 is the distance from the antenna of Device 1 to the center point of Device 1; r2 is the maximum distance from the center point of Device 2 to the edge, d2 is the distance from the antenna of Device 2 to the center point of Device 2; the distance recognition accuracy of the antenna is a centimeter.

[0156] Table 5

[0157]

[0158] If the distance between TV 101 and TV 102 is less than or equal to the size of the maximum combined radius corresponding to TV 101 and TV 102, it indicates that the two devices have an intention to assemble (combination intention) or are in an assembled state (combination state).

[0159] The following takes TV 101 and TV 102 as an example of the same size. When TV 101 determines that the distance between TV 101 and TV 102 is less than or equal to R1, TV 101 and TV 102 can be marked as the ready-to-combine state (ready-to-splice state). That is, when TV 101 determines that the distance (placement interval) between TV 101 and TV 102 is less than or equal to the maximum combined radius R1 corresponding to TV 101 and TV 102, it is determined that TV 101 and TV 102 have a combination intention and can be prepared for combination. TV 101 can group and organize the devices marked as the ready-to-combine state to form a screen combination preparation device group, that is, TV 101 and TV 102 can form a screen combination preparation device group.

[0160] TV 101 can also measure the distance D2 between TV 101 and TV 103, and determine the size relationship between D2 and the maximum combined radius R2 between TV 101 and TV 103. TV 101 can also measure the distance D3 between TV 101 and TV 104, and determine the size relationship between D3 and the maximum combined radius R3 between TV 101 and TV 104. The specific process can refer to the relevant description above and will not be elaborated here.

[0161] 406b. TV 102 determines that the distance D1 between TV 101 and TV 102 is less than or equal to the maximum combined radius R1 corresponding to TV 101 and TV 102.

[0162] TV 102 can measure the distance between TV 102 and each device in the nearby device list, and determine the size relationship between the distance between two devices and the maximum combined radius corresponding to the two devices.

[0163] For the specific process, please refer to the relevant description in step 406a, which will not be elaborated here.

[0164] 406c. TV 102 sends the first information to TV 101, and the first information includes the distance information measured by TV 102.

[0165] TV 101 can receive the first information from TV 102. The first information may include the distances between TV 102 and each device in the list of nearby devices of TV 102, and / or the comparison results of the distances between every two devices determined by TV 102 and the sizes of the maximum combined radii corresponding to the every two devices.

[0166] Optionally, TV 101 can also receive the distance information and / or comparison results measured by other devices from other devices.

[0167] For another example, TV 101 can also receive the second information from TV 103. The second information may include the distances between TV 103 and each device in the list of nearby devices of TV 103, and / or the comparison results of the distances between every two devices determined by TV 103 and the sizes of the maximum combined radii corresponding to the every two devices.

[0168] For yet another example, TV 101 can also receive the third information from TV 104. The third information may include the distances between TV 104 and each device in the list of nearby devices of TV 104, and / or the comparison results of the distances between every two devices determined by TV 104 and the sizes of the maximum combined radii corresponding to the every two devices.

[0169] In this way, TV 101 can determine the distances between every two devices (each pair of devices) in the current local area network, and / or the comparison results of the distances between every two devices and the sizes of the maximum combined radii corresponding to the two devices, so that TV 101 can determine the multiple devices that need to be spliced together currently, and the multiple devices that need to be spliced together can form a screen group.

[0170] It should be understood that a local area network may include multiple screen combination preparation device groups (abbreviated as screen groups). Each screen group may include at least two devices, and the at least two devices can be spliced together, and the at least two devices can be directly or indirectly connected. For example, TV 101 and TV 102 can form a screen group, and TV 101 and TV 102 can be directly connected (the distance D1 between TV 101 and TV 102 is less than or equal to the maximum combined radius corresponding to TV 101 and TV 102).

[0171] 406d. TV 101 sends the second information to TV 102, and the second information includes the distance information measured by TV 101.

[0172] For the specific process, reference can be made to step 406c, which will not be elaborated here.

[0173] If TV 101 determines that the distance D1 between TV 101 and TV 102 is less than the maximum combined radius R1 corresponding to TV 101 and TV 102, that is, it is determined that TV 101 and TV 102 need to form a screen group, TV 101 can execute step 407.

[0174] 407. TV 101 displays a first prompt message for prompting the user whether to form a screen.

[0175] The user can set the screen combination strategy on TV 101 in advance. For example, the user can set to automatically perform screen combination or manually perform screen combination.

[0176] If the user sets to automatically perform screen combination, TV 101 can automatically start the detection program for screen combination and splicing through short-range communication technology. The detection program for screen combination and splicing is to detect whether the distance between two devices is less than or equal to the maximum combined radius corresponding to the two devices, so as to determine whether screen combination and splicing are required. Optionally, the detection program for screen combination and splicing can be automatically started through specific scenarios such as power-on startup and standby wake-up.

[0177] If the user sets to manually perform screen combination, the user can enter, for example, the screen splicing management service in the Smart Life APP, the Smart Interconnection APP or the settings application, and manually (for example, by clicking a specific control) start the detection program for screen assembly and splicing. TV 101 can give an interface prompt and determine whether to form a screen according to the user's operation, which can avoid the error of automatically triggering screen combination.

[0178] Exemplarily, as Figure 3C shown in (a) therein, the main interface 300 of TV 101 is shown. When TV 101 determines that the distance D1 between TV 101 and TV 102 is less than or equal to the maximum combined radius R1 corresponding to TV 101 and TV 102, as Figure 3C shown in (b) therein, TV 101 can pop up a pop-up window 301 to prompt the user that a device nearby is detected. The pop-up window 301 may include a "Yes" button 302 and a "No" button 303 for the user to select whether to perform screen combination.

[0179] Optionally, TV 101 can also prompt the user with the identifier or ID of the nearby device. For example, as Figure 3DAs shown, the TV 101 can pop up a dialog box 304 to prompt the user that the device 222xxx (222xxx is the ID of the TV 102) is detected nearby. The dialog box 304 can include a "Yes" button 302 and a "No" button 303 for the user to select whether to combine the screen of the current device with the TV in the living room.

[0180] Optionally, the TV 102 can also display a first prompt message. When both the TV 101 and the TV 102 give interface prompts for the user to select whether to combine the screen, if the user has confirmed on one device (for example, has confirmed on the TV 101), the TV 101 can send the user's confirmation information to the TV 102, without the user having to confirm one by one on each device.

[0181] In some embodiments, any device in the screen group (for example, the TV 101) can give an interface prompt (for example, display a first prompt message) for the user to select whether to combine the screen, that is, the TV 102 can no longer give an interface prompt.

[0182] 408a. The user clicks the button to agree to screen combination.

[0183] Exemplarily, as shown in (c) of Figure 3C , in response to the user's operation of selecting the "Yes" button 302 (for example, selecting the button 302 through a remote control or a touch screen), as shown in (d) of Figure 3C , the user can be prompted that the screen combination is in progress. In response to the user clicking the button to agree to screen combination, step 409 can be executed. Alternatively, if the user sets automatic screen combination, at this time, there is no need to prompt whether to combine the screen in the interface (that is, steps 407 and 408 do not need to be executed), and step 409 can be directly started.

[0184] 409. The TV 101 and the TV 102 form a screen group, and the main device is elected as the TV 101.

[0185] The TV 101 can score the resources of each device in the current screen group according to the resource situation of each device, sort them in descending order according to the resource score, and take the device with the highest real-time resource score as the main device. The resource situation of the device can include hardware resource capabilities such as a central processing unit (CPU) / read only memory (ROM) / random access memory (RAM). After determining the main device, the remaining devices in the screen group can be used as slave devices. For example, if the TV 101 is used as the main device, then the TV 102 can be used as the slave device.

[0186] Optionally, the user can manually select the master device. For example, the user can enter the device settings application to select the master device. Alternatively, after the master device is automatically elected, TV 101 can pop up a dialog box to prompt the user with the identification of the current master device (e.g., it can remind the user that the current master device is the living room TV (i.e., TV 101)). The user can confirm to set TV 101 as the master device based on the OK button in the dialog box, or the user can modify the master device based on the Modify button in the dialog box. In this case, when TV 101 serves as the master device, it can be understood to include a host for controlling the screen group and a screen for displaying images, and the host is integrated in TV 101.

[0187] The following takes TV 101 as the master device for illustration:

[0188] 410. The master device sends a first notification message to TV 102, and the first notification message is used to notify TV 102 to take a photo and perform orientation recognition.

[0189] Exemplarily, in the scenario where TV 101 and TV 102 are combined, the master device (i.e., TV 101) can use its camera to take a photo. At the same time, TV 101 can send a first notification message to TV 102, which is used to notify TV 102 to use its camera to take a photo (image / picture), and to notify TV 102 to perform orientation recognition based on the taken photo and the photos obtained from other devices.

[0190] 411a. TV 101 takes a photo.

[0191] TV 101 can control the built-in camera of TV 101 to take a photo through the screen splicing management service.

[0192] 411b. After receiving the first notification message, TV 102 takes a photo.

[0193] TV 102 can control the built-in camera of TV 102 to take a photo through the screen splicing management service. TV 101 and TV 102 can negotiate to take photos at the same moment.

[0194] 412a. TV 101 sends the photo taken by TV 101 to TV 102.

[0195] 412b. TV 102 sends the photo taken by TV 102 to TV 101.

[0196] 412c. TV 101 determines the orientation relationship between TV 101 and TV 102 based on the photo taken by TV 101 and the photo taken by TV 102.

[0197] After TV 101 receives the photos taken by TV 102, it can perform image matching (comparison) on the photos taken by itself and the photos taken by TV 102 through an image matching algorithm to determine the overlapping area of the two photos (i.e., the similar image part / image content). Among them, image matching is to determine the overlapping part of the two photos through the analysis of the corresponding relationships of the image content, features, structures, relationships, textures, and gray levels of the two photos, as well as the similarity and consistency.

[0198] For example, the image matching algorithm can include Scale Invariant Feature Transform (SIFT), speed up robust features (SURF), Flann-based matcher, etc.

[0199] Then, TV 101 can determine the relative position relationship (relative azimuth relationship) between TV 101 and TV 102 based on the position of the overlapping area on the photo taken by TV 101. That is, through the mapping relationship between the overlapping area position and the camera azimuth, the relative position relationship between TV 101 and TV 102 is determined. The relative azimuth relationship between TV 101 and TV 102 can be, for example, that TV 101 is located in the upper, lower, left, right, upper left, lower left, upper right, lower right, etc. directions of TV 102. Or, the relative azimuth relationship between TV 101 and TV 102 can be that TV 102 is located in the upper, lower, left, right, upper left, lower left, upper right, lower right, etc. directions of TV 101.

[0200] The splicing modes of TV 101 and TV 102 can include three modes: up-and-down splicing, left-and-right splicing, or diagonal splicing. For example, when TV 101 is located above or below TV 102, the splicing mode of TV 101 and TV 102 can be up-and-down splicing; when TV 101 is located to the left or right of TV 102, the splicing mode of TV 101 and TV 102 can be left-and-right splicing; when TV 101 is located in the upper left, lower left, upper right, or lower right of TV 102, the splicing mode of TV 101 and TV 102 can be diagonal splicing.

[0201] Exemplarily, as Figure 6 shown, the dashed box represents the photo taken by screen TV 101, and the solid box represents the photo taken by screen TV 102. Then, the azimuth of screen TV 101 relative to screen TV 102 is shown in Table 6.

[0202] Table 6

[0203]

[0204] Exemplarily, as Figure 6As shown in (a) therein, if the overlapping area is in the lower right area (lower right corner) of the photo taken by TV 101 and in the upper left area (upper left corner) of the photo taken by TV 102, it is determined that TV 101 is located in the upper left of TV 102 (upper left corner). As Figure 6 As shown in (b) therein, if the overlapping area is in the lower half area (directly below) of the photo taken by TV 101 and in the upper half area (directly above) of the photo taken by TV 102, it is determined that TV 101 is located above TV 102 (directly above); As Figure 6 As shown in (c) therein, if the overlapping area is in the lower left corner (lower left) of the photo taken by TV 101 and in the upper right corner (upper right) of the photo taken by TV 102, it is determined that TV 101 is located in the upper right of TV 102 (upper right corner); As Figure 6 As shown in (d) therein, if the overlapping area is in the right half area of the photo taken by TV 101 and in the left half area of the photo taken by TV 102, it is determined that TV 101 is located to the left of TV 102; As Figure 6 As shown in (e) therein, if the overlapping area is in the left half area of the photo taken by TV 101 and in the right half area of the photo taken by TV 102, it is determined that TV 101 is located to the right of TV 102; As Figure 6 As shown in (f) therein, if the overlapping area is in the upper right corner of the photo taken by TV 101 and in the lower left corner of the photo taken by TV 102, it is determined that TV 101 is located in the lower left of TV 102; As Figure 6 As shown in (g) therein, if the overlapping area is in the upper half area of the photo taken by TV 101 and in the lower half area of the photo taken by TV 102, it is determined that TV 101 is located below TV 102; As Figure 6 As shown in (h) therein, if the overlapping area is in the upper left corner (lower left) of the photo taken by TV 101 and in the lower right corner (lower right) of the photo taken by TV 102, it is determined that TV 101 is located in the lower right of TV 102 (lower right corner).

[0205] In a possible design, image matching can be performed on the photos taken by TV 101 and the photos taken by TV 102 to identify the overlapping area of the two photos. Then, calculate respectively which azimuth area the overlapping area is in the photo taken by TV 101 and the photo taken by TV 102, and then find out the relative positions of TV 101 and TV 102 through Table 6. For example, if the overlapping area is in the lower half area of the photo taken by TV 101, the azimuth of TV 101 relative to TV 102 is up, that is, TV 101 is located above TV 102.

[0206] In another possible design, the photos taken by each device (e.g., TV 101 or TV 102) can be divided into several sub-regions (e.g., 6 / 9 / 12, etc., which are not limited in this application). Each sub-region in the photo taken by TV 101 is matched with each sub-region in the photo taken by TV 102 to determine the numbers of the matching sub-regions. Based on the numbers of the matching sub-regions, determine which azimuth area the matching sub-regions are located in the photos taken by TV 101 and TV 102, and then find out the relative positions of TV 101 and TV 102 through Table 6.

[0207] Exemplarily, as Figure 7 shown, assume that the photo taken by TV 101 can be divided into 6 sub-regions, including ①④⑦②⑤⑧ respectively, and the photo taken by TV 102 can be divided into 6 sub-regions, including ②⑤⑧③⑥⑨ respectively. It can be known that the matching sub-regions include ②⑤⑧. Since ②⑤⑧ are located in the right half area of the photo taken by TV 101, by looking up Table 6, it can be known that TV 101 is on the left of TV 102, that is, TV 102 is on the right of TV 101. Or, since ②⑤⑧ are located in the left half area of the photo taken by TV 102, by looking up Table 6, it can be known that TV 101 is on the left of TV 102, that is, TV 102 is on the right of TV 101.

[0208] In yet another possible design, during the process of identifying the relative azimuth between devices for screen combination and splicing, specific identification objects such as human faces, human actions, specific objects, utensils, etc. can be added to the field of view of the cameras of the devices.

[0209] Exemplarily, after the screen combination and splicing program starts, an operation prompt can be first displayed on TV 101 and / or TV 102 to let the user ensure that a specific identification object (e.g., a human face) can be seen in the pictures of the cameras of TV 101 and / or TV 102, and then determine the relative azimuth between the devices according to the position of the human face in the photos taken by TV 101 and / or TV 102.

[0210] It should be noted that the position of the specific identification object in the photo taken by each device can have multiple direction dimensions, such as up-down dimension and left-right dimension. The same direction dimensions can be ignored, and different direction dimensions are used as the basis for judging the azimuth between devices. Exemplarily, as Figure 8As shown, assume that the sub-regions of the photos taken by TV 101 and TV 102 respectively include ①②④⑤⑦⑧ and ②③⑤⑥⑧⑨, and the face is located in region ②. Since region ② is located in the upper right of the photo taken by TV 101 and in the upper left of the photo taken by TV 102, it can be determined through lookup table 7 that TV 101 is located to the left of TV 102. That is, ignoring the same direction dimension (i.e., ignoring the "upper" direction dimension in upper right and upper left), using direction dimensions such as "left" and "right" as the basis for judging the orientation between devices.

[0211] Table 7

[0212]

[0213] Optionally, relevant programs can be preset in the screen splicing management service to prompt the user to give cooperation measures through methods such as screen display and voice prompts, so as to accelerate the recognition of the camera for specific positions, thereby accelerating the recognition speed of the relative positions of pictures, or directly input specific images to the camera of a certain device to mark the orientation of the corresponding device.

[0214] 412d. Optionally, TV 102 determines the orientation relationship between TV 101 and TV 102 based on the photos taken by TV 102 and the photos taken by TV 101.

[0215] For the specific process, reference can be made to the description in step 412a, and only the execution subject and other content need to be simply replaced, which will not be elaborated here.

[0216] 412e. TV  102 sends the orientation relationship between TV 101 and TV 102 determined by TV 102 to TV 101.

[0217] 413a. The master device determines the relative orientation relationships of all devices in the screen group.

[0218] The master device can collect the relative orientation information between every two devices in the screen group for summarization, and make unified arrangements in a coordinate system according to the orientation information, and record information such as numbers and coordinates for each device respectively.

[0219] The orientations of all devices in the screen group can be represented by an array. For example, it can be (device 1, device 2, the direction of device 1 relative to device 2). For example, assume that the screen group only includes TV 101 and TV 102, then the orientation of TV 101 relative to TV 102 can be: (TV 101, TV 102, up), indicating that TV 101 is located above TV 102. Or, the orientation of TV 102 relative to TV 101 can be: (TV 102, TV 101, down), indicating that TV 102 is located below TV 101.

[0220] Such as Figure 9As shown in (a) therein, if the master device determines that the orientation of TV 101 relative to TV 102 is (TV 101, TV 102, left), that is, TV 101 is located on the left side of TV 102, then TV 101 and TV 102 can be arranged horizontally in the coordinate system, and the sorting order of TV 101 and TV 102 can be (1), (2), that is, TV 101 and TV 102 are from left to right respectively.

[0221] As Figure 9 As shown in (b) therein, if the master device determines that the orientation of TV 101 relative to TV 102 is (TV 101, TV 102, up), that is, TV 101 is located above TV 102, then TV 101 and TV 102 can be arranged vertically in the coordinate system, and the sorting order of TV 101 and TV 102 can be (1), (2), that is, TV 101 and TV 102 are from top to bottom respectively.

[0222] In a possible design, the master device can number the devices in the screen group one by one in the direction from the upper left to the lower right. Exemplarily, the spliced devices can be arranged in an n*m matrix, where n can represent the number of rows, m can represent the number of columns, n is an integer greater than or equal to 1, m is an integer greater than or equal to 1, and n and m are not both 1 at the same time. For example, as Figure 9 As shown in (a) therein, assuming n = 1 and m = 2, when encoding, the encoding can start from the device at the top of the first column. After encoding the devices in the first column, the encoding can then start from the device at the top of the second column until the devices in the second column are encoded completely. In this way, the n*m devices can be encoded completely. Another example, as Figure 9 As shown in (b) therein, assuming n = 2 and m = 1, when encoding, the encoding can start from the device at the leftmost of the first row. After encoding the devices in the first row, the encoding can then start from the device at the leftmost of the second row until the devices in the second row are encoded completely. In this way, the n*m devices can be encoded completely.

[0223] 413b. The master device synchronizes the basic information of the screen group splicing to TV 102.

[0224] The master device can synchronize the basic information of the screen group splicing to all devices within the screen group. Each device within the screen group can receive the synchronization message sent by the master device. Among them, the synchronization message includes the basic information of the screen group splicing. The basic information of the screen group splicing includes the number of devices included in the screen group, the MAC / ID of each device, the master-slave information (i.e., the information of the master device and the slave device), the orientation information between devices, etc. Exemplarily, the current basic information of the screen group splicing can include the number of devices included in the screen group (e.g., 2), the MAC / ID of each device (e.g., the ID of TV 101 and TV 102), the master-slave information (e.g., the master device is TV 101 and the slave device is TV 102), and the orientation information between devices (e.g., TV 101 and TV 102 are in a left-right splicing state).

[0225] After each device within the screen group receives the synchronization message sent by the master device, a heartbeat link can be established between every two devices to maintain the combined relationship between devices in real time. Exemplarily, assuming a heartbeat link is established between TV 101 and TV 102, TV 101 can send a heartbeat monitoring data frame (also called a heartbeat packet) to TV 102 every 1 minute (or 30s, 2 minutes, 3 minutes, etc.). After TV 102 receives the heartbeat monitoring data frame, it can send a response frame. Then TV 101 determines that the connection is normal, otherwise it indicates that the connection is disconnected or abnormal.

[0226] 414a. TV 101 determines the display information of TV 101 and TV 102 respectively according to the basic information of the screen group splicing.

[0227] TV 101 can determine the display information of TV 101 and TV 102 respectively according to the basic information of the screen group splicing. That is, during the operation of the screen group system, the master device can realize the display arrangement of the screen group's picture output, the switching of the interface focus, etc. based on the basic information of the screen group splicing.

[0228] 414c. TV 101 sends the display information of TV 102 to TV 102.

[0229] 414c. TV 101 displays the corresponding display picture according to the display information of TV 101.

[0230] 414d. TV 102 displays the corresponding display picture according to the display information of TV 102.

[0231] For example, TV 101 can divide the display content of TV 101 into N parts (e.g., 2 parts) and then allocate them to each device in the screen group (e.g., TV 101 (itself) and TV 102). Among them, N is less than or equal to the number of devices included in the screen group.

[0232] Exemplarily, assume that the display contents of TV 101 and TV 102 before splicing are respectively as Figure 3EAs shown in (a) and (b) therein, after splicing, as Figure 3E shown in (c) therein, TV 101 and TV 102 can jointly display the desktop of the master device (e.g., TV 101), or, as Figure 3E shown in (d) therein, TV 101 and TV 102 can respectively display the desktop of the master device (e.g., TV 101).

[0233] It should be noted that during the process of using the screen combination, each device in the screen group can continuously detect the addition and removal of devices and refresh the basic information of the screen group splicing. Among them, the newly added device can be a device newly added to the current screen group, and the removed device can be that some devices are actively removed / removed from the current screen group, or some devices are powered off and passively offline. TV 101 can detect whether there is a new device added through short-range communication. TV 101 can detect and confirm whether the peer device is offline according to the heartbeat link. Or, TV 101 can detect and determine whether a certain device is offline through short-range communication. Or, the user can manually remove a certain device from the management interface.

[0234] In some embodiments, if TV 103 is newly added to the screen group composed of TV 101 and TV 102, as Figure 10 shown, the screen group splicing method may further include:

[0235] 415a. Send short-range signals between TV 101 and TV 102.

[0236] Short-range signals can be periodically sent between TV 101 and TV 102.

[0237] 415b. Send short-range signals between TV 102 and TV 103.

[0238] Short-range signals can be periodically sent between TV 102 and TV 103.

[0239] 415c. TV 101 / TV 102 determines that TV 103 needs to be newly added to the current screen group according to the short-range signal.

[0240] TV 101 can measure the distance between TV 101 and TV 103 according to the short-range signal, and TV 102 can measure the distance between TV 102 and TV 103 according to the short-range signal.

[0241] If one of the following conditions is met, it can be considered that TV 103 needs to be newly added to the current screen group. (1). The distance D2 between TV 101 and TV 103 is less than or equal to the corresponding maximum combination radius R2 between TV 101 and TV 103, that is, D2≤R2; (2). The distance D4 between TV 102 and TV 103 is less than or equal to the corresponding maximum combination radius R4 between TV 102 and TV 103, that is, D4≤R4.

[0242] The TV 101 compares the distance D2 between the TV 101 and the TV 103, and the magnitude of the corresponding maximum combined radius R2 between the TV 101 and the TV 103. Optionally, the TV 101 can also obtain the distance D4 between the TV 102 and the TV 103, and the information of the corresponding maximum combined radius R4 between the TV 102 and the TV 103 from the TV 102.

[0243] If the TV 101 determines that the distance D2 between the TV 101 and the TV 103 is less than or equal to the maximum combined radius R2 between the TV 101 and the TV 103, i.e., D2 ≤ R2, the TV 101 determines that the TV 101, the TV 102, and the TV 103 can form a screen group. Among them, the determination process of the maximum combined radius R2 between the TV 101 and the TV 103 can refer to the relevant description in step 406a, which will not be elaborated here.

[0244] The TV 102 can compare the distance D4 between the TV 102 and the TV 103, and the magnitude of the corresponding maximum combined radius R4 between the TV 102 and the TV 103. If D4 > R4, the TV 102 can also obtain the information of D2 ≤ R2 from the TV 101, so as to determine that the TV 103 needs to be added to the current screen group.

[0245] Exemplarily, in the scenario where three devices such as the TV 101, the TV 102, and the TV 103 are combined, the TV 101 and the TV 102 can be within the corresponding maximum combined radius (i.e., the maximum combined version radius corresponding to the TV 101 and the TV 102), the TV 101 and the TV 103 can be within the corresponding maximum combined radius (i.e., the maximum combined version radius corresponding to the TV 101 and the TV 103), and the TV 102 and the TV 103 may not be within the corresponding maximum combined radius (i.e., the maximum combined version radius corresponding to the TV 102 and the TV 103). That is, the TV 103 can be indirectly spliced with the TV 102 (the distance D2 between the TV 103 and the TV 102 is greater than the maximum combined radius between the TV 103 and the TV 102), and the TV 103 can be directly spliced with the TV 101 (the distance D3 between the TV 103 and the TV 101 is less than or equal to the maximum combined radius between the TV 103 and the TV 101). Since the TV 101 and the TV 102 are spliced together, and the TV 101 and the TV 103 are spliced together, the TV 101, the TV 102, and the TV 103 are spliced together.

[0246] Televisions 101, 102, and 103 can perform orientation recognition pairwise (i.e., recognize the splicing mode between pairwise devices). For example, television 101 and television 102 can perform orientation recognition based on the captured photos (i.e., recognize whether television 101 and television 102 are spliced vertically, horizontally, or diagonally), television 101 and television 103 can perform orientation recognition based on the captured photos, and television 102 and television 103 can perform orientation recognition based on the captured photos.

[0247] It should be noted that if television 103 is a device that has joined the local area network, television 101 / television 102 can directly execute step 416a. If television 103 is a newly joined device in the local area network, then televisions 101 and 102 can establish a connection with television 103 based on the local area network and discover each other based on short-range communication technology; or, televisions 101 and 102 can establish a direct connection with television 103; televisions 101 and 102 can refresh the list of nearby devices, and television 103 can create a new list of nearby devices; then, television 101 / television 102 can execute step 416a.

[0248] 416a. Televisions 101 and 102 display a second prompt message, which is used to prompt the user that a new device has been detected in the current screen group.

[0249] Exemplarily, as shown in (b) of Figure 3F a pop-up window 305 can pop up on televisions 101 and 102, prompting the user that device 123xxx (123xxx is the ID of television 103) has been detected. The pop-up window 305 can include a "Yes" button 302 and a "No" button 303, so that the user can choose whether to add the device to the screen group. Optionally, television 103 can also prompt the user with the identification or ID of nearby devices. For example, television 103 can pop up a pop-up window 306, prompting the user that device 111xxx (111xxx can be the ID of television 101) has been detected. The pop-up window 306 can include a "Yes" button 302 and a "No" button 303, so that the user can choose whether to combine the current device with television 103 for screen combination.

[0250] 416b. The user clicks the button to agree to add the new device to the screen group.

[0251] In response to the user's operation of clicking the button to agree to add the new device to the screen group, step 417 can be executed.

[0252] 417. Televisions 101, 102, and 103 form a screen group, and the master device is elected as television 101.

[0253] The master device election process can refer to the description in step 409 and will not be elaborated here. In this case, when TV 101 is the master device, it can be understood as including a host for controlling the screen group and a screen for displaying images, and the host is integrated in TV 101.

[0254] The following takes TV 101 as the master device for illustration:

[0255] 418a. TV 101 sends a first notification message to TV 102.

[0256] The first notification message is used to notify TV 102 to take a photo and perform orientation recognition based on the photo it takes and the photos obtained from other devices.

[0257] 418b. TV 101 sends a second notification message to TV 103.

[0258] The second notification message is used to notify TV 103 to take a photo and perform orientation recognition based on the photo it takes and the photos obtained from other devices.

[0259] 419a. TV 101 takes a photo.

[0260] TV 101 can control the camera built in TV 101 to take a photo through the screen splicing management service.

[0261] 419b. After receiving the first notification message sent by the master device, TV 102 takes a photo.

[0262] TV 102 can control the camera built in TV 102 to take a photo through the screen splicing management service. TV 101 and TV 102 can negotiate to take photos at the same time.

[0263] 419c. After receiving the second notification message sent by the master device, TV 103 takes a photo.

[0264] TV 103 can control the camera built in TV 103 to take a photo through the screen splicing management service. TV 101 and TV 103 can negotiate to take photos at the same time.

[0265] It can be understood that TV 101, TV 102, and TV 103 can negotiate to take photos at the same time.

[0266] The photos taken by TV 101, TV 102, and TV 103 respectively can be shared, that is, steps 419d - 419i can be executed:

[0267] 419d. TV 101 sends the photo taken by TV 101 to TV 102.

[0268] 419e. The television 102 sends the photos taken by the television 102 to the television 103.

[0269] 419f. The television 102 sends the photos taken by the television 102 to the television 101.

[0270] 419g. The television 103 sends the photos taken by the television 103 to the television 102.

[0271] 419h. The television 101 sends the photos taken by the television 101 to the television 103.

[0272] 419i. The television 103 sends the photos taken by the television 103 to the television 101.

[0273] 420a. The television 101 respectively obtains the photos taken by the television 102 and the television 103, and identifies the orientation relationships between the television 101 and the television 102, and the television 101 and the television 103 respectively.

[0274] For the corresponding orientation identification process, reference can be made to the relevant description in step 412a, which will not be elaborated here.

[0275] 420b. The television 102 respectively obtains the photos taken by the television 101 and the television 103, and identifies the orientation relationships between the television 102 and the television 101, and the television 102 and the television 103 respectively.

[0276] For the corresponding orientation identification process, reference can be made to the relevant description in step 412a, which will not be elaborated here.

[0277] 420c. The television 103 respectively obtains the photos taken by the television 101 and the television 102, and identifies the orientation relationships between the television 103 and the television 101, and the television 103 and the television 102 respectively.

[0278] For the corresponding orientation identification process, reference can be made to the relevant description in step 412a, which will not be elaborated here.

[0279] 420d. The television 102 sends the orientation relationship between the television 102 and other devices to the television 101.

[0280] 420e. The television 103 sends the orientation relationship between the television 103 and other devices to the television 101.

[0281] 421. The master device determines the relative orientation relationships of all devices in the screen group.

[0282] Exemplarily, the relative orientation relationship between the television 101 and the television 102 can be, for example, that the television 101 is located in the upper, lower, left, right, upper left, lower left, upper right, lower right, etc. directions of the television 102.

[0283] If there are more than two devices in the screen group, the relative orientations between the devices are identified pairwise. Among them, every two devices within the maximum combination radius (i.e., the distance between two devices is less than or equal to the maximum combination radius corresponding to these two devices) can be regarded as a same-radius screen group. Identifying the relative orientation of each same-radius screen group can identify the orientation of each device relative to other devices.

[0284] For example, assume that there are 3 devices in the screen group, namely TV 101, TV 102, and TV 103. By identifying the relative orientations between the devices pairwise, the orientation of each device relative to other devices can be identified.

[0285] As Figure 11A shown, taking 3 devices arranged horizontally as an example, such as taking TV 101, TV 102, and TV 103 as an example, assume that the order of pairwise identification between the devices from left to right is TV 103 and TV 101, TV 101 and TV 102. The process of determining the sorting of these 3 devices can be: first traverse the orientations between pairwise devices to determine that TV 103 is on the leftmost side of the remaining two devices (TV 101, TV 102), then traverse the orientations between pairwise devices to determine that TV 101 is on the leftmost side of the remaining one device (TV 102), and then traverse the orientations between pairwise devices to determine that TV 102 is on the rightmost side. Finally, it is determined that the sorting of the 3 devices TV 103, TV 101, and TV 102 is (1), (2), (3), that is, from left to right are TV 103, TV 101, and TV 102.

[0286] In some cases, the orientation information between some devices is redundant. These information can be not used, or the redundant information can be referred to for verifying the recognition result. As Figure 11A shown, based on the orientations between TV 103 and TV 101, and between TV 101 and TV 102, the overall screen group orientation is recognized. At this time, the orientation information between TV 103 and TV 102 is redundant. Optionally, the overall screen group orientation can be verified based on the orientation between TV 103 and TV 102 to improve the accuracy of the overall screen group orientation recognition.

[0287] In addition, TV 101, TV 102, and TV 103 are also spliced vertically (stacked vertically). The relative orientation relationship of each device refers to the relevant description above and will not be elaborated here.

[0288] Another example, assume that there are 9 devices in the screen group, namely TV 101, TV 102, TV 103, TV 104, TV 105, TV 105, TV 107, TV 108, and TV 109. After identifying the orientation information between pairwise devices, the orientation relationships between pairwise devices can be statistically summarized to summarize the relative orientation relationships of all devices in the screen group.

[0289] As Figure 11B shown, taking three devices arranged horizontally as an example, such as taking TV 101, TV 102, and TV 103 as an example, the order of pairwise identification between devices from left to right is TV 101 and TV 102, TV 101 and TV 103, TV 102 and TV 103. The process of determining the sorting of these three devices can be as follows: First, read the relative orientation between TV 101 and TV 102 to determine that TV 101 is on the left side of TV 102. Then, read the relative orientation between TV 101 and TV 103 to determine that TV 103 is on the right side of TV 101. At this point, the relative orientation between TV 101 and TV 103 cannot be determined yet, and it is necessary to further read the relative orientation between TV 102 and TV 103. Finally, the sorting of the three devices, namely TV 101, TV 102, and TV 103, is determined as (1), (2), (3), that is, from left to right are TV 101, TV 102, and TV 103 respectively.

[0290] Taking three devices arranged diagonally as an example, such as taking TV 101, TV 105, and TV 109 as an example, the order of pairwise identification from the upper left to the lower right between devices is TV 101 and TV 105, TV 101 and TV 109, TV 105 and TV 109. After traversing the relative orientations of pairwise devices, it can be determined that the leftmost and uppermost device is TV 101, TV 105 is located at the lower right of TV 101, and TV 109 is located at the lower right of TV 105. Therefore, the sorting of the three devices, namely TV 101 and TV 109, TV 105 and TV 109, is finally determined as (1), (5), (9), that is, from the upper left to the lower right are TV 101, TV 105, and TV 109 respectively.

[0291] It should be noted that the above is an example of a method for determining the relative orientation relationship of all devices in the screen group. In fact, there are many other methods for determining the relative orientation relationship of all devices in the screen group, and the present application does not make any limitations.

[0292] In some other embodiments, step 420a - step 421 can be replaced by step S1:

[0293] S1. TV 101 respectively obtains the photos taken by TV 102 and TV 103, and respectively identifies the orientation relationship between TV 101 and TV 102, the orientation relationship between TV 101 and TV 103, and the orientation relationship between TV 102 and TV 103. That is, the master device can identify the orientation relationship of each device in the screen group. In this way, TV 102 and TV 103 do not need to perform orientation identification, which can save the power consumption of TV 102 and TV 103.

[0294] 422a. The master device synchronizes the basic information for screen group splicing to TV 102.

[0295] 422b. The master device synchronizes the basic information of the screen group splicing to TV 103.

[0296] The master device synchronizes the basic information of the screen group splicing to each device within the screen group.

[0297] Exemplarily, the current basic information of the screen group splicing may include the number of devices included in the screen group (e.g., 3), the MAC / ID of each device (e.g., the IDs of TV 101, TV 102, and TV 103), the master-slave information (e.g., the master device is TV 101, and the slave devices include TV 102 and TV 103), and the orientation information between devices (e.g., TV 103, TV 101, and TV 102 are spliced in sequence from left to right).

[0298] 422c. TV 101 determines the display information of TV 101, TV 102, and TV 103 respectively according to the basic information of the screen group splicing.

[0299] TV 101 can determine the display information of TV 101 and TV 102 respectively according to the basic information of the screen group splicing. That is, during the operation of the screen group system, the master device can realize the display arrangement of the screen group's picture output and the switching of the interface focus based on the basic information of the screen group splicing. For example, TV 101 can divide the display content of TV 101 into N parts (e.g., 3 parts) and then distribute them to each device in the screen group (e.g., TV 101 (itself), TV 102, and TV 103). Among them, N is less than or equal to the number of devices included in the screen group.

[0300] 422d. TV 101 sends the display information of TV 102 to TV 102.

[0301] 422e. TV 101 sends the display information of TV 103 to TV 103.

[0302] 422f. TV 101 displays the corresponding display screen according to the display information of TV 101.

[0303] 422g. TV 102 displays the corresponding display screen according to the display information of TV 102.

[0304] 422h. TV 103 displays the corresponding display screen according to the display information of TV 103.

[0305] In some embodiments, devices can be removed from a screen group. Herein, removing a device can be actively dismantling / removing some devices from the current screen group, or some devices are powered off and passively offline. For example, each device in the screen group can detect and confirm whether a certain device is offline based on a heartbeat link. Or, each device in the screen group can detect and determine whether a certain device is offline through short-range communication. Or, in response to a user's operation of manually deleting a certain device from the management interface in the screen group, the information of the device deleted by the user can be marked, so that each device in the screen group determines that a certain device is offline.

[0306] If the TV 103 is deleted (removed) from the screen group composed of the TV 101, the TV 102, and the TV 103, as Figure 12 shown, the screen group splicing method may further include:

[0307] 423a. Sending short-range signals between the TV 101 and the TV 102.

[0308] Short-range signals may be periodically sent between the TV 101 and the TV 102 to measure the distance between the TV 101 and the TV 102 according to the short-range signals.

[0309] 423b. Sending short-range signals between the TV 102 and the TV 103.

[0310] Short-range signals may be periodically sent between the TV 101 and the TV 103 to measure the distance between the TV 101 and the TV 103 according to the short-range signals.

[0311] 423c. The TV 101 / TV 102 deletes the TV 103 from the current screen group according to the short-range signals.

[0312] Exemplarily, if the TV 101 detects and determines whether the TV 103 is offline through short-range signals, the TV 101 can compare the distance D2 between the TV 101 and the TV 103, and the size of the corresponding maximum combination radius R2 between the TV 101 and the TV 103. If the TV 101 determines that the distance D2 between the TV 101 and the TV 103 is greater than the maximum combination radius R2 between the TV 101 and the TV 103, that is, D2 > R2, the TV 101 determines that the TV 101 and the TV 103 are not in a splicing state, and deletes the TV 103 from the screen group composed of the TV 101, the TV 102, and the TV 103.

[0313] The TV 102 can obtain the information of D2 > R2 from the TV 101, so as to determine that the TV 103 needs to be deleted from the current screen group.

[0314] 424. The TV 101 and the TV 102 display a third prompt message for prompting the user that a device has been removed from the current screen group.

[0315] In some embodiments, as Figure 3G As shown in (a) in FIG, the screen group consisting of the TV 101, the TV 102 and the TV 103 can display corresponding display content together. If it is necessary to delete the TV 103 from the screen group consisting of the TV 101, the TV 102 and the TV 103 (for example, when the TV 101, the TV 102 and the TV 103 are spliced together, the TV 103 is removed), for example, Figure 3G As shown in (b) of FIG, TV 101 and TV 102 may display a pop-up window 307, prompting the user that device 123xxx in the current screen group has been removed. Pop-up window 307 may include an OK button 308. In response to the user clicking OK button 308, TV 101 determines that the user already knows the information in pop-up window 307 and may hide pop-up window 307. Alternatively, pop-up window 307 may automatically disappear after appearing for a few seconds (e.g., 2 seconds) to avoid affecting the displayed content on TV 101 and TV 102.

[0316] In other embodiments, when TV 101 / TV 102 detects that TV 103 is removed, it can prompt the user that a device has been removed, and in response to the user confirming the operation of removing the device, the device can be removed from the current screen group. Figure 3H As shown in (a) in FIG, the screen group consisting of the TV 101, the TV 102 and the TV 103 can display corresponding display content together. If it is necessary to delete the TV 103 from the screen group consisting of the TV 101, the TV 102 and the TV 103 (for example, when the TV 101, the TV 102 and the TV 103 are spliced together, the TV 103 is removed), for example, Figure 3H As shown in (b) of FIG, TV 101, TV 102 and TV 103 may still maintain their previous display contents, and TV 101 and TV 102 may pop up a pop-up box 309, prompting the user to confirm whether to remove device 123xxx from the current screen group? The pop-up box 309 may include an OK button 310 and a Cancel button 311. In response to the user clicking the OK button 310, it is determined to remove TV 103 from the screen group. Figure 3H As shown in (c) of FIG, the TV 101 and the TV 102 can jointly display the corresponding display content (the display content of the TV 101 and the TV 102 can be determined by the processor of the TV 101 (the main device)), and the TV 103 can display the corresponding display content alone (the content displayed by the TV 103 can be determined by the processor of the TV 103). In addition, if the user clicks the cancel button 311, as shown in FIG. Figure 3H As shown in (b) in FIG. 1 , the TV 101 , the TV 102 and the TV 103 may still maintain their previous display contents.

[0317] It should be noted that if the removed device is the master device in the current screen group, the remaining devices in the screen group can re-elect the master device.

[0318] The master device refreshes the screen group splicing basic information and synchronizes it to all devices in the screen group, so that each device in the screen group knows which device has been removed from the screen group. Exemplarily, the refreshed screen group splicing basic information may include the number of devices included in the screen group (for example, 2), the MAC / ID of each device (for example, the IDs of TV 101 and TV 102), the master-slave information (for example, the master device is TV 101 and the slave device is TV 102), and the orientation information between devices (for example, TV 101 and TV 102 are in a left-right splicing state). During the operation of the screen group system, the master device can implement functions such as picture output display layout and interface focus switching based on the screen group splicing basic information. For example, TV 101 can divide the display content of TV 101 into N parts (for example, 2 parts) and then distribute them to each device in the screen group (for example, TV 101 (itself) and TV 102). Wherein, N is less than or equal to the number of devices included in the screen group.

[0319] It should be noted that when a device is removed from the screen group, it can be considered that the screen group has been reorganized, and the relative orientation relationship between the devices in the screen group can be re-determined. For example, steps 410-414 can be re-executed.

[0320] Based on the method provided in the embodiments of the present application, during the screen combination and splicing process, the built-in cameras of the devices can be used to take pictures, and the pictures taken by each device can be identified and compared. For example, the orientation of the photo where the overlapping area is located can be determined, and then the relative orientation relationship between the two devices can be identified, without the need for manual setting by the user, which can improve the user experience. Moreover, the embodiments of the present application can automatically identify the combination intention between devices and start the screen assembly program by dynamically monitoring the distance between devices, without the need for manual setting by the user, which is more intelligent and convenient.

[0321] In addition, in some embodiments, the orientation relationship between devices can be determined through human-computer interaction. For example, different actions (gestures) or objects can be used in the front area directly in front of the cameras of TV 101 and TV 102 to indicate the orientations of different devices. Exemplarily, as Figure 13 shown in (a) of, first, TV 101 and TV 102 can prompt the user to select the arrangement method between devices. The arrangement methods between devices can include, for example: (1) vertical arrangement; (2) horizontal arrangement. As Figure 13As shown in (b) of , in response to the user's selection (2) when arranged side by side, TV 101 and TV 102 can prompt the user that "in the area directly in front of the camera of the first device from the left, gesture 1, and in the area directly in front of the camera of the second device from the left, gesture 2". After the user reads the prompt, the user can make gesture 1 in the area directly in front of the camera of the first device from the left (e.g., TV 101), and make gesture 2 in the area directly in front of the camera of the second device from the left (e.g., TV 102). TV 101 can detect whether a human hand appears in the field of view of the camera. If it is determined that a human hand appears, an image can be captured. At the same time, TV 102 can detect whether a human hand appears in the field of view of the camera. If it is determined that a human hand appears, an image can be captured. TV 101 determines whether the gesture in the image it captured matches gesture 1 or gesture 2. If it matches gesture 1, it is determined that TV 101 is the first device from the left. TV 102 can determine whether the gesture in the image it captured matches gesture 1 or gesture 2. If it matches gesture 2, it is determined that TV 102 is the second device from the left. In this way, it can be determined that TV 101 is on the left of TV 102. In this way, the user's participation and interest in the screen splicing process can be improved.

[0322] As Figure 14 shown, an embodiment of the present application provides a screen combination method, which is applied to a screen splicing system. The screen splicing system includes at least two screens and a host. The at least two screens include a first screen and a second screen. The host is integrated in the first screen or the second screen; or the host is independent of the first screen or the second screen. The method includes:

[0323] 1401. The first screen and the second screen form a first screen group, and the first screen and the second screen are communicatively connected.

[0324] Optionally, before the first screen and the second screen form the first screen group, the method further includes: the first screen or the second screen sends a first short-range signal to each other at a preset frequency, and the first screen or the second screen determines the distance between the first screen and the second screen according to the received signal strength indication (RSSI) of the first short-range signal transmitted between the first screen and the second screen; when the distance between the first screen and the second screen is less than or equal to the maximum combination radius corresponding to the first screen and the second screen, the first screen and the second screen form the first screen group; wherein, the maximum combination radius corresponding to the first screen and the second screen is determined according to the sizes of the first screen and the second screen and the positions of the antennas.

[0325] Optionally, before the first screen and the second screen form the first screen group, the first screen and / or the second screen can display a first prompt message, and the first prompt message is used to prompt the user that a device has been detected nearby and ask whether to perform screen splicing.

[0326] 1402. The host sends a first instruction to the first screen.

[0327] In some embodiments, if the host is placed in the first screen (such as TV 101), the first instruction may be a signal sent by the host to the camera of TV 101.

[0328] 1403. The host sends a second instruction to the second screen.

[0329] In some embodiments, if the host is placed in the first screen (such as TV 101), TV 101 may send a second instruction to the second screen (such as TV 102). The second instruction may refer to the first notification message in the above text and will not be elaborated here.

[0330] 1404. The first screen captures a first image according to the first instruction.

[0331] Wherein, the first image refers to the image (photo / picture) captured by the first screen (such as TV 101).

[0332] 1405. The second screen captures a second image according to the second instruction.

[0333] Wherein, the second image refers to the image (photo / picture) captured by the second screen (such as TV 102).

[0334] 1406. Determine the orientation information of the first screen and the second screen according to the first image and the second image.

[0335] In some embodiments, determining the orientation information of the first screen and the second screen according to the first image and the second image includes: the first screen sends the first image to the second screen; the second screen sends the second image to the first screen; the first screen and the second screen respectively determine the orientation information of the first screen and the second screen according to the first image and the second image; the first screen and the second screen respectively send the orientation information determined by the first screen and the orientation information determined by the second screen to the host; the host determines the orientation information of the first screen and the second screen according to the orientation information determined by the first screen and the orientation information determined by the second screen.

[0336] In some other embodiments, determining the orientation information of the first screen and the second screen according to the first image and the second image includes: the first screen sends the first image to the host; the second screen sends the second image to the host; the host determines the orientation information of the first screen and the second screen according to the first image and the second image.

[0337] In a possible design, determining the orientation information of the first screen and the second screen based on the first image and the second image includes: performing image matching on the first image and the second image according to an image matching algorithm to determine the overlapping area of the first image and the second image; determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image and the orientation of the second image. Among them, the image matching algorithm includes at least one of the Scale-Invariant Feature Transform (SIFT) algorithm, the Speeded-Up Robust Features (SURF) algorithm, and the Fast Nearest Neighbor Search (FLANN) algorithm.

[0338] Exemplarily, if the overlapping area is located in the lower half of the first image and the upper half of the second image, it is determined that the first screen is above the second screen; if the overlapping area is located in the lower left corner of the first image and the upper right corner of the second image, it is determined that the first screen is in the upper right of the second screen; if the overlapping area is located in the left half of the first image and the right half of the second image, it is determined that the first screen is to the right of the second screen; if the overlapping area is located in the upper left corner of the first image and the lower right corner of the second image, it is determined that the first screen is in the lower right of the second screen; if the overlapping area is located in the upper half of the first image and the lower half of the second image, it is determined that the first screen is below the second screen; if the overlapping area is located in the upper right corner of the first image and the lower left corner of the second image, it is determined that the first screen is in the lower left of the second screen; if the overlapping area is located in the right half of the first image and the left half of the second image, it is determined that the first screen is to the left of the second screen; if the overlapping area is located in the lower right area of the first image and the upper left area of the second image, it is determined that the first screen is in the upper left of the second screen.

[0339] In another possible design, if it is determined that the first image and the second image include a target object, the orientation of the first screen relative to the second screen is determined according to the orientation of the target object in the first image and the second image.

[0340] In still other embodiments, before the first screen captures the first image according to the first instruction and the second screen captures the second image according to the second instruction, the method further includes: the host sends layout information to the first screen and the second screen, and the layout information includes at least one combination mode; in response to the user's operation of selecting one combination mode from at least one combination mode, the host sends operation information to the first screen and the second screen, and the first screen and / or the second screen instructs the user to perform a first gesture or action at a first position and a second gesture or action at a second position according to the operation information; determining the orientation information of the first screen and the second screen according to the first image and the second image includes: if it is determined that the area including the first gesture or action in the first image is greater than or equal to a preset threshold, it is determined that the first screen is at the first position; if it is determined that the area including the second gesture or action in the second image is greater than or equal to a preset threshold, it is determined that the second screen is at the second position.

[0341] Optionally, the first screen or the second screen rates the resource status of the first screen and the second screen; wherein, the resource status includes at least one of the processing capabilities of a central processing unit (CPU), the storage capabilities of a read-only memory (ROM), or the storage capabilities of a random access memory (RAM); if the rating of the first screen is higher than that of the second screen, the host is integrated into the first screen; if the rating of the second screen is higher than that of the first screen, the host is integrated into the second screen.

[0342] Optionally, the host determines the display information corresponding to the first screen and the second screen according to the orientation information of the first screen and the second screen; the host sends the display information corresponding to the first screen to the first screen; the first screen displays the corresponding display screen according to the display information corresponding to the first screen; the host sends the display information corresponding to the second screen to the second screen; after receiving the display information corresponding to the second screen, the second screen displays the corresponding display screen according to the display information corresponding to the second screen.

[0343] Optionally, the screen splicing system further includes a third screen, and the method further includes: the first screen and the third screen send second short-range signals to each other, and the second screen and the third screen send third short-range signals to each other; determining the distance between the first screen and the third screen according to the RSSI of the second short-range signal; determining the distance between the second screen and the third screen according to the RSSI of the third short-range signal; when the distance between the first screen and the third screen is less than or equal to the maximum combination radius corresponding to the first screen and the third screen, forming a second screen group with the first screen, the second screen, and the third screen; wherein, the maximum combination radius corresponding to the first screen and the third screen is determined according to the sizes of the first screen and the third screen and the positions of the antennas; or when the distance between the second screen and the third screen is less than or equal to the maximum combination radius corresponding to the second screen and the third screen, forming a second screen group with the first screen, the second screen, and the third screen; wherein, the maximum combination radius corresponding to the second screen and the third screen is determined according to the sizes of the second screen and the third screen and the positions of the antennas.

[0344] Optionally, the first screen and / or the second screen displays a second prompt message, and the second prompt message is used to prompt the user that a new device is detected and ask whether to perform screen splicing.

[0345] In a possible design, if the first condition is met, the method further includes: the first screen and / or the second screen displays a third prompt message, and the third prompt message is used to prompt the user that the third screen has been removed from the current screen group.

[0346] Among them, the first condition includes: the heartbeat connection between the third screen and the first screen is disconnected, or the heartbeat connection between the third screen and the second screen is disconnected; or the host receives an operation from the user to delete the third screen; or the distance between the first screen and the third screen is greater than the maximum combined radius corresponding to the first screen and the third screen; or the distance between the second screen and the third screen is greater than the maximum combined radius corresponding to the second screen and the third screen.

[0347] After the third screen is removed from the current screen group, the method further includes: the host re-determines the display information corresponding to the first screen and the second screen respectively according to the orientation information of the first screen and the second screen.

[0348] It should be noted that Figure 14 the first screen in the above embodiments may be the TV 101 in the previous embodiments, the second screen may be the TV 102, and the third screen may be the TV 103. Figure 14 For the parts not detailed in the above embodiments, reference may be made to the previous embodiments, and details are not described herein again.

[0349] Based on the method provided in the embodiments of the present application, during the process of screen combination and splicing, a camera built in the device (the first screen or the second screen) can be used to take pictures, and the pictures taken by each device are identified and compared. For example, the orientation of the picture where the overlapping area is located can be determined, and then the relative orientation relationship between the two devices can be identified, without the need for manual setting by the user, which can improve the user experience. Moreover, the embodiments of the present application can automatically identify the combination intention between devices and start the screen assembly program by dynamically monitoring the distance between devices, without the need for manual setting by the user, which is more intelligent and convenient.

[0350] Another embodiment of the present application provides a chip system, as Figure 15 shown, the chip system includes at least one processor 1501 and at least one interface circuit 1502. The processor 1501 and the interface circuit 1502 can be interconnected by lines. For example, the interface circuit 1502 can be used to receive signals from other devices (for example, the memory of the first screen, the memory of the second screen, or the memory of the third screen). Again, for example, the interface circuit 1502 can be used to send signals to other devices (for example, the processor 1501).

[0351] For example, the interface circuit 1502 can read the instructions stored in the memory of the device and send the instructions to the processor 1501. When the instructions are executed by the processor 1501, the first screen or the second screen (such as Figure 2A the screen 110 shown) can execute each step in the above embodiments.

[0352] Of course, the chip system may also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0353] Some other embodiments of the present application provide a first screen (such as the screen 110 shown in ​ Figure). The first screen may include a communication module, a memory, and one or more processors. The communication module, the memory are coupled to the processor. The memory is used to store computer program code, and the computer program code includes computer instructions.

[0354] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When the computer instructions run on the first screen or the second screen (such as the screen 110 shown in ​ Figure), the screen 110 is caused to execute each function or step that the television 101 or the television 102 executes in the above method embodiments.

[0355] Embodiments of the present application also provide a computer program product. When the computer program product runs on a computer, the computer is caused to execute each function or step that the first screen (for example, the television 101) or the second screen (for example, the television 102) executes in the above method embodiments.

[0356] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0357] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the module or unit is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0358] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0359] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0360] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0361] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A screen combination method is applied to a screen splicing system. The screen splicing system includes at least two screens and a host. The at least two screens include a first screen and a second screen. It is characterized in that, Including: The first screen and the second screen form a first screen group, and the first screen and the second screen are communicatively connected; The host sends a first instruction to the first screen and a second instruction to the second screen; The first screen captures a first image according to the first instruction; The second screen captures a second image according to the second instruction; Determine the orientation information of the first screen according to the first image; Determine the orientation information of the second screen according to the second image; Wherein, if it is determined that the area of the first gesture or action in the first image is greater than or equal to a preset threshold, it is determined that the first screen is located at a first position; if it is determined that the area of the second gesture or action in the second image is greater than or equal to the preset threshold, it is determined that the second screen is located at a second position.

2. The method according to claim 1, wherein Determine the orientation information of the first screen according to the first image; Determine the orientation information of the second screen according to the second image, including: The first screen sends the first image to the second screen; The second screen sends the second image to the first screen; The first screen determines the orientation information of the first screen according to the first image; the second screen determines the orientation information of the second screen according to the second image; The first screen sends the orientation information determined by the first screen to the host; The second screen sends the orientation information determined by the second screen to the host; The host determines the orientation information of the first screen and the second screen according to the orientation information determined by the first screen and the orientation information determined by the second screen.

3. The method according to claim 1 or 2, characterized in that, Determine the orientation information of the first screen according to the first image; Determine the orientation information of the second screen according to the second image, including: The first screen sends the first image to the host; The second screen sends the second image to the host; The host determines the orientation information of the first screen according to the first image; determines the orientation information of the second screen according to the second image.

4. The method according to claim 1 or 2, characterized in that, Before the first screen and the second screen form a first screen group, the method further includes: The first screen or the second screen mutually sends a first short-range signal at a preset frequency, and the first screen or the second screen determines the distance between the first screen and the second screen according to the received signal strength indication (RSSI) of the first short-range signal transmitted between the first screen and the second screen; When the distance between the first screen and the second screen is less than or equal to the maximum combined radius corresponding to the first screen and the second screen, the first screen and the second screen form the first screen group; Wherein, the maximum combined radius corresponding to the first screen and the second screen is determined according to the sizes of the first screen and the second screen and the positions of the antennas.

5. The method according to claim 1 or 2, characterized in that, Before the first screen and the second screen form a first screen group, the method further includes: The first screen and / or the second screen display a first prompt message; The first screen and / or the second screen obtain an instruction from the user, and the instruction is used to confirm screen splicing.

6. The method according to claim 1 or 2, characterized in that, Determining the orientation information of the first screen and the second screen based on the first image and the second image includes: Performing image matching on the first image and the second image according to an image matching algorithm to determine an overlapping area of the first image and the second image; Determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image and the orientation of the second image.

7. The method according to claim 6, characterized in that Determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image includes: If the overlapping area is located in the lower half area of the first image and in the upper half area of the second image, determining that the first screen is located above the second screen; If the overlapping area is located in the lower left corner of the first image and in the upper right corner of the second image, determining that the first screen is located in the upper right of the second screen; If the overlapping area is located in the left half area of the first image and in the right half area of the second image, determining that the first screen is located on the right of the second screen; If the overlapping area is located in the upper left corner of the first image and in the lower right corner of the second image, determining that the first screen is located in the lower right of the second screen; If the overlapping area is located in the upper half area of the first image and in the lower half area of the second image, determining that the first screen is located below the second screen; If the overlapping area is located in the upper right corner of the first image and in the lower left corner of the second image, determining that the first screen is located in the lower left of the second screen; If the overlapping area is located in the right half area of the first image and in the left half area of the second image, determining that the first screen is located on the left of the second screen; If the overlapping area is located in the lower right area of the first image and in the upper left area of the second image, determining that the first screen is located in the upper left of the second screen.

8. The method according to claim 6, wherein The image matching algorithm includes at least one of Scale-Invariant Feature Transform (SIFT) algorithm, Speeded-Up Robust Features (SURF) algorithm, and Fast Nearest Neighbor Search (FLANN) algorithm.

9. The method according to claim 1 or 2, characterized in that, Determining the orientation information of the first screen and the second screen based on the first image and the second image includes: If it is determined that the first image and the second image include a target object, determining the orientation of the first screen relative to the second screen according to the orientation of the target object in the first image and the second image.

10. The method according to claim 1 or 2, characterized in that, Before the first screen captures a first image according to the first instruction and the second screen captures a second image according to the second instruction, the method further includes: The host sends layout information to the first screen and the second screen, and the layout information includes at least one combination mode; In response to an operation in which a user selects a combination mode from the at least one combination mode, the host sends operation information to the first screen and the second screen, and the first screen and / or the second screen instructs the user to perform a first gesture or action at a first position and a second gesture or action at a second position according to the operation information.

11. The method according to claim 1 or 2, characterized in that, The host is integrated in the first screen or the second screen, and the first screen and the second screen form a first screen group. The method further includes: The first screen or the second screen scores the resource conditions of the first screen and the second screen; wherein, the resource conditions include at least one of the processing capabilities of a central processing unit (CPU), the storage capabilities of a read-only memory (ROM), or the storage capabilities of a random access memory (RAM). If the score of the first screen is higher than the score of the second screen, the host is integrated in the first screen. If the score of the second screen is higher than the score of the first screen, the host is integrated in the second screen.

12. The method according to claim 1 or 2, characterized in that, The method further includes: The host determines the display information corresponding to the first screen and the second screen according to the orientation information of the first screen and the second screen. The host sends the display information corresponding to the first screen to the first screen. The first screen displays a corresponding display screen according to the display information corresponding to the first screen. The host sends the display information corresponding to the second screen to the second screen. After receiving the display information corresponding to the second screen, the second screen displays a corresponding display screen according to the display information corresponding to the second screen.

13. The method according to claim 1 or 2, characterized in that, The screen splicing system further includes a third screen. The method further includes: The first screen and the third screen send second short-range signals to each other. The second screen and the third screen send third short-range signals to each other. Determine the distance between the first screen and the third screen according to the RSSI of the second short-range signal; determine the distance between the second screen and the third screen according to the RSSI of the third short-range signal. When the distance between the first screen and the third screen is less than or equal to the maximum combination radius corresponding to the first screen and the third screen, form a second screen group with the first screen, the second screen, and the third screen; wherein, the maximum combination radius corresponding to the first screen and the third screen is determined according to the sizes of the first screen and the third screen and the positions of the antennas; or When the distance between the second screen and the third screen is less than or equal to the maximum combination radius corresponding to the second screen and the third screen, form a second screen group with the first screen, the second screen, and the third screen; wherein, the maximum combination radius corresponding to the second screen and the third screen is determined according to the sizes of the second screen and the third screen and the positions of the antennas.

14. The method according to claim 13, characterized in that The method further includes: The first screen and / or the second screen display second prompt information. The first screen and / or the second screen obtains an instruction from the user for confirming screen splicing.

15. The method according to claim 13, wherein The method further includes: The first screen and / or the second screen detect whether a first condition is satisfied; If the first condition is satisfied, the first screen and / or the second screen remove the third screen from the second screen group.

16. The method according to claim 15, wherein The first condition includes: The heartbeat connection between the third screen and the first screen is disconnected, or the heartbeat connection between the third screen and the second screen is disconnected; or The host receives an operation by the user to delete the third screen; or The distance between the first screen and the third screen is greater than the maximum combination radius corresponding to the first screen and the third screen; or the distance between the second screen and the third screen is greater than the maximum combination radius corresponding to the second screen and the third screen.

17. A screen splicing system, the screen splicing system includes at least two screens and a host, the at least two screens include a first screen and a second screen, the first screen and the second screen form a first screen group, the first screen and the second screen are communicatively connected, and it is characterized in that, Includes: The host sends a first instruction to the first screen, and the first instruction is used to instruct the first screen to capture a first image; The host sends a second instruction to the second screen, and the second instruction is used to instruct the second screen to capture a second image; The host determines the orientation information of the first screen according to the first image; Determine the orientation information of the second screen according to the second image; Wherein, if it is determined that the area of the first gesture or action in the first image is greater than or equal to a preset threshold, it is determined that the first screen is located at a first position; if it is determined that the area of the second gesture or action in the second image is greater than or equal to the preset threshold, it is determined that the second screen is located at a second position.

18. The screen splicing system according to claim 17, wherein The host determines the orientation information of the first screen according to the first image; Determining the orientation information of the second screen according to the second image includes: The host receives the first image from the first screen; The host receives the second image from the second screen; The host determines the orientation information of the first screen according to the first image; determines the orientation information of the second screen according to the second image.

19. The screen splicing system according to claim 17 or 18, characterized in that, Determining the orientation information of the first screen and the second screen according to the first image and the second image includes: The host performs image matching on the first image and the second image according to an image matching algorithm to determine the overlapping area of the first image and the second image; Determine the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image and the orientation of the second image.

20. The screen splicing system according to claim 19, wherein Determining the orientation of the first screen relative to the second screen according to the orientation of the overlapping area in the first image includes: If the overlapping area is located in the lower half area of the first image and in the upper half area of the second image, it is determined that the first screen is located above the second screen; If the overlapping area is located in the lower left corner of the first image and in the upper right corner of the second image, it is determined that the first screen is located in the upper right of the second screen; If the overlapping area is located in the left half area of the first image and in the right half area of the second image, it is determined that the first screen is located on the right side of the second screen; If the overlapping area is located at the upper left corner of the first image and at the lower right corner of the second image, it is determined that the first screen is located at the lower right of the second screen; If the overlapping area is located in the upper half of the first image and in the lower half of the second image, it is determined that the first screen is located below the second screen; If the overlapping area is located at the upper right corner of the first image and at the lower left corner of the second image, it is determined that the first screen is located at the lower left of the second screen; If the overlapping area is located in the right half of the first image and in the left half of the second image, it is determined that the first screen is located to the left of the second screen; If the overlapping area is located in the lower right area of the first image and in the upper left area of the second image, it is determined that the first screen is located at the upper left of the second screen.

21. The screen splicing system according to claim 19, wherein the image matching algorithm includes at least one of Scale-Invariant Feature Transform (SIFT) algorithm, Speeded-Up Robust Features (SURF) algorithm, and Fast Nearest Neighbor Search (FLANN) algorithm.

22. The screen splicing system according to claim 17, wherein Determining the orientation information of the first screen and the second screen according to the first image and the second image includes: If it is determined that the first image and the second image include a target object, the orientation of the first screen relative to the second screen is determined according to the orientation of the target object in the first image and the second image.

23. The screen splicing system according to claim 17, wherein Before the first screen takes the first image according to the first instruction and the second screen takes the second image according to the second instruction, it further includes: The host sends layout information to the first screen and the second screen, and the layout information includes at least one combination mode; In response to the user's operation of selecting one combination mode from the at least one combination mode, the host sends operation information to the first screen and the second screen, and the operation information is used to instruct the user to perform a first gesture or action at a first position and a second gesture or action at a second position.

24. The screen splicing system according to claim 17 or 18, characterized in that, It further includes: The host determines the display information corresponding to the first screen and the second screen according to the orientation information of the first screen and the second screen; The host sends the display information corresponding to the first screen to the first screen; The host sends the display information corresponding to the second screen to the second screen.

25. An electronic device, characterized in that, The electronic device includes: a display screen, a wireless communication module, a memory, and one or more processors; the wireless communication module, the memory are coupled to the processor; wherein, the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device is caused to execute the method according to any one of claims 1-16.

26. A computer-readable storage medium, characterized in that, including computer instructions; When the computer instructions run on the electronic device, the electronic device is caused to execute the method according to any one of claims 1-16.

Images