A portable machine multi-screen cooperative display control method and system supporting spliced screens

Abstract

This invention discloses a method and system for controlling multi-screen collaborative display on a portable device that supports spliced ​​screens, relating to the field of display control technology. The method includes the following steps: setting the current portable device as the main control display device; when a collaborative display device is detected entering a preset near-field control range of the portable device, exchanging device identification data between the main control display device and the collaborative display device in real time; the device identification data includes native display attributes, display color and brightness references, display encoding / decoding capability matrix, and display space topology seed; virtually mapping the main control display device and the collaborative display device to obtain a virtual coordinate system for the screen display; this invention ensures absolute logical continuity of lines and shapes; combined with an edge coupling region compensation mechanism, it performs smooth transition processing of brightness and color for adjacent edge pixels, filling the visual blanks caused by physical borders, making the physical gaps between screens imperceptible to the user's naked eye.

Description

Technical Field

[0001] This invention relates to the field of display control technology, and in particular to a method and system for controlling multi-screen collaborative display on a portable computer that supports spliced ​​screens. Background Technology

[0002] With the increasing demand for mobile office and multimedia entertainment, the performance of portable devices is constantly improving. However, due to the requirement of portability, the physical size of a single screen is often insufficient to meet users' display needs for a large field of view and multi-tasking in parallel. Therefore, using multiple portable devices for screen splicing or multi-screen collaboration has become an important means to improve display efficiency.

[0003] However, existing technologies mostly adopt a mode of full rendering on the main control end combined with video streaming transmission, which not only consumes high bandwidth and has a heavy load on the main control, but also lacks effective compensation for color differences between heterogeneous screens, parallax of physical seam space and display timing synchronization, resulting in problems such as inconsistent image quality, line misalignment and tearing stuttering when displaying across screens.

[0004] Therefore, there is a need for a portable multi-screen collaborative display control method that improves screen display speed and optimizes display effect. Summary of the Invention

[0005] This invention aims to provide a portable multi-screen collaborative display control method and system that supports spliced ​​screens. By using virtual coordinate mapping based on spatial topology and heterogeneous capabilities, full-screen rendering is decoupled into a distributed vector instruction set. By utilizing global chroma and brightness mapping and timestamp synchronization mechanism, pixel-level alignment, seamless splicing, and low-latency collaboration of cross-screen display are achieved.

[0006] A method for controlling multi-screen collaborative display on a portable computer that supports spliced ​​screens includes the following steps:

[0007] The current portable device is set as the main control display device; when a cooperating display device is detected to enter the preset near-field control range of the portable device, the device identification data of the main control display device and the cooperating display device are exchanged in real time; the device identification data includes native display attributes, display color and brightness reference, display encoding and decoding capability matrix and display space topology seed;

[0008] Virtual mapping is performed between the main control display device and the collaborative display device to obtain a screen display virtual coordinate system. When the coordinate vector of the target display object is captured to be within the preset screen boundary of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output.

[0009] Based on the device identification data, the main control rendering instruction set is localized for rasterization rendering to obtain the main control image frame; at the same time, the collaborative display device is independently rasterized for rasterization rendering based on the device identification data to generate the collaborative image frame; based on the main control image frame and the collaborative image frame, heterogeneous rendering compensation is performed to obtain the modified target display object frame, thus completing the display control of the multi-screen collaboration of the portable device.

[0010] As a preferred embodiment of the present invention, the specific steps for real-time exchange of device identification data between the main control display device and the collaborative display device when the collaborative display device is detected to have entered the preset near-field control range of the portable device include:

[0011] When the main control display device enters the splicing screen connection stage, the main control display device continuously sends out external detection frequency band beacons; when the cooperative display device is detected to enter the preset portable near-field control range, the cooperative display device establishes a direct hardware connection channel for the screen through the external detection frequency band beacon; in the direct hardware connection channel for the screen, compressed and structured device identification data are exchanged.

[0012] The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. When the main control display device and the cooperating display device receive the device identification data, a heterogeneous rendering context environment is generated.

[0013] As a preferred embodiment of the present invention, the specific steps of outputting the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device include:

[0014] Construct a virtual coordinate system for the display; obtain the initial physical edge distance and relative angle between the main display device and the cooperating display device based on the display space topology seed; construct a 3D rotation matrix based on the relative angle, rotate the local planar coordinate system of the cooperating display device to an aligned state with the main display device, and determine the normal vector direction of the cooperating display device in the virtual coordinate system; calculate the visual compensation blind zone width based on the initial physical edge distance; translate the starting coordinates of the cooperating display device in the virtual coordinate system by the visual compensation blind zone width along a direction away from the main display device; obtain the corrected virtual coordinate system for the display based on the normal vector direction and the visual compensation blind zone width.

[0015] Based on the corrected virtual coordinate system of the screen display, obtain the virtual line mapping of the physical seam between the main control display device and the collaborative display device; using the virtual line mapping of the physical seam as the cutting reference, cut the target display object into a set of main control sub-elements located on the main control display device and a set of collaborative sub-elements located on the collaborative display device;

[0016] Extract the vertex coordinates of the master control sub-element set and the cooperative sub-element set in the on-screen virtual coordinate system to obtain the master control vertex coordinates and the cooperative vertex coordinates; map the master control vertex coordinates and the cooperative vertex coordinates to the on-screen virtual coordinate system to obtain the master control local graph data and the cooperative local graph data;

[0017] Obtain the display encoding and decoding capability matrix, serialize and encapsulate the collaborative local graph data according to the collaborative display device, and generate a collaborative rendering instruction set; combine the native display attributes of the main control display device, convert the main control local graph data, and generate a main control rendering instruction set.

[0018] As a preferred embodiment of the present invention, the specific steps for heterogeneous rendering compensation based on the master control image frame and the cooperative image frame include:

[0019] The display optoelectronic transmission differences between the main control display device and the cooperating display devices are determined based on the display color and brightness benchmark; a global color and brightness mapping lookup table is constructed based on the display optoelectronic transmission differences.

[0020] Obtain the master rendering parameters of the master image frame; based on the master rendering parameters, use the global chroma-luminance mapping lookup table to perform pixel-level remapping on the rendering pipeline of the generated collaborative image frame to obtain the color temperature gamma response curve for unifying the collaborative image frame and the master image frame.

[0021] Based on the virtual lines mapped from the physical seams of the devices, edge coupling regions are defined at the adjacent edges of the master image frame and the cooperative image frame; rendering compensation is performed on the edge coupling regions to obtain the compensated edge mask.

[0022] As a preferred embodiment of the present invention, the specific steps for rendering compensation of edge coupling regions include:

[0023] The physical pixel density ratio between the main display device and the cooperating display device is obtained based on the native display attributes; a compensation edge mask is generated in the edge coupling area based on the physical pixel density ratio; the brightness of the edge pixels of the main image frame and the cooperating image frame is compensated using the compensation edge mask to obtain the compensation image edge frame;

[0024] Based on the color temperature gamma response curve and the compensation image edge frame, the collaborative image frame and the master image frame are optimized to obtain the modified target display object frame.

[0025] As a preferred technical solution of the present invention, when completing the display control of multi-screen collaboration of a portable device, it is necessary to perform content attribute analysis on the collaborative sub-element set; if the collaborative sub-element set contains dynamic video textures, then the collaborative sub-element set is subjected to local compression control; if the collaborative sub-element set contains static vector elements, then the collaborative sub-element set is subjected to lossless vector format control; a hybrid collaborative rendering instruction set is generated based on local compression control or lossless vector format control.

[0026] The same global display timestamp based on the heterogeneous rendering context is attached to the main rendering instruction set and the collaborative rendering instruction set. After receiving the collaborative rendering instruction set or the hybrid collaborative rendering instruction set, the collaborative display device performs a rendering instruction suspension operation until the local clock is aligned with the global display timestamp, and then triggers independent rasterization rendering.

[0027] Meanwhile, the main control display device releases idle display resources according to the cutting benchmark, and the collaborative display device preloads display resources according to the collaborative rendering instruction set or the hybrid collaborative rendering instruction set.

[0028] A portable multi-screen collaborative display control system supporting splicing screens includes:

[0029] The multi-screen collaboration analysis module includes a display analysis unit and a display mapping unit. The display analysis unit sets the current portable device as the main control display device. When a collaborative display device is detected entering the preset near-field control range of the portable device, the device identification data of the main control display device and the collaborative display device are exchanged in real time. The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. The display mapping unit is used to virtually map the main control display device and the collaborative display device to obtain a virtual coordinate system for the screen display. When the coordinate vector of the target display object is captured to be within the preset screen boundary of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output.

[0030] The multi-screen collaboration control module includes a display control unit. The display control unit performs local rasterization rendering of the main control rendering instruction set based on device identification data to obtain the main control image frame. At the same time, the collaborative display device performs independent rasterization rendering based on the device identification data to generate collaborative image frames. Heterogeneous rendering compensation is performed based on the main control image frame and the collaborative image frame to obtain the modified target display object frame, thus completing the display control of the multi-screen collaboration of the portable device.

[0031] The present invention has the following advantages:

[0032] 1. This invention introduces a unified virtual coordinate system based on physical pixel density normalization and a dynamic geometric cutting algorithm to identify and process graphic objects that cross the physical black border of the screen. It uses linear interpolation to generate new vertices at the cutting boundary, ensuring the absolute logical continuity of lines and shapes. At the same time, combined with the compensation mechanism for edge coupling regions, a Gaussian weighted fusion mask is generated based on the ratio of the physical pixel density of the two screens to perform smooth transition processing of brightness and color for adjacent edge pixels, effectively filling the visual blank caused by the physical border, so that the physical gap between the screens cannot be perceived by the naked eye.

[0033] 2. This invention constructs a global chromaticity and luminance mapping lookup table and utilizes a polynomial fitting algorithm to quantify the photoelectric transmission differences between the main control screen and the collaborative screen. It also forces the nonlinear gamma curve and color gamut response of the collaborative screen to be mapped to the standard color space of the main control screen, unifying the color temperature, contrast, and grayscale response of the two screens. Furthermore, it eliminates color differences caused by variations in panel materials and backlight modules, ensuring that the same image presents a completely consistent visual effect on different screens. This significantly improves the quality of collaborative work in color-sensitive scenarios such as professional design and film editing. Through in-depth content attribute analysis of the collaborative sub-primitive set, it distinguishes between dynamic video textures and static vector elements, and employs local compression control and lossless vector format control strategies respectively. This reduces data transmission bandwidth usage and improves rendering efficiency, achieving an optimal balance between dynamic content and static interface. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of a portable multi-screen collaborative display control system that supports splicing screens, as used in an embodiment of the present invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0036] Example 1: A method for controlling multi-screen collaborative display on a portable computer that supports spliced ​​screens, comprising the following steps:

[0037] The current portable device is set as the main control display device; when a cooperating display device is detected to enter the preset near-field control range of the portable device, the device identification data of the main control display device and the cooperating display device are exchanged in real time; the device identification data includes native display attributes, display color and brightness reference, display encoding and decoding capability matrix and display space topology seed;

[0038] In actual implementation, the portable computer (such as a laptop or high-performance tablet) currently being operated by the user is logically defined as the main control display device, which serves as the command center of the multi-screen collaborative network.

[0039] When a collaborative display device is detected entering the preset near-field control range of the portable device, the specific steps for real-time exchange of device identification data between the main control display device and the collaborative display device include:

[0040] When the main control display device enters the splicing screen connection stage, the main control display device continuously sends out external detection frequency band beacons; when the cooperative display device is detected to enter the preset portable near-field control range, the cooperative display device establishes a direct hardware connection channel for the screen through the external detection frequency band beacon; in the direct hardware connection channel for the screen, compressed and structured device identification data are exchanged.

[0041] Utilizing the low-power near-field communication (NFC) hardware module built into the main control display device, which does not rely on any user-installed third-party applications but is directly controlled by the operating system's underlying driver, the NFC module continuously broadcasts a special external probe frequency beacon. This external probe frequency beacon is a radio pulse signal with extremely simplified encoding, containing only the device's basic identity verification code and synchronization clock information. It aims to achieve a continuous presence announcement, similar to a lighthouse, with extremely low power consumption, ensuring that potential cooperating devices in the vicinity can be quickly detected without consuming a large amount of power. Furthermore, the transmission frequency of this signal is set to milliseconds to ensure that it can be captured the instant a device approaches, thus avoiding the several-second delay common in traditional wireless scanning technologies.

[0042] When the collaborative display device moves to the preset near-field control range of the portable device (usually referring to the effective interaction area within a physical distance of one to three meters), the receiving antenna of the collaborative display device will capture the aforementioned external detection frequency band beacon and immediately determine whether the distance meets the standard by measuring the signal strength or time of flight. Once it is confirmed that it has entered the range, the collaborative display device uses the handshake protocol information carried in the beacon to establish a direct hardware connection channel with the main control display device. The direct hardware connection channel is not a traditional general network connection based on Internet protocol, but a point-to-point private link optimized for display data exchange. It bypasses the complex routing and application layer negotiation process and directly establishes an encrypted data transmission channel between the network cards or dedicated chips of the two devices. The establishment process of the direct hardware connection channel takes only a few milliseconds, realizing a seamless connection and ensuring the real-time performance and security of subsequent large data exchange.

[0043] The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. When the main control display device and the cooperating display device receive the device identification data, a heterogeneous rendering context environment is generated.

[0044] After the screen hardware direct connection channel is successfully established, the two devices immediately begin exchanging compressed and structured device identification data. This is a technical means of integrating the originally scattered screen parameters into a single, efficient data packet. Compressed and structured means that the system first collects various raw parameters, then removes redundant information, such as repeated reserved fields, and rearranges and compresses them according to a predefined compact binary format. This reduces the information that originally required thousands of bytes to describe to a very small size, so that it can be transmitted instantly in the low-bandwidth initial connection stage, reducing the communication overhead of the handshake stage and providing channel resources for subsequent high-definition video streams or instruction set transmissions.

[0045] The exchanged device identification data specifically includes four types of core information. Native display attributes refer to the physical hard indicators determined when the screen panel is manufactured, including the screen's physical resolution (i.e., the total number of pixels horizontally and vertically), pixel density (i.e., the number of pixels arranged in a length of one inch, which determines the level of detail of the image), the panel's native refresh rate (the number of times the screen updates the image per second), and gray-to-gray response time (the time required for a pixel to change from one color to another). The above parameters determine the basic clarity and smoothness of the screen display.

[0046] Secondly, there is the display color brightness benchmark, which is calibration data used to describe the screen's color performance capabilities. It covers the color gamut range that the screen can display (i.e., the breadth of colors it can present), the maximum luminous brightness value, the factory-set white point coordinates (defining the standard color tendency of white), and the gamma curve (a curve describing the non-linear relationship between input signal strength and output brightness). The above data is used to eliminate visual color differences between screens of different brands.

[0047] The display codec capability matrix is ​​a list of technical parameters that details the types of image and video compression algorithms supported by the device hardware (such as specific lossless compression formats or efficient video coding standards) and the maximum data throughput bandwidth. This allows both parties to negotiate the optimal data packing and unpacking strategy before transmission.

[0048] The display space topology seed is a geometric data calculated based on the time difference or phase change of a signal propagating in the air. It is used to accurately record the real-time straight-line distance and relative rotation angle between the physical edges of the screens of two devices, which is equivalent to determining the precise relative positional relationship between the two screens in virtual space.

[0049] Once the main display device and the cooperating display device each receive complete device identification data, the system will immediately generate a heterogeneous rendering context environment. The heterogeneous rendering context environment is a temporary software running state. The system loads the received native display attributes, display color brightness reference and other parameters into the video memory, pre-builds the corresponding color lookup table (for real-time correction of color differences) and rasterization configuration template (for adapting to different resolutions and refresh rates), and converts the display space topology seed into mapping rules in the virtual coordinate system.

[0050] Virtual mapping is performed between the main control display device and the collaborative display device to obtain a screen display virtual coordinate system. When the coordinate vector of the target display object is captured to be within the preset screen boundary of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output.

[0051] The specific steps for outputting the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device include:

[0052] Construct a virtual coordinate system for the display; obtain the initial physical edge distance and relative angle between the main display device and the cooperating display device based on the display space topology seed; construct a 3D rotation matrix based on the relative angle, rotate the local planar coordinate system of the cooperating display device to an aligned state with the main display device, and determine the normal vector direction of the cooperating display device in the virtual coordinate system; calculate the visual compensation blind zone width based on the initial physical edge distance; translate the starting coordinates of the cooperating display device in the virtual coordinate system by the visual compensation blind zone width along a direction away from the main display device; obtain the corrected virtual coordinate system for the display based on the normal vector direction and the visual compensation blind zone width.

[0053] Specifically, the process of constructing the virtual coordinate system for the screen begins with a deep analysis of the topological seed of the display space. The system first extracts the initial physical edge distance value and relative angle between the main control display device and the cooperating display device. The initial physical edge distance value refers to the actual physical gap length between the center points of the black borders of the two screens, while the relative angle describes the non-parallel tilt state that the two screens may have when placed on the desktop.

[0054] Based on the acquired relative angle, a 3D rotation matrix is ​​constructed. This 3D rotation matrix is ​​a mathematical transformation tool used to rotate the local plane coordinate system of the cooperating display device, which may have been tilted in the logical space, so that it is in a completely coplanar aligned state with the main control display device in the virtual space. Based on this, the direction of the normal vector of the cooperating display device in the screen display virtual coordinate system is determined. This normal vector direction represents the vertical outward direction of the screen, ensuring that the lighting and viewpoint calculations for subsequent graphics rendering are consistent.

[0055] Subsequently, the system calculates the visual compensation blind zone width using the initial distance value of the physical edge. The visual compensation blind zone width is not a simple physical distance, but rather a buffer area that needs to be reserved in the virtual coordinates when the cursor flies within the physical black border, calculated by combining the user's typical viewing distance and human eye angle. In the screen display virtual coordinate system, the starting coordinates of the cooperating display device are translated away from the main control display device along the direction calculated above for the visual compensation blind zone width. This step ensures that a blank area with the same width as the actual physical black border is reserved between the two screens in the virtual coordinate system, preventing graphic elements from logically overlapping or stretching incorrectly. Combining the normal vector direction and the position information after translation correction, the system generates a corrected screen display virtual coordinate system. This coordinate system truly reflects the dual-screen spatial layout, including the physical gap, providing a geometric benchmark for subsequent precise cutting.

[0056] Based on the corrected virtual coordinate system of the screen display, obtain the virtual line mapping of the physical seam between the main control display device and the collaborative display device; using the virtual line mapping of the physical seam as the cutting reference, cut the target display object into a set of main control sub-elements located on the main control display device and a set of collaborative sub-elements located on the collaborative display device;

[0057] After obtaining the corrected virtual coordinate system, the system further acquires the virtual line mapping of the physical seam between the main display device and the cooperating display device. This virtual line is a logical dividing line located in the center of the physical black border of the two screens, clearly defining the boundary between the main control area and the cooperating area in the virtual coordinate system. When the system detects that the coordinate vector of the target display object (such as a window being dragged or a mouse pointer) has moved to the vicinity of this boundary, it uses this virtual line mapping as the cutting reference to perform a geometric cutting operation. The system dynamically splits the complete graphic data of the target display object into two independent parts: one part is the main control sub-primitive set located entirely or primarily on the side of the main control display device, and the other part is the cooperating sub-primitive set that crosses the boundary or is located entirely on the side of the cooperating display device. The sub-primitive set refers to the set of basic geometric units (such as triangles, lines, and texture blocks) that constitute the image. Through this cutting, the originally continuous window is logically and precisely divided into two parts, ensuring that each part of the graphic data strictly corresponds to the physical screen area to be displayed, avoiding cross-screen deformation caused by overall stretching in traditional technologies.

[0058] Extract the vertex coordinates of the master control sub-element set and the cooperative sub-element set in the on-screen virtual coordinate system to obtain the master control vertex coordinates and the cooperative vertex coordinates; map the master control vertex coordinates and the cooperative vertex coordinates to the on-screen virtual coordinate system to obtain the master control local graph data and the cooperative local graph data;

[0059] After completing the graphic segmentation, the vertex coordinates of the master sub-element set and the cooperative sub-element set in the screen display virtual coordinate system are extracted respectively. The vertex coordinates define the precise position of each corner point of the graphic in the virtual space, thus obtaining the master vertex coordinates and the cooperative vertex coordinates. A coordinate mapping operation is performed to map the master vertex coordinates back to the local pixel grid of the master display device according to the native resolution and pixel density of the master display device, generating master local graphic data. At the same time, the cooperative vertex coordinates are mapped back to the local pixel grid of the cooperative display device according to the native resolution and pixel density of the cooperative display device, generating cooperative local graphic data. The above mapping process includes complex scaling and translation algorithms, which ensures that the same window is displayed as a fine image on a high-resolution screen and automatically adapts to a clear image on a low-resolution screen. This eliminates the problem of blurry fonts or inconsistent interface sizes caused by the difference in pixel density between the two screens, allowing each screen to render its part of the screen in its best native state.

[0060] Obtain the display encoding and decoding capability matrix, serialize and encapsulate the collaborative local graph data according to the collaborative display device, and generate a collaborative rendering instruction set; combine the native display attributes of the main control display device, convert the main control local graph data, and generate a main control rendering instruction set;

[0061] The system acquires the previously exchanged display encoding / decoding capability matrix and, based on the supported capabilities declared by the cooperating display devices in this matrix, serializes and encapsulates the cooperating local graphics data. This involves converting the graphics data into a binary instruction stream that the cooperating devices can directly recognize and decode, thereby generating a cooperating rendering instruction set. The cooperating rendering instruction set contains only drawing commands and necessary texture indexes, with a very small data volume and no need for complex video encoding processes. Simultaneously, the system combines the native display attributes of the main control display device to convert the main control local graphics data into a low-level drawing command stream that can be directly executed by the local graphics driver, generating the main control rendering instruction set. The generation of these two instruction sets marks a fundamental shift in rendering logic, from traditional full-screen video streaming to distributing drawing commands. The main control display device and the cooperating display devices will each utilize their respective graphics processors to independently perform rasterization rendering locally based on the received instruction sets. This not only completely eliminates the latency and stuttering caused by network bandwidth limitations but also achieves perfect synchronization and seamless splicing of cross-screen images.

[0062] Based on the device identification data, the main control rendering instruction set is localized for rasterization rendering to obtain the main control image frame; at the same time, the collaborative display device is independently rasterized for rasterization rendering based on the device identification data to generate collaborative image frames; based on the main control image frame and the collaborative image frame, heterogeneous rendering compensation is performed to obtain the modified target display object frame, thus completing the display control of multi-screen collaboration on the portable device.

[0063] The specific steps for heterogeneous rendering compensation based on master image frames and cooperative image frames include:

[0064] The display optoelectronic transmission differences between the main control display device and the cooperating display devices are determined based on the display color and brightness benchmark; a global color and brightness mapping lookup table is constructed based on the display optoelectronic transmission differences.

[0065] Based on the display color brightness benchmark in the previously exchanged device identification data, the display photoelectric transmission difference between the main control display device and the cooperative display device is determined. The display photoelectric transmission difference refers to the physical deviation in brightness output efficiency, color reproduction accuracy and grayscale response characteristics when the two screens convert the same digital signal into light signals that are actually visible to the human eye. This deviation is due to the inherent characteristics of the light-emitting materials, backlight modules and driving circuits of different brands or models of screen panels.

[0066] Based on the quantized differences in display photoelectric transmission, a global chromaticity and luminance mapping lookup table is constructed. Essentially, this table is a high-precision three-dimensional data matrix. Its input dimension consists of the original red, green, and blue primary color signal values, while its output dimension is the corrected target signal value. The construction process involves mapping the measured gamma curves and color gamut coordinates of the cooperating devices to the standard color space of the main control device using a mathematical interpolation algorithm. The required compensation coefficients for each grayscale level and each color are calculated, and these coefficients are pre-calculated and stored in a table format for rapid retrieval. This provides accurate data support for subsequent real-time color correction, ensuring that any input signal can find its corresponding optimal correction output value.

[0067] Obtain the master rendering parameters of the master image frame; based on the master rendering parameters, use the global chroma-luminance mapping lookup table to perform pixel-level remapping on the rendering pipeline of the generated collaborative image frame to obtain the color temperature gamma response curve for unifying the collaborative image frame and the master image frame.

[0068] The system acquires the master rendering parameters of the currently generated master image frame in real time. These parameters include the average brightness level, dynamic contrast settings, color space mode, and specific post-processing enhancement effects, representing the visual style baseline that the user expects to see on the master screen. Using the master rendering parameters as the absolute baseline, the system performs pixel-level remapping of the rendering pipeline for the collaborative image frame to be output by the collaborative display device using the aforementioned global chroma-luminance mapping lookup table. Specifically, it iterates through each pixel in the collaborative image frame, reads its original color value, and uses it as an index. The system queries the global chromaticity and brightness mapping lookup table to obtain the corrected new color value and replaces the old value with the new value. Through real-time pixel-by-pixel replacement, the system generates a color temperature gamma response curve to unify the collaborative image frame and the master control image frame. The color temperature gamma response curve logically forces the color performance of the collaborative device to bend to a state completely consistent with that of the master control device, eliminating the phenomenon of cross-screen color difference, uneven brightness, or uneven color temperature caused by different screen hardware. This makes the same image displayed across two screens appear to the human eye as if it were presented on a complete and uniform screen.

[0069] Based on the virtual lines mapped from the physical seams of the equipment, edge coupling regions are defined at the adjacent edges of the master image frame and the cooperative image frame; rendering compensation is performed on the edge coupling regions to obtain the compensation edge mask;

[0070] The specific steps for rendering compensation of edge coupling regions include:

[0071] The physical pixel density ratio between the main display device and the cooperating display device is obtained based on the native display attributes; a compensation edge mask is generated in the edge coupling area based on the physical pixel density ratio; the brightness of the edge pixels of the main image frame and the cooperating image frame is compensated using the compensation edge mask to obtain the compensation image edge frame;

[0072] Based on the determined virtual lines mapped from the physical seams of the devices, a variable-width edge coupling region is defined at the adjacent edges where the master image frame and the cooperative image frame are about to be spliced. The edge coupling region is not a fixed pixel row on the physical screen, but a virtual buffer band that logically extends several pixels across the two physical black borders of the two screens. Its function is specifically to handle the visual discontinuity caused by the physical gaps between the screens and the differences in pixel arrangement.

[0073] Within the edge coupling region, the system accurately calculates and obtains the physical pixel density ratio between the main control display device and the collaborative display device based on the native display attributes in the identification data of both devices. This ratio reflects the difference in the number of pixels on the two screens per unit physical length. For example, if the main control screen has five pixels per millimeter while the collaborative screen has only three pixels per millimeter, the ratio is five to three. This data is the key basis for subsequent sub-pixel level alignment. Based on the calculated physical pixel density ratio, the system generates a compensation edge mask in the edge coupling region through an interpolation algorithm. The compensation edge mask is a grayscale weight matrix. Its generation steps include: first, logically stretching the edge pixels of the low-density screen or compressing the edge pixels of the high-density screen to the same physical scale according to the ratio; then calculating the pixel coverage weight in the overlapping area; setting the weight of the mask center area (near the physical black edge) as a transition value; and gradually transitioning to full weight towards the inside of the two screens, thereby forming a smooth brightness and color mixing template.

[0074] Based on the color temperature gamma response curve and the compensation image edge frame, the collaborative image frame and the master image frame are optimized to obtain the modified target display object frame. The generated compensation edge mask is used to perform brightness compensation calculations on the edge pixels of the master and collaborative image frames located within the edge coupling region. Specifically, the pixel color value at the corresponding position in the original image frame is multiplied channel-by-channel with the corresponding weight coefficient in the mask, and the calculation results of the two screens within the coupling region are weighted and superimposed to obtain the compensation image edge frame. These steps fill the visual gaps caused by the physical black borders and eliminate the discontinuity caused by hard edges through a smooth brightness transition. The system achieves a seamless, pixel-level connection between the two screens, making lines or gradient color blocks that were originally cut off by physical gaps appear visually continuous. Finally, the system deeply integrates and optimizes the color correction parameters determined by the previously generated color temperature gamma response curve with the edge frames of the currently obtained compensated image. This means that while ensuring overall color consistency, the system particularly enhances the smoothness of the edge areas, eliminating possible color breaks or brightness abrupt changes. The final output is the modified target display frame, which achieves pixel-level seamless connection at the dual-screen splicing point. Users cannot perceive the physical boundary between the two independent screens with the naked eye, thus completing high-fidelity multi-screen collaborative display control for portable devices.

[0075] When controlling the display of multi-screen collaboration on a portable device, it is necessary to perform content attribute analysis on the collaborative sub-primitive set; if the collaborative sub-primitive set contains dynamic video textures, then local compression control is performed on the collaborative sub-primitive set; if the collaborative sub-primitive set contains static vector elements, then lossless vector format control is performed on the collaborative sub-primitive set; a hybrid collaborative rendering instruction set is generated based on local compression control or lossless vector format control.

[0076] It should be noted that in completing the display control process for multi-screen collaboration on portable devices, the first step is to conduct in-depth content attribute analysis on the collaborative sub-graphics set to be presented on the collaborative display devices. This step aims to identify the essential characteristics of the graphic data in order to optimize the transmission strategy. The analysis process involves scanning the data structure and metadata tags of the graphics to determine whether it is a display dynamic video texture containing a continuously changing frame sequence or a geometric shape defined by mathematical formulas, i.e., a static vector element. Display dynamic video textures refer to video image data with time-dimensional changes, huge data volume, and a large amount of inter-frame redundancy, while static vector elements refer to graphic objects such as icons, text, or interface borders that are composed of coordinate points and path curves, do not lose quality when enlarged, and have a small data volume.

[0077] If the analysis confirms that the collaborative sub-primitive set contains dynamic video textures, a local compression control mechanism is immediately activated. This mechanism does not use full-frame compression but instead targets only the specific region containing the video texture. It uses an inter-frame prediction algorithm to extract motion vectors and remove spatiotemporal redundancy, converting the image data into a highly compressed binary stream to significantly reduce bandwidth usage. Conversely, if the collaborative sub-primitive set is found to contain static vector elements, lossless vector format control is executed, preserving all coordinate precision, color definitions, and path curvature parameters without any lossy quantization, ensuring absolute sharpness of graphic edges. Finally, based on the above judgment results, the locally compressed video data stream or the vector description data that maintains its original precision is encapsulated together to generate a hybrid collaborative rendering instruction set that can efficiently process both dynamic images and static interfaces, achieving the best balance between data transmission efficiency and display quality.

[0078] The same global display timestamp based on the heterogeneous rendering context is attached to the main rendering instruction set and the collaborative rendering instruction set. After receiving the collaborative rendering instruction set or the hybrid collaborative rendering instruction set, the collaborative display device performs a rendering instruction suspension operation until the local clock is aligned with the global display timestamp, and then triggers independent rasterization rendering.

[0079] To ensure perfect frame-level synchronization between the main display device and the collaborative display device, avoiding screen tearing or audio-visual asynchrony, the system attaches a global display timestamp based on a heterogeneous rendering context to both the main rendering instruction set and the aforementioned hybrid collaborative rendering instruction set. This global display timestamp is a high-precision time stamp derived from the system clock synchronized when the connection is established, representing the exact moment the frame should be displayed in the physical world, rather than the moment data is sent or received. When the collaborative display device receives the collaborative rendering instruction set or the hybrid collaborative rendering instruction set, it does not immediately execute drawing operations. Instead, it first performs a rendering instruction suspension operation, storing the received instructions in a queue and pausing parsing while continuously monitoring the local hardware clock. Only when the local clock reading is perfectly aligned with the global display timestamp carried in the instruction will the collaborative device instantly release the suspension, triggering an independent rasterization rendering process and converting the instruction into pixel signals for output. This timestamp-based waiting mechanism eliminates latency differences caused by network transmission jitter, ensuring that both screens refresh the corresponding content at the same microsecond, achieving true visual synchronization.

[0080] Meanwhile, the main control display device releases idle display resources according to the cutting benchmark, and the collaborative display device preloads display resources according to the collaborative rendering instruction set or the hybrid collaborative rendering instruction set;

[0081] To maximize system resource utilization and ensure smooth rendering, the main display device and the collaborative display device each execute resource management strategies. After successfully segmenting the graphics task and generating the distributed instruction set, the main display device immediately releases idle display resources, including video memory, graphics processor computing units, and memory bandwidth, that were originally used to store complete large images or process graphics in collaborative areas, based on the segmentation criteria. These released resources can be immediately reallocated to other applications on the main screen or used to increase the rendering frame rate of the main area, avoiding unnecessary resource occupation. On the collaborative display device side, during the suspension waiting period after receiving the instruction set but before reaching the global display timestamp, the collaborative display device uses this brief time window to preload display resources according to the collaborative rendering instruction set or the hybrid collaborative rendering instruction set.

[0082] The specific steps include pre-decompressing the compressed video texture data to the video memory, compiling the vector path data into a sequence of low-level instructions that the graphics processor can directly execute, and pre-allocating the frame buffer and shader resources required for rendering. This pre-loading mechanism ensures that all necessary data and computing resources are ready the moment the clock alignment triggers rendering, thereby achieving zero-latency real-time screen presentation and greatly improving the response speed and operational stability of the multi-screen collaborative system.

[0083] In practical implementation, all calculations and simulations are based on a unified virtual space mapping and distributed heterogeneous rendering method. Core algorithms include 3D affine transformation, dynamic geometric segmentation, adaptive color space conversion, and timestamp-based frame synchronization scheduling. Specifically, a logically infinite 2D or 3D virtual coordinate system is first constructed, treating all physical display devices as independent planar nodes within this virtual space. By collecting data on the physical dimensions, resolution, placement angle, and relative position of each device, a rotation and translation matrix is ​​constructed using a 3D affine transformation algorithm. This matrix maps the coordinates of each device's local coordinate system to the virtual coordinate system, thereby eliminating geometric distortions caused by physical placement. When different data units are involved... To address this, the system employs a physical pixel density normalization strategy. Instead of directly using the number of pixels as the distance unit, it introduces logical inches or standard physical millimeters as an intermediate, universal unit of measurement. Specifically, it reads the pixels per inch (PPI) parameter of each display device, divides the input pixel coordinates by the device's PPI value to obtain the physical length value, performs all distance calculations, gap compensation, and geometric cutting operations in the virtual coordinate system, and then multiplies the calculated physical length value by the target display device's PPI value to convert it back to the device's unique pixel coordinates. This ensures that the physical size and relative position of the graphics remain absolutely consistent when exchanging data between screens of different resolutions and sizes, avoiding image stretching or compression caused by differences in pixel density.

[0084] In the geometric cutting and content analysis phase, the system utilizes a dynamic geometric cutting algorithm to process graphic objects that cross screen boundaries. Specifically, a cutting baseline representing the physical black border center is defined in a virtual coordinate system. By calculating the spatial relationship between graphic primitives (such as triangle vertices) and this baseline, it determines whether the primitive is completely contained, completely excluded, or cut. For cut primitives, the algorithm uses linear interpolation to generate new vertices on the cutting line, splitting the original primitive into two new primitive sets belonging to the main control screen and the collaborative screen, ensuring the integrity of the geometric topology. Subsequently, during content attribute analysis, a feature recognition algorithm is employed. The metadata of the scan primitive set is analyzed. If texture coordinates change over time or are associated with a video decoder handle, it is identified as a dynamic video texture, triggering the local compression control module. This module uses block-based motion estimation and discrete cosine transform algorithms to encode only the video region with a high compression ratio. If primitives are identified as consisting of Bézier curves or line segments without texture dependency, they are identified as static vector elements, triggering the lossless vector format control module to preserve the original mathematical description. This hybrid processing method generates an optimized hybrid collaborative rendering instruction set, ensuring both smooth transmission of dynamic images and clarity of static interfaces.

[0085] In the color compensation and edge blending stage, the system executes an adaptive color space conversion algorithm and a weighted fusion algorithm. Specifically, it is based on a pre-built global chromaticity and brightness mapping lookup table. This table maps the nonlinear photoelectric transmission curve of the source device to the standard curve of the target device through a polynomial fitting algorithm. The implementation steps are to take the RGB value of each pixel as input, perform three-dimensional interpolation operation in the lookup table, and output the corrected RGB value, thereby unifying the color temperature and gamma response.

[0086] For edge coupling areas, the system uses a Gaussian weighted fusion algorithm to generate a compensation edge mask. The specific steps are as follows: based on the ratio of physical pixel density of the two screens, the normalized distance of each pixel from the physical seam in the overlapping area is calculated, and the Gaussian function is substituted to generate a weight coefficient between 0 and 1. Then, the edge pixels of the main control screen and the edge pixels of the collaborative screen are linearly mixed according to this weight. That is, the new pixel value is equal to the main control pixel value multiplied by its weight plus the collaborative pixel value multiplied by its complementary weight. In this way, the brightness discontinuity and color inconsistency caused by the physical black edge are visually smoothed out, and a seamless splicing effect is achieved.

[0087] In the final rendering and synchronization scheduling stage, the system adopts a deterministic scheduling algorithm based on a global clock source to coordinate the work of multiple devices. Specifically, each frame rendering task is assigned a unique global display timestamp, which is generated based on the system clock synchronized by a high-precision network time protocol. After receiving the rendering instruction set, the master device and the cooperating device do not immediately perform rasterization, but enter a suspended waiting state and continuously compare the local hardware clock with the global timestamp. Only when the local clock reaches or exceeds the time specified by the global timestamp will an interrupt be triggered and the rasterization pipeline of the graphics processor be started.

[0088] During this process, if the data unit involves time, the system uniformly uses microseconds as the smallest unit of measurement to eliminate the difference in the scheduling granularity of the operating system. At the same time, the resource management module dynamically adjusts the video memory allocation based on the cutting results. The main control screen releases the video memory blocks of the tasks that have been handed over, and the collaborative screen uses the waiting time difference to preload the texture and shader resources in the instruction set to the video memory. This ensures that at the moment the timestamps are aligned, both sides can complete independent rasterization rendering at full load, and finally present a complete picture in the physical world that is time-synchronized, color-consistent, and geometrically continuous.

[0089] Example 2: A portable multi-screen collaborative display control system supporting splicing screens, see [link to example]. Figure 1 As shown, it includes:

[0090] The multi-screen collaboration analysis module includes a display analysis unit and a display mapping unit. The display analysis unit sets the current portable device as the main control display device. When a collaborative display device is detected entering the preset near-field control range of the portable device, the device identification data of the main control display device and the collaborative display device are exchanged in real time. The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. The display mapping unit is used to virtually map the main control display device and the collaborative display device to obtain a virtual coordinate system for the screen display. When the coordinate vector of the target display object is captured to be within the preset screen boundary of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output.

[0091] The multi-screen collaboration control module includes a display control unit. The display control unit performs local rasterization rendering of the main control rendering instruction set based on device identification data to obtain the main control image frame. At the same time, the collaborative display device performs independent rasterization rendering based on the device identification data to generate collaborative image frames. Heterogeneous rendering compensation is performed based on the main control image frame and the collaborative image frame to obtain the modified target display object frame, thus completing the display control of the multi-screen collaboration of the portable device.

[0092] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A method for controlling multi-screen collaborative display on a portable computer that supports spliced ​​screens, characterized in that, Includes the following steps: The current portable device is set as the main control display device; when a cooperating display device is detected to enter the preset near-field control range of the portable device, the device identification data of the main control display device and the cooperating display device are exchanged in real time; the device identification data includes native display attributes, display color and brightness reference, display encoding and decoding capability matrix and display space topology seed; Virtual mapping is performed between the main control display device and the collaborative display device to obtain a virtual coordinate system for the screen display. When the coordinate vector of the target display object is captured to be within the preset boundary of the screen of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output. Based on the device identification data, the main control rendering instruction set is localized for rasterization rendering to obtain the main control image frame; at the same time, the collaborative display device is independently rasterized for rasterization rendering based on the device identification data to generate collaborative image frames; based on the main control image frame and the collaborative image frame, heterogeneous rendering compensation is performed to obtain the modified target display object frame, thus completing the display control of multi-screen collaboration on the portable device. The specific steps for outputting the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device include: Construct a virtual coordinate system for the display; obtain the initial physical edge distance and relative angle between the main display device and the cooperating display device based on the display space topology seed; construct a 3D rotation matrix based on the relative angle, rotate the local planar coordinate system of the cooperating display device to an aligned state with the main display device, and determine the normal vector direction of the cooperating display device in the virtual coordinate system; calculate the visual compensation blind zone width based on the initial physical edge distance; translate the starting coordinates of the cooperating display device in the virtual coordinate system by the visual compensation blind zone width along a direction away from the main display device; obtain the corrected virtual coordinate system for the display based on the normal vector direction and the visual compensation blind zone width. Based on the corrected virtual coordinate system of the screen display, obtain the virtual line mapping of the physical seam between the main control display device and the collaborative display device; using the virtual line mapping of the physical seam as the cutting reference, cut the target display object into a set of main control sub-elements located on the main control display device and a set of collaborative sub-elements located on the collaborative display device; Extract the vertex coordinates of the master control sub-element set and the cooperative sub-element set in the on-screen virtual coordinate system to obtain the master control vertex coordinates and the cooperative vertex coordinates; map the master control vertex coordinates and the cooperative vertex coordinates to the on-screen virtual coordinate system to obtain the master control local graph data and the cooperative local graph data; Obtain the display encoding and decoding capability matrix, serialize and encapsulate the collaborative local graph data according to the collaborative display device, and generate a collaborative rendering instruction set; combine the native display attributes of the main control display device, convert the main control local graph data, and generate a main control rendering instruction set; The specific steps for heterogeneous rendering compensation based on master image frames and cooperative image frames include: The display optoelectronic transmission differences between the main control display device and the cooperating display devices are determined based on the display color and brightness benchmark; a global color and brightness mapping lookup table is constructed based on the display optoelectronic transmission differences. Obtain the master rendering parameters of the master image frame; based on the master rendering parameters, use the global chroma-luminance mapping lookup table to perform pixel-level remapping on the rendering pipeline of the generated collaborative image frame to obtain the color temperature gamma response curve for unifying the collaborative image frame and the master image frame. Based on the virtual lines mapped from the physical seams of the devices, edge coupling regions are defined at the adjacent edges of the master image frame and the cooperative image frame; rendering compensation is performed on the edge coupling regions to obtain the compensated edge mask.

2. The portable device multi-screen cooperative display control method supporting screen splicing according to claim 1, characterized in that, When a collaborative display device is detected entering the preset near-field control range of the portable device, the specific steps for real-time exchange of device identification data between the main control display device and the collaborative display device include: When the main control display device enters the splicing screen connection stage, the main control display device continuously sends out external detection frequency band beacons; when the cooperative display device is detected to enter the preset portable near-field control range, the cooperative display device establishes a direct hardware connection channel for the screen through the external detection frequency band beacon; in the direct hardware connection channel for the screen, compressed and structured device identification data are exchanged. The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. When the main control display device and the cooperating display device receive the device identification data, a heterogeneous rendering context environment is generated.

3. The portable device multi-screen cooperative display control method of claim 2, wherein, The specific steps for rendering compensation of edge coupling regions include: The physical pixel density ratio between the main display device and the cooperating display device is obtained based on the native display attributes; a compensation edge mask is generated in the edge coupling area based on the physical pixel density ratio; the brightness of the edge pixels of the main image frame and the cooperating image frame is compensated using the compensation edge mask to obtain the compensation image edge frame; Based on the color temperature gamma response curve and the compensation image edge frame, the collaborative image frame and the master image frame are optimized to obtain the modified target display object frame.

4. The portable device multi-screen cooperative display control method of claim 3, wherein, When controlling the display of multi-screen collaboration on a portable device, it is necessary to perform content attribute analysis on the collaborative sub-primitive set; if the collaborative sub-primitive set contains dynamic video textures, then local compression control is performed on the collaborative sub-primitive set; if the collaborative sub-primitive set contains static vector elements, then lossless vector format control is performed on the collaborative sub-primitive set; a hybrid collaborative rendering instruction set is generated based on local compression control or lossless vector format control. The same global display timestamp based on the heterogeneous rendering context is attached to the main rendering instruction set and the collaborative rendering instruction set. After receiving the collaborative rendering instruction set or the hybrid collaborative rendering instruction set, the collaborative display device performs a rendering instruction suspension operation until the local clock is aligned with the global display timestamp, and then triggers independent rasterization rendering. Meanwhile, the main control display device releases idle display resources according to the cutting benchmark, and the collaborative display device preloads display resources according to the collaborative rendering instruction set or the hybrid collaborative rendering instruction set.

5. A portable multi-screen cooperative display control system supporting a spliced screen, characterized in that, A method for controlling multi-screen collaborative display on a portable device that supports splicing screens, as described in any one of claims 1-4, includes: The multi-screen collaboration analysis module includes a display analysis unit and a display mapping unit. The display analysis unit sets the current portable device as the main control display device. When a collaborative display device is detected entering the preset near-field control range of the portable device, the device identification data of the main control display device and the collaborative display device are exchanged in real time. The device identification data includes native display attributes, display color and brightness references, display encoding and decoding capability matrix, and display space topology seed. The display mapping unit is used to virtually map the main control display device and the collaborative display device to obtain a virtual coordinate system for the screen display. When the coordinate vector of the target display object is captured to be within the preset screen boundary of the main control display device, the original display rendering logic is stopped, and the main control rendering instruction set located on the main control display device and the collaborative rendering instruction set located on the collaborative display device are output. The multi-screen collaboration control module includes a display control unit. The display control unit performs local rasterization rendering of the main control rendering instruction set based on device identification data to obtain the main control image frame. At the same time, the collaborative display device performs independent rasterization rendering based on the device identification data to generate collaborative image frames. Heterogeneous rendering compensation is performed based on the main control image frame and the collaborative image frame to obtain the modified target display object frame, thus completing the display control of the multi-screen collaboration of the portable device.

Images