A soft bus-based multi-screen cooperative fragment rendering method under link quality degradation

Abstract

The application provides a link quality degradation based soft bus multi-screen cooperative slice rendering method, and relates to the technical field of image transmission. The method comprises the following steps: a sending end acquires a cooperative picture and divides the cooperative picture into multiple logical slices with priorities, real-time detects a distributed soft bus link state, and performs differentiated transmission scheduling according to the priorities under a weak network; a receiving end performs differentiated rendering according to the priorities and cooperatively displays, and when a slice exception is detected, only retransmits and splices and outputs the abnormal slice. The beneficial effect is that the method realizes fine allocation of bandwidth resources, avoids screen quality degradation leading to key area blurring, guarantees key area display quality through differentiated rendering, reduces bandwidth overhead and repair delay through local retransmission, avoids stuttering aggravation, and improves the stability and interactive experience of multi-screen cooperation under link quality degradation.

Description

Technical Field

[0001] This invention relates to the field of image transmission technology, and in particular to a multi-screen collaborative slice rendering method based on a soft bus under link quality degradation. Background Technology

[0002] Distributed soft bus technology is the core underlying technology for enabling multi-screen collaboration and cross-device content flow across heterogeneous devices, and it is currently widely used in multi-terminal collaboration scenarios. However, in special scenarios with degraded link quality (i.e., weak network, high electromagnetic interference, and large bandwidth fluctuations) (such as in the field, tunnels, and industrial workshops), existing multi-screen collaborative rendering and transmission solutions have significant technical shortcomings and cannot simultaneously meet the collaborative requirements of high reliability and low latency.

[0003] Analysis reveals the following main problems with existing technologies: Full-screen degradation leads to blurring of critical areas: Existing technologies, under conditions of degraded link quality, generally adopt a method of uniformly reducing the resolution, frame rate, or increasing the compression ratio of the entire screen to reduce bandwidth consumption. This method causes critical areas such as the user interface operation area, control buttons, and real-time data to become blurred synchronously, affecting operational accuracy and making it unsuitable for scenarios with high operational precision requirements.

[0004] Full-screen retransmission exacerbates stuttering: When data packet loss or screen distortion occurs, existing technologies typically request a full-screen retransmission. This significantly increases bandwidth consumption and, in environments with degraded link quality, can easily lead to a combination of stuttering and latency, and in severe cases, even screen freezing, failing to meet the needs of real-time collaboration.

[0005] Undifferentiated transmission strategy and unreasonable resource allocation: Existing technologies apply the same transmission priority and rendering strategy to all areas of the screen, failing to distinguish between critical and non-critical areas. This results in valuable bandwidth resources being occupied by non-critical areas, while the transmission quality of critical areas cannot be guaranteed, leading to a poor overall collaborative experience.

[0006] In summary, how to provide a multi-screen rendering method that can ensure the clarity and smoothness of critical areas while avoiding the overhead of full-screen retransmission and achieving reasonable allocation of bandwidth resources in a link quality degradation environment is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a multi-screen collaborative fragmented rendering method based on a soft bus under link quality degradation, comprising the following steps: Step S1: The sending end obtains the collaborative screen to be shared, divides the collaborative screen into multiple logical segments according to the screen content, and determines the priority of each logical segment. Step S2: The transmitting end detects the link status of the distributed soft bus between itself and the receiving end in real time; Step S3: When the link status is degraded, the transmitting end transmits each logical segment to the receiving end through the distributed soft bus according to the priority of each logical segment, and performs differentiated transmission scheduling. Step S4: The receiving end executes differentiated rendering strategies on each of the received logical segments according to the priority and performs collaborative display. Step S5: The receiving end detects the transmission or rendering status of each logical segment. When any logical segment is detected to be abnormal, it triggers a local retransmission of the abnormal logical segment and splices the repaired logical segment with other logical segments for output.

[0008] Preferably, the method further includes the following steps before performing step S1: Step S0: The transmitting end and the receiving end complete the networking based on the distributed soft bus. After networking, the segmentation and hierarchical benchmark rules and transmission and rendering benchmark parameters are preset. Steps S1, S3, and S4 respectively execute the corresponding partitioning, transmission scheduling, and rendering strategies based on the fragmentation and hierarchical benchmark rules and the transmission and rendering benchmark parameters.

[0009] Preferably, step S1 includes: Step S11: Divide the collaborative screen into high-priority segments, medium-priority segments, and low-priority segments according to the screen content; The high-priority segments correspond to the core interaction and data areas, the medium-priority segments correspond to the regular graphic and text areas, and the low-priority segments correspond to the background areas.

[0010] Preferably, step S1 further includes: In step S12, in response to changes in the screen content or movement of the interactive focus, the sending end dynamically adjusts the coverage of each logical segment in real time and updates its priority synchronously.

[0011] Preferably, step S2 includes: Step S21: Collect the communication link parameters of the distributed soft bus at a preset period to serve as the basis for evaluating the link status. The communication link parameters include at least one of packet loss rate, transmission delay, and available bandwidth. Step S22: Compare the collected communication link parameters with a preset link quality degradation judgment threshold. When the communication link parameters reach the link quality degradation judgment threshold, the current link state is determined to be a link quality degradation state; When the communication link parameters do not reach the link quality degradation judgment threshold, the current link status is determined to be a normal network status.

[0012] Preferably, step S3 includes: For each logical fragment with different priorities, the corresponding transmission scheduling strategy is executed in parallel: For the high-priority fragments, low-latency coding and redundancy error correction mechanisms are used, and soft bus bandwidth is allocated preferentially for transmission. For the medium-priority fragmentation, the transmission rate is dynamically adjusted according to the link status. For the low-priority fragments, high compression ratio encoding is used, and transmission is delayed when bandwidth is insufficient.

[0013] Preferably, step S4 includes: Step S41: Determine whether the computing power of the receiving device meets the preset high computing power condition. If so, proceed to step S42; If not, proceed to step S43; Step S42, the receiving end renders all the logical segments locally, wherein the high-priority segments are rendered without loss, and the medium-priority segments and the low-priority segments are rendered with local degraded quality. In step S43, the receiving end offloads the rendering task of the low-priority segment to the edge node for completion. The receiving end renders only the high-priority segment and the medium-priority segment locally, and integrates the rendering results of the low-priority segment transmitted by the edge node for screen output.

[0014] Preferably, step S5 includes: When any logical segment is detected to have continuous packet loss or screen distortion, a partial retransmission repair process is executed: the receiving end triggers a partial retransmission request to the sending end, only requesting the retransmission of the logical segment that has an abnormality, and uses a seamless splicing algorithm to integrate the retransmitted and repaired logical segment into the current screen for updating and display. It also includes: when the receiving end determines that the current link status is a link quality degradation state or that any logical fragment has lost packets, it starts a fragment caching mechanism to temporarily cache the high-priority fragments.

[0015] Preferred options also include: Step S6: When the link state recovers from the link quality degradation state to the normal network state, the transmission parameters and rendering parameters of each logical segment are automatically adjusted to the corresponding normal network baseline configuration.

[0016] Preferably, the method runs at the application layer of the distributed soft bus; and the method further includes: In step S7, the sending end and / or the receiving end respond to the collaboration termination command, clear the temporary data during the multi-screen collaboration process and disconnect the soft bus networking connection between the sending end and the receiving end.

[0017] The above technical solution has the following advantages or beneficial effects: It divides the shared screen into multiple logical segments with different priorities based on the screen content, while simultaneously detecting the link status of the distributed soft bus in real time. Based on this, it executes differentiated transmission scheduling according to segment priority and current link status, ensuring that high-priority segments (such as core interaction and data areas) receive priority and reliable transmission guarantees even under degraded link quality conditions, while non-critical areas reduce transmission resource consumption as needed. This achieves refined and rational allocation of bandwidth resources, avoiding the blurring problem in critical areas caused by uniform degradation of the entire screen. Furthermore, the receiving end executes differentiated rendering strategies based on segment priority to ensure the display quality of critical areas. When any logical segment is detected to have a transmission or rendering anomaly, only the abnormal segment is retransmitted locally and spliced ​​with other normal segments for output. This completely abandons the traditional scheme of full-screen retransmission, significantly reducing bandwidth overhead and repair latency under degraded link quality conditions, and effectively preventing increased stuttering. Attached Figure Description

[0018] Figure 1 In a preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is provided under conditions of degraded link quality. Figure 2 This is a schematic diagram of a sub-process of step S1 in a preferred embodiment of the present invention. Figure 3 This is a schematic diagram of a sub-process of step S2 in a preferred embodiment of the present invention; Figure 4 This is a schematic diagram of the sub-process of step S4 in a preferred embodiment of the present invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0020] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a multi-screen collaborative fragmented rendering method based on a soft bus is provided under conditions of degraded link quality, such as... Figure 1 As shown, it includes the following steps: Step S1: The sending end obtains the collaborative screen to be shared, divides the collaborative screen into multiple logical segments according to the screen content, and determines the priority of each logical segment. Step S2: The transmitting end detects the link status of the distributed soft bus between itself and the receiving end in real time; Step S3: When the link state is in a link quality deterioration state, the transmitting end transmits each logical segment to the receiving end through a distributed soft bus according to the priority of each logical segment, and performs differentiated transmission scheduling. Step S4: The receiving end executes differentiated rendering strategies on each received logical segment according to priority and performs collaborative display. Step S5: The receiving end detects the transmission or rendering status of each logical segment. When any logical segment is detected to be abnormal, it triggers a local retransmission of the abnormal logical segment and splices the repaired logical segment with other logical segments for output.

[0021] Specifically, in this embodiment, the technical architecture of the present invention revolves around the optimized design of multi-screen collaborative rendering in scenarios with degraded link quality and high interference. The core functionality is achieved through layering to realize image segmentation and hierarchical classification, differentiated transmission, adaptive rendering, and local repair functions, abandoning the existing approach of unified processing of the entire screen. The sending and receiving ends utilize various heterogeneous devices, either system-mounted or non-system-mounted, such as command terminals (i.e., sending ends), vehicle-mounted terminals, industrial control screens, and portable terminals (i.e., receiving ends). No deep system modification is required; only lightweight adaptation components need to be installed. This supports image segmentation recognition and local rendering, and can adapt to the needs of heterogeneous devices with different computing power.

[0022] Specifically, in this embodiment, the approach of unified processing of the entire screen in the existing technology is abandoned, and an adaptive segmentation processing mechanism based on content and link status is introduced.

[0023] To implement the multi-screen collaborative segmented rendering method of the present invention, multiple functional modules are set in the sending end and the receiving end to execute the various steps of the method. Among them, step S1 is executed by the screen segmentation and hierarchical function module of the sending end. This module includes a screen content recognition unit and a priority marking unit, which is responsible for acquiring collaborative screens, recognizing screen content, dividing logical segments and marking priorities, realizing screen space decoupling, and providing accurate boundaries for subsequent differentiated transmission and rendering; Step S2 is executed by the link quality detection module at the sending end. This module is embedded in the soft bus application layer and does not require modification of the underlying soft bus kernel. It obtains link parameters by periodically sending detection messages, perceives the current network status in real time, and provides a basis for decision-making for adjusting the transmission scheduling strategy. Step S3 is executed by the soft bus fragmentation scheduling module of the sending end. This module maintains a multi-level transmission queue and allocates corresponding transmission resources and encoding strategies to fragments of different priorities according to fragmentation priority and link status, so as to tilt bandwidth resources toward the core interaction area. Step S4 is executed by the adaptive rendering module of the receiving end. This module configures differentiated rendering parameters for different slices based on the slice priority and the local device computing power to ensure the rendering quality of high-priority slices while taking into account the device load. Step S5 is executed by the local repair and splicing module of the receiving end. This module includes a fragment verification unit, a retransmission request unit and a picture splicing unit. It is responsible for detecting fragment transmission and rendering anomalies. When an anomaly occurs, only the local retransmission of the abnormal fragment is triggered, and the seamless splicing repair of the picture is completed, reducing the fault tolerance granularity from the whole frame level to the fragment level.

[0024] Step S1 intelligently identifies the content of the screen and divides the entire collaborative screen into independent logical segments in geometric space, thereby enabling fine-grained management of the visual features and interactive attributes of different areas in the screen, breaking the spatial coupling of traditional video streams.

[0025] Step S2 utilizes the underlying capabilities of the distributed soft bus to provide highly elastic, full-scenario communication link status awareness between the sending and receiving ends, and to obtain key parameters such as packet loss rate, transmission latency, and available bandwidth in real time.

[0026] Step S3 establishes a two-dimensional scheduling mapping between content priority and network status, breaking the traditional first-in-first-out or blindly rate-limiting transmission mode, and ensuring that limited bandwidth resources are preferentially allocated to critical data.

[0027] Step S4 implements differentiated rendering with on-demand allocation of computing power on the receiving end side, maximizing the decoding and graphics rendering performance of the receiving end hardware device. Step S5 uses a local granular error correction scheme based on logical fragmentation to reduce the fault tolerance boundary of network anomalies from the frame level to the fragment level, avoiding the huge overhead caused by full-screen retransmission.

[0028] This embodiment realizes end-to-end adaptive closed-loop control from screen spatial division, network awareness, differential transmission, terminal rendering to local fault tolerance. It solves the defects of the multi-screen collaborative system in harsh wireless network environments, where the overall screen is prone to pauses and global screen distortion caused by local data loss. It ensures the basic smoothness of cross-screen collaboration, guarantees the clarity and smoothness of key interactive areas, and effectively controls the overall bandwidth consumption and transmission delay.

[0029] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is included under conditions of degraded link quality, wherein, before performing step S1, the method further includes: In step S0, the sending end and the receiving end complete the networking based on the distributed soft bus. After networking, the fragmentation and hierarchical benchmark rules and transmission and rendering benchmark parameters are preset. Steps S1, S3 and S4 respectively execute the corresponding partitioning, transmission scheduling and rendering strategies according to the fragmentation and hierarchical benchmark rules and transmission and rendering benchmark parameters.

[0030] Specifically, in this embodiment, during the preliminary preparation and initial configuration phase, the devices are first networked. The high-performance HarmonyOS command terminal is used as the transmitter, two low-performance non-HarmonyOS industrial control screens and edge node devices are used as receivers in an outdoor environment with degraded link quality and high electromagnetic interference. After all devices are started, self-discovery and trusted networking are completed based on the distributed soft bus, and a stable soft bus communication link is established. The networking process does not require manual intervention and adapts to the network communication differences of heterogeneous devices.

[0031] Then, each functional module is started and initialized. Finally, parameters are preset, including the fragmentation hierarchy benchmark rules and transmission and rendering benchmark parameters, as follows: high-priority fragmentation transmission latency threshold ≤ 5ms, using low-latency encoding; medium-priority fragmentation bitrate can be dynamically adjusted according to link quality; low-priority fragmentation uses high compression ratio encoding, with the frame rate preset to a reasonable range; the link quality degradation judgment threshold is when the packet loss rate reaches a preset high value or the available bandwidth is lower than a preset low value, which is judged as a weak network high interference state, i.e., a link quality degradation state. The distributed soft bus abstracts the spatially separated sender and receiver into a unified virtual peripheral resource pool through discovery, connection, and networking mechanisms. Subsequent steps S1, S3, and S4 use this initialization configuration as the reference benchmark for the global context, thereby eliminating ambiguity in fragmentation protocol parsing between the two ends.

[0032] This embodiment solves the problems of asynchronous states at both ends and lack of comparison benchmarks for parameters in the early stage of distributed collaboration by presetting static rules and dynamic parameter benchmarks in the timing pre-sequence stage, thereby improving the stability of system initialization and chain establishment, and ensuring the rapid start-up and stable operation of the system.

[0033] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is provided under conditions of degraded link quality, wherein step S1 specifically includes the following sub-steps: Step S11: Divide the collaborative screen into high-priority segments, medium-priority segments, and low-priority segments according to the screen content; among them, the high-priority segments correspond to the core interaction and data areas, the medium-priority segments correspond to the regular graphic and text areas, and the low-priority segments correspond to the background areas.

[0034] Specifically, in this embodiment, during the intelligent image segmentation and hierarchical implementation phase, image acquisition is performed first. The high-computing-power HarmonyOS command terminal acts as the transmitting end, acquiring the image content of multi-screen collaboration in real time, including the situation data area, operation button area, and background area. The image resolution and frame rate meet the usage requirements of the scenario.

[0035] Sub-step S11 performs semantic and structured analysis on the collaborative screen to achieve a three-level pyramid-style semantic segmentation. The sending end uses an image semantic segmentation algorithm to analyze the shared screen content in real time. Core interaction and data areas are typically geometric areas that are extremely sensitive to interaction latency and clarity, such as mouse cursor trails, text input box focus, playing video windows, or frequently refreshed chart controls. These are classified as high-priority segments with a priority label of P1. Regular text and image areas are static web pages, document viewing areas, or menu bars, which have a certain information load but a low refresh rate. These are classified as medium-priority segments with a priority label of P2. Background areas are static areas that are purely visual aids, such as desktop wallpapers and application borders. These are classified as low-priority segments with a priority label of P3.

[0036] This embodiment maps logical segments to specific business scenario areas, solving the problem of not being able to effectively trim non-critical information when bandwidth is insufficient. It provides precise logical entity boundaries for subsequent on-demand transmission scheduling and computing power allocation, enabling the transmission and rendering modules to clearly identify areas that need to be prioritized.

[0037] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is included under conditions of degraded link quality, wherein step S1 further includes the following sub-steps: In step S12, in response to changes in screen content or movement of interactive focus, the sending end dynamically adjusts the coverage of each logical segment in real time and updates its priority synchronously.

[0038] Specifically, in this embodiment, sub-step S12 introduces dynamic perception capabilities in the time dimension. Since multi-screen collaborative interaction is a time-varying process, the coordinates and size of the core interaction area on the screen frequently change with window dragging, click switching, and multitasking. The sending end, by hooking into the operating system's underlying event callbacks or performing differential pixel detection, recalculates the geometric coordinates of the bounding box in real time once it detects a focus shift or a drastic local screen refresh. This causes the geometric coverage of high, medium, and low priority segments and their corresponding priority labels to shift accordingly. For example, when a user clicks on a new window that was previously in a non-priority area, the area where the new window is located will dynamically be included in the coverage of a high-priority segment, and the priority of the original high-priority area will be lowered accordingly after losing focus.

[0039] This embodiment avoids transmission misalignment caused by the inability of static fragmentation strategies to adapt to dynamic changes in the screen, solves the problem of inaccurate protection when core business areas drift, and ensures that bandwidth and computing resources are always anchored on the user's current visual and interactive focus, greatly improving the real-time performance and flexibility of the interaction.

[0040] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is included under conditions of degraded link quality, wherein, as... Figure 3 As shown, step S2 specifically includes the following sub-steps: Step S21: Collect communication link parameters of the distributed soft bus at a preset period to serve as the basis for evaluating the link status. The communication link parameters include at least one of packet loss rate, transmission delay, and available bandwidth. Step S22: Compare the collected communication link parameters with the preset link quality degradation judgment threshold. When the communication link parameters reach the link quality degradation judgment threshold, the current link status is determined to be a link quality degradation state; When the communication link parameters do not reach the threshold for judging link quality degradation, the current link status is determined to be a normal network status.

[0041] Specifically, in this embodiment, sub-steps S21 and S22 construct a closed-loop link evaluation network model. The sending end embeds probes into the transmission control layer of the distributed soft bus, periodically and quantitatively sending probe messages to the channel to accurately obtain the current packet loss rate L, bidirectional transmission delay T, and estimated real-time available bandwidth B. Packet loss rate detection involves sending test data packets, counting the number of data packets returned by the receiving end, and calculating the packet loss rate using the formula: Packet loss rate = (Number of data packets sent - Number of data packets received) / Number of data packets sent × 100%. Transmission delay detection involves recording the time difference between the sending end and the receiving end receiving the test data packets, and taking the average of 10 probes as the current transmission delay.

[0042] To achieve accurate quantitative comparison, this embodiment introduces a comprehensive calculation formula for link quality evaluation: in, These represent preset weighting coefficients for packet loss rate, transmission delay, and the reciprocal of available bandwidth, respectively, and satisfy the following conditions: The preset threshold for judging link quality degradation is denoted as... In sub-step S22, the real-time calculated comprehensive link quality evaluation value Q is compared with the link quality degradation judgment threshold. Perform a logical comparison: when Q≥ When Q < 0, it means that the network has experienced at least one of the following anomalies: high packet loss, high latency, or extremely low bandwidth, and the current link state is determined to be a link quality degradation state; when Q < 0. When this happens, the current link status is determined to be a normal network status.

[0043] This embodiment uses a mathematical model that integrates multi-dimensional parameters for quantitative evaluation, which solves the technical defects of single-index judgment of network state, which is prone to misjudgment and violent oscillation, and provides deterministic triggering conditions for adaptive strategy switching.

[0044] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is included in a link quality degradation environment, wherein step S3 specifically includes the following sub-steps: For each logical segment with different priorities, the corresponding transmission scheduling strategy is executed in parallel: For high-priority fragmentation, low-latency coding and redundancy error correction mechanisms are used, and soft bus bandwidth is allocated first for transmission. For medium-priority fragmentation, the transmission rate is dynamically adjusted based on the link status. For low-priority fragments, high compression ratio encoding is used, and transmission is delayed when bandwidth is insufficient.

[0045] Specifically, in this embodiment, a multi-threaded parallel scheduling engine performs differentiated resource allocation on each logical segment. Based on the link status and segment priority, the three priority segments are allocated to different transmission queues to implement differentiated transmission strategies.

[0046] High-priority fragment P1 is assigned to the high-priority transmission queue, using ultra-low latency Intra frame encoding mode or configuring a very small quantization step size, and introducing redundant packets in conjunction with forward error correction coding. Even under conditions of limited soft bus bandwidth and severe packet loss and link quality degradation, it prioritizes ensuring that packets in the core area have absolute control over the highest quality of service queue of the soft bus by squeezing out other fragment channels.

[0047] Medium-priority fragments P2 are allocated to the medium-priority transmission queue, using conventional encoding. The bitrate is dynamically adjusted based on link quality to balance clarity and bandwidth usage, allowing for a small amount of packet loss.

[0048] Low-priority fragments P3 are allocated to low-priority transmission queues, using high compression ratio encoding, adjusting bit rate and frame rate as needed, allowing moderate packet loss, delaying transmission when bandwidth is insufficient, and prioritizing the release of bandwidth to high-priority fragments.

[0049] The link quality detection module continuously monitors the link status. When the link quality improves, the scheduling module automatically adjusts the transmission parameters, increases the bit rate and frame rate of medium and low priority fragments, and ensures that the transmission strategy matches the link status in real time.

[0050] This embodiment solves the problem of indiscriminate consumption of global bandwidth through parallel asymmetric transmission control. Even when the overall bandwidth drops drastically, it can still maintain zero-lag real-time feedback in the main operating area, achieving optimal resource allocation.

[0051] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is included under conditions of degraded link quality, wherein, as... Figure 4 As shown, step S4 specifically includes the following sub-steps: Step S41: Determine whether the computing power of the receiving device meets the preset high computing power condition. If so, proceed to step S42; If not, proceed to step S43; Step S42: The receiving end renders all logical segments locally, including lossless rendering of high-priority segments and local degraded rendering of medium-priority and low-priority segments. In step S43, the receiving end offloads the rendering task of the low-priority slice to the edge node for completion. The receiving end only renders the high-priority slice and the medium-priority slice locally, and integrates the rendering results of the low-priority slice transmitted from the edge node for screen output.

[0052] Specifically, in this embodiment, differentiated rendering is performed on different shards based on the link status and device computing power, while the rendering task is unloaded from low-computing-power devices.

[0053] Sub-step S41 implements a flexible offloading mechanism for computing power that takes into account both hardware and software contexts. The receiving end periodically monitors its own CPU load, GPU rendering pipeline utilization, and hardware decoder queuing latency. If hardware resources are sufficient, high computing power conditions are met, and full rendering is started on the local canvas. However, when allocating pixel-level texture overhead, full-fidelity, lossless texture mapping technology is used for high-priority slices, while bilinear interpolation or local degradation methods such as pixel format reduction are applied to medium and low-priority slices to reduce the burden on the GPU shader. Conversely, if the current load of the receiving end is too high or the hardware specifications are low, computing power routing is triggered. The low-priority background slice data stream with high data throughput but low technical requirements is routed to the edge computing node or sending end within the local area network for proxy rendering via the underlying soft bus. The receiving end focuses on parallel decoding and high refresh rate rendering of high and medium-priority slices locally, and finally uses an image compositor to perform alpha blending between the local and edge images.

[0054] In this embodiment, for the high-computing-power HarmonyOS command terminal (i.e., the transmitting end), all three priority segments are rendered locally. The high-priority segment P1 maintains lossless rendering to ensure clear operation buttons and real-time visibility of situational data. The rendering parameters of the medium and low-priority segments are adjusted as needed based on link quality to balance display effect and resource consumption. For the low-computing-power non-HarmonyOS industrial control screen (i.e., the receiving end), due to limited device computing power, it cannot handle the synchronous rendering task of the three segments. The rendering task of the low-priority segment P3 is offloaded to the edge node, and only the high-priority segment P1 and the medium-priority segment P2 are rendered locally. The high-priority segment P1 maintains the original resolution and 30fps frame rate for lossless rendering to ensure that operators can clearly see the operation buttons and situational data. The medium-priority segment P2 is rendered with moderately reduced quality to balance rendering load and display effect. After the low-priority segment P3 is rendered by the edge node, the rendering result is transmitted to the low-computing-power industrial control screen for image output only, without consuming the local computing power of the industrial control screen. The rendering modules of all devices maintain timing synchronization to ensure consistent display across multiple screens with no significant latency difference, thus guaranteeing a smooth multi-screen collaborative operation experience.

[0055] This embodiment solves the problem of frame dropping and overheating / stuttering caused by decoding and rendering overload in complex scenarios with degraded link quality at low-computing-power receivers. It achieves lightweight adaptive presentation of terminal devices, enabling low-computing-power devices to participate in high-quality multi-screen collaboration even under degraded link quality conditions.

[0056] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is provided under conditions of degraded link quality, wherein step S5 specifically includes the following sub-steps: When any logical segment is detected to have continuous packet loss or screen distortion, a partial retransmission repair process is executed: the receiving end triggers a partial retransmission request to the sending end, only requesting the retransmission of the logical segment that has an error, and uses a seamless splicing algorithm to integrate the retransmitted and repaired logical segment into the current screen for updating and display. When the current link status is determined to be a link quality degradation state or a packet loss occurs in any logical segment, the segment caching mechanism is started in parallel to temporarily cache high-priority segments.

[0057] Specifically, this embodiment refines the fault tolerance and protection chain. When the receiving end detects that any logical segment has experienced continuous packet loss exceeding a set threshold due to network congestion, or when the decoder throws a severe block artifact anomaly, it does not suspend the entire display thread or request full-screen keyframes as in traditional technologies. Instead, the receiving end generates a simplified retransmission instruction frame carrying only the specific segment ID. Upon receiving this frame, the sending end precisely reassembles the image data of that segment from its internal cache and retransmits it. After receiving the repaired segment data, the receiving end applies a seamless stitching algorithm based on the dirty rectangle theory, without affecting other normal segment rendering pipelines, to erase and overwrite pixels only in the local coordinate area where the segment is located on the canvas. Furthermore, to cope with sudden jitter caused by link quality degradation, the receiving end starts a temporary cache for high-priority segments, pre-caching data from the next few frames. When a brief packet loss occurs, the receiving end can play the high-priority images in the cache, thus avoiding stuttering or freezing in critical areas and ensuring the continuity of core operations.

[0058] This embodiment solves the persistent technical problems of frequent flickering, tearing, and screen distortion in multi-screen collaboration when the network is severely jittery by combining local precise noise reduction and retransmission with active high-quality caching. It ensures the spatial topology stability of the collaborative screen, and the user is almost unaware of the entire repair process.

[0059] In another preferred embodiment of the present invention, a multi-screen collaborative segmented rendering method based on a soft bus is provided under a link quality degradation environment. The method is characterized by further including a dynamic adjustment and adaptation step S6: when the link state recovers from the link quality degradation state to the normal network state, the transmission parameters and rendering parameters of each logical segment are automatically adjusted to the corresponding normal network baseline configuration.

[0060] Specifically, in this embodiment, step S6 provides a performance rebound and smooth recovery mechanism after network environment improvement. The link quality detection module continues to work. When step S22 determines that the comprehensive evaluation value Q of the link quality has fallen back below the preset threshold and maintained the set debouncing observation period, the system determines that the link status has officially escaped the shadow of link quality degradation and entered a normal network state. At this time, the sending end and receiving end work together to remove the previous compression, delayed transmission, and degraded rendering restrictions, and automatically and smoothly increase the encoding bitrate, frame rate, and rendering resolution of each segment to the normal network baseline configuration state preset in step S0. For example, the sending end increases the transmission bitrate and frame rate of the originally compressed medium and low priority segments and stops the delayed transmission strategy for low priority segments; the receiving end correspondingly increases the rendering parameters, restores the medium priority segments from local degraded rendering to normal rendering, and for low computing power devices that previously underwent task offloading, the rendering tasks of low priority segments are also taken back to local execution.

[0061] This embodiment realizes fully automatic bidirectional elastic switching of the strategy between poor and good networks, which solves the problem of system lag or inability to restore high-definition image quality when the network improves, and ensures the efficiency of the collaborative process and the matching of image quality.

[0062] In another preferred embodiment of the present invention, a multi-screen collaborative fragmented rendering method based on a soft bus is provided under a link quality degradation environment. The method is characterized in that it runs at the application layer of a distributed soft bus and further includes a collaborative termination step S7: the sending end and / or the receiving end respond to the collaborative termination instruction, clear the temporary data in the multi-screen collaborative operation process and disconnect the soft bus networking connection between the sending end and the receiving end.

[0063] Specifically, in this embodiment, step S7 defines the graceful termination control of the multi-screen collaboration lifecycle. The method is encapsulated as an independent service at the application layer within the distributed soft bus architecture. All adaptive slice rendering logic is implemented at the application layer of the distributed soft bus, without requiring modification of the underlying transmission protocol and kernel. This allows the invention to be compatible with various existing soft bus implementations, exhibiting good universality and portability. When a user clicks to disconnect the collaboration control on any terminal, or when the system triggers a collaboration termination command due to force majeure, the action-issuing entity immediately responds, actively invoking the underlying memory reclamation and resource unbinding handles. At this time, temporary data generated during application layer operation, such as slice ring buffers, dirty rectangular coordinate temporary matrices, edge unloading connection sockets, and multi-threaded concurrent queues, are completely cleared and released from memory. Subsequently, a disjointment notification is sent to the soft bus control layer, safely and without residue terminating the distributed network topology connection between the sending and receiving ends.

[0064] This embodiment addresses the drawbacks of memory leaks and zombie connections consuming system resources that may occur after long-term operation of multi-screen collaboration and frequent connection establishment and disconnection through a clear cleanup and ungrouping process, thus ensuring the stability and security of the distributed terminal system after long-term operation.

[0065] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A multi-screen collaborative fragmented rendering method based on a soft bus under link quality degradation, characterized in that, Includes the following steps: Step S1: The sending end obtains the collaborative screen to be shared, divides the collaborative screen into multiple logical segments according to the screen content, and determines the priority of each logical segment. Step S2: The transmitting end detects the link status of the distributed soft bus between itself and the receiving end in real time; Step S3: When the link status is degraded, the transmitting end transmits each logical segment to the receiving end through the distributed soft bus according to the priority of each logical segment, and performs differentiated transmission scheduling. Step S4: The receiving end executes differentiated rendering strategies on each of the received logical segments according to the priority and performs collaborative display. Step S5: The receiving end detects the transmission or rendering status of each logical segment. When any logical segment is detected to be abnormal, it triggers a local retransmission of the abnormal logical segment and splices the repaired logical segment with other logical segments for output.

2. The multi-screen collaborative segmented rendering method according to claim 1, characterized in that, Before performing step S1, the following is also included: Step S0: The transmitting end and the receiving end complete the networking based on the distributed soft bus. After networking, the segmentation and hierarchical benchmark rules and transmission and rendering benchmark parameters are preset. Steps S1, S3, and S4 respectively execute the corresponding partitioning, transmission scheduling, and rendering strategies based on the fragmentation and hierarchical benchmark rules and the transmission and rendering benchmark parameters.

3. The multi-screen collaborative segmented rendering method according to claim 1, characterized in that, Step S1 includes: Step S11: Divide the collaborative screen into high-priority segments, medium-priority segments, and low-priority segments according to the screen content; The high-priority segments correspond to the core interaction and data areas, the medium-priority segments correspond to the regular graphic and text areas, and the low-priority segments correspond to the background areas.

4. The multi-screen collaborative segmented rendering method according to claim 3, characterized in that, Step S1 further includes: In step S12, in response to changes in the screen content or movement of the interactive focus, the sending end dynamically adjusts the coverage of each logical segment in real time and updates its priority synchronously.

5. The multi-screen collaborative segmented rendering method according to claim 1, characterized in that, Step S2 includes: Step S21: Collect the communication link parameters of the distributed soft bus at a preset period to serve as the basis for evaluating the link status. The communication link parameters include at least one of packet loss rate, transmission delay, and available bandwidth. Step S22: Compare the collected communication link parameters with a preset link quality judgment threshold. When the communication link parameters reach the link quality judgment threshold, the current link status is determined to be a link quality deterioration state. When the communication link parameters do not reach the link quality judgment threshold, the current link status is determined to be a normal network status.

6. The multi-screen collaborative segmented rendering method according to claim 3, characterized in that, Step S3 includes: For each logical fragment with different priorities, the corresponding transmission scheduling strategy is executed in parallel: For the high-priority fragments, low-latency coding and redundancy error correction mechanisms are used, and soft bus bandwidth is allocated preferentially for transmission. For the medium-priority fragmentation, the transmission rate is dynamically adjusted according to the link status. For the low-priority fragments, high compression ratio encoding is used, and transmission is delayed when bandwidth is insufficient.

7. The multi-screen collaborative segmented rendering method according to claim 3, characterized in that, Step S4 includes: Step S41: Determine whether the computing power of the receiving device meets the preset high computing power condition. If so, proceed to step S42; If not, proceed to step S43; Step S42, the receiving end renders all the logical segments locally, wherein the high-priority segments are rendered without loss, and the medium-priority segments and the low-priority segments are rendered with local degraded quality. In step S43, the receiving end offloads the rendering task of the low-priority segment to the edge node for completion. The receiving end renders only the high-priority segment and the medium-priority segment locally, and integrates the rendering results of the low-priority segment transmitted by the edge node for screen output.

8. The multi-screen collaborative segmented rendering method according to claim 2, characterized in that, Step S5 includes: When any logical segment is detected to have continuous packet loss or screen distortion, a partial retransmission repair process is executed: the receiving end triggers a partial retransmission request to the sending end, only requesting the retransmission of the logical segment that has an abnormality, and uses a seamless splicing algorithm to integrate the retransmitted and repaired logical segment into the current screen for updating and display. It also includes: when the receiving end determines that the current link status is a link quality degradation state or that any logical fragment has lost packets, it starts a fragment caching mechanism to temporarily cache the high-priority fragments.

9. The multi-screen collaborative segmented rendering method according to claim 5, characterized in that, Also includes: Step S6: When the link state recovers from the link quality degradation state to the normal network state, the transmission parameters and rendering parameters of each logical segment are automatically adjusted to the corresponding normal network baseline configuration.

10. The multi-screen collaborative slice rendering method according to any one of claims 1 to 9, characterized in that, The method operates at the application layer of the distributed soft bus; and the method further includes: In step S7, the sending end and / or the receiving end respond to the collaboration termination command, clear the temporary data during the multi-screen collaboration process and disconnect the soft bus networking connection between the sending end and the receiving end.

Images