An edge computing-based button-triggered adaptive control method for wireless screen projection

Abstract

This invention discloses an edge computing-based button-triggered adaptive control method for wireless screen projection, relating to the field of wireless screen projection technology. Responding to a physical button trigger signal, the method reads the original extended display identifier data of the connected display device through a physical interface and parses the display device's maximum refresh rate parameter, optimal resolution parameter, and variable refresh rate support parameter. Based on the maximum refresh rate parameter and variable refresh rate support parameter, it generates reconstructed extended display identifier data, sets the timing descriptor containing the maximum refresh rate parameter as the preferred timing, and triggers a hot-plug detection signal at the wireless projection transmitter via a wireless link. This controls the signal source device to read the reconstructed extended display identifier data and output a video stream. The wireless projection transmitter parses the auxiliary video information frames and signal format characteristics in the video stream, identifies the content type priority of the video stream, and processes the video stream according to the content type priority and signal format characteristics.

Description

Technical Field

[0001] This invention relates to the field of wireless screen projection technology, and in particular to an adaptive control method for button-triggered wireless screen projection based on edge computing. Background Technology

[0002] In the field of wireless screen projection and display technology, with the popularization of mobile games and high-definition video applications, users' demand for projecting the screens of mobile terminals such as smartphones onto large-screen display devices is increasing. Existing wireless screen projection solutions mainly adopt a transmission architecture of signal source, transmitter, receiver, and display device, achieving screen mirroring and extended display through a wireless network.

[0003] Currently, Chinese patent application number 202511207638.7 discloses a wireless screen projection method and system integrating Bluetooth control and power management. The transmitting device adds a corresponding preset delay to the Bluetooth data packet response based on its own energy status, encoding the energy status into the Bluetooth communication timing. The receiving device decodes the energy status level of the transmitting end by measuring the difference between the round-trip response time of the Bluetooth data packet and the reference value. When the decoded energy status meets the preset conditions, the receiving device actively sends a control message to the transmitting end, which includes a screen projection strategy adjustment proposal to reduce the resolution, frame rate, or bit rate. After reaching a consensus, both parties jointly execute the strategy adjustment, thereby realizing the active perception and adaptive adjustment of power decay and avoiding sudden screen projection interruption due to power depletion. Existing technologies have the following shortcomings: They passively follow the preferred settings in the extended display identifier data of the display device. Even if the mobile terminal has sufficient power and powerful performance, it cannot break through the default limit of the extended display identifier data to actively enable 120Hz or 144Hz high refresh rate output. Although the image quality can be reduced according to the battery level to ensure battery life, there is no mechanism that allows users to physically trigger the device to forcibly activate the high refresh rate potential, resulting in the display capabilities of high-end hardware being idle. Existing technologies mostly adjust the encoding strategy based on physical layer parameters and fail to identify the content type. When processing high refresh rate signals, blindly introducing motion compensation technology can improve video smoothness, but it will produce huge input delays, making game operation feel sticky. If it is completely pass-through, it cannot eliminate video jitter. Existing technologies solve the transfer of energy state through Bluetooth timing encoding, but when transmitting variable refresh rate video streams, the inherent jitter of WiFi networks is not effectively suppressed. The lack of sub-frame level slicing processing and anti-jitter protection on the edge side makes it easy for VRR signals to cause the display to go black or flicker due to frame arrival timing deviations, which cannot meet the stability requirements of professional-grade e-sports screen projection. Summary of the Invention

[0004] The technical problem solved by this invention is that existing technologies are difficult to actively trigger the signal source device to output high refresh rate video streams in wireless screen projection scenarios, cannot adaptively switch between low latency in games and high smoothness in videos according to the content type, and are difficult to overcome the problem of abnormal display of high refresh rate or variable refresh rate signals caused by wireless network transmission jitter.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a button-triggered adaptive control method for wireless screen projection based on edge computing, comprising the following steps: Step 1: In response to the physical button trigger signal in the wireless projection system, the wireless projection receiver reads the original extended display identification data of the connected display device through the physical interface, and parses out the display device's maximum refresh rate parameters, optimal resolution parameters, and variable refresh rate support parameters. Step 2: The wireless projection receiver generates recombined extended display identifier data based on the maximum refresh rate parameter and the variable refresh rate support parameter, sets the timing descriptor containing the maximum refresh rate parameter as the preferred timing, and instructs the wireless projection transmitter to trigger a hot-plug detection signal through the wireless link, controlling the signal source device to read the recombined extended display identifier data and output the video stream. Step 3: The wireless projection transmitter parses the auxiliary video information frames and signal format features in the video stream, identifies the content type priority of the video stream, and packages the content type priority with the video stream encoding and transmits it to the wireless projection receiver. Step 4: The wireless screen mirroring receiver receives data packets and processes the video stream according to the content type priority and signal format characteristics. When the content type priority meets the first preset condition, the wireless screen mirroring receiver sends an automatic low latency mode command to the display device and transmits the video stream. When the content type priority meets the second preset condition and the video stream frame rate is less than the maximum refresh rate parameter, the wireless screen projection receiver starts the dynamic frame interpolation algorithm to perform frame rate compensation and then outputs the video stream. When the signal format characteristics indicate a variable refresh rate signal, the wireless screen projection receiver executes a dynamic synchronization processing strategy.

[0006] Preferably, the physical button trigger signal in step 1 is generated by a physical button on the wireless projection transmitter or the wireless projection receiver. When the physical button is located at the wireless projection transmitter, the wireless projection transmitter will encode the trigger status into a control command and send it to the wireless projection receiver via the wireless network. The wireless projection receiver will then parse the control command and read the original extended display identifier data of the connected display device. When the physical button is located at the wireless projection receiver, the wireless projection receiver responds and reads the original extended display identifier data of the connected display device.

[0007] Preferably, the specific process of generating the recombinant extended display identifier data in step 2 includes: The wireless screen projection receiver parses the Video Electronics Standards Association (VESA) data block and Consumer Electronics Association (CEA) data block in the original extended display identifier data and extracts a detailed timing descriptor list. The wireless projection receiver obtains the maximum transmission bandwidth capability of the wireless projection system, performs an intersection operation between the detailed timing descriptor list and the maximum transmission bandwidth capability, filters out candidate timing descriptors, and selects the target timing descriptor containing the highest refresh rate parameter from the candidate timing descriptors. The wireless screen projection receiver moves the target timing descriptor to the first address of the detailed timing descriptor list and encapsulates the reordered detailed timing descriptor list into recombined extended display identifier data.

[0008] Preferably, the specific method by which the wireless projection transmitter triggers the hot-plug detection signal in step 2 is as follows: Adjust the voltage value of the hot-plug detection pin on the connection interface between the wireless projection transmitter and the signal source device, reduce the voltage value to below a preset low-level judgment threshold, and after maintaining a preset reset time threshold, raise the voltage value to above a preset high-level judgment threshold.

[0009] Preferably, the content type priority in step 3 includes game mode priority and non-game mode priority; The wireless projection transmitter determines the current content type priority by parsing the logical state of the content type flag bits CN0 and CN1 in the auxiliary video information frame; When the logical state of CN0 and CN1 indicates that the content type is graphics or game, it is determined to be game mode priority; When the logical state of CN0 and CN1 indicates that the content type is a movie or a photo, it is determined to be a non-game mode priority.

[0010] Preferably, in step 4, the first preset condition corresponds to the game mode priority; Under the first preset condition, when the wireless screen projection receiver is coordinating the adjustment of the screen projection strategy, it disables the internal frame buffer queue and image post-processing algorithm, and locks the output clock to synchronize with the input video stream clock.

[0011] Preferably, in step 4, the second preset condition corresponds to the non-game mode priority; Under the second preset condition, the wireless projection receiver uses motion estimation and motion compensation algorithms to calculate the motion vectors of the two frames before and after the original frame, synthesizes the intermediate interpolated frame based on the motion vectors, and outputs the original frame and the interpolated frame alternately to match the highest refresh rate parameter of the display device.

[0012] Preferably, in step 3, the encoding and packaging process of the video stream by the wireless projection transmitter includes: The wireless projection transmitter decodes the received minimized transmission differential signal to separate the original image data and information frame data; The original image data is compressed using a preset encoder, and the content type priority is embedded as metadata in the wireless transmission protocol header and sent synchronously with the compressed image data.

[0013] Preferably, the dynamic synchronization processing strategy in step 4 specifically includes: The wireless projection receiver receives video data segments after they have been sliced ​​and encoded by the wireless projection transmitter. The wireless projection receiver decodes the video data segments and fills them into the frame buffer, and monitors the fullness of the frame buffer in real time. If the data in the frame buffer is detected to meet the output requirements, the decoded image is output to the display device according to the dynamic output frequency of the signal source device; If the frame buffer is found to be insufficient due to wireless transmission jitter, the display interface is controlled to continue outputting the previous frame image data until a new video data segment arrives and is decoded.

[0014] Preferably, the method is executed based on a distributed edge computing architecture, specifically including: The wireless projection transmitter acts as an edge-aware node and adopts a sub-frame-level slice encoding strategy to decompose one frame of the variable refresh rate signal into N horizontal strip slices. After capturing a slice, it is immediately compressed and sent without waiting for the complete frame to be captured. The wireless projection receiver acts as an edge rendering node, establishing a pipelined parallel mechanism for decoding and display. While decoding the currently received slice and writing it into the frame buffer, it simultaneously triggers the reading and display scanning of the ready slice data.

[0015] The beneficial effects of this invention are as follows: At the wireless receiving end, the original extended display identifier data is parsed and reassembled, and the detailed timing descriptor containing the highest refresh rate parameter is rearranged and set as the preferred optimal resolution. The wireless transmitting end triggers a hot-plug detection signal to simulate physical cable plugging and unplugging actions, inducing the signal source device to reread the original extended display identifier data and force the output of a high refresh rate video signal. By parsing the auxiliary video information frame, the content type is accurately identified. For game mode, an automatic low latency mode command is intelligently sent and the buffer is disabled to ensure real-time operation. For non-game mode, dynamic interpolation technology or anti-jitter slicing processing is activated to compensate for the frame rate and eliminate wireless jitter. Users only need to trigger the button to obtain a high-quality visual experience that combines forced high refresh rate and scene adaptation in the wireless projection environment. Attached Figure Description

[0016] Figure 1 The flowchart illustrates the steps of a button-triggered adaptive control method for wireless screen projection based on edge computing, as provided in one embodiment of the present invention.

[0017] Figure 2 This is a flowchart illustrating an edge computing-based button-triggered adaptive control method for wireless screen projection, as provided in one embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram illustrating the logical mapping of a button-triggered adaptive control method for wireless screen projection based on edge computing, provided as an embodiment of the present invention, which utilizes auxiliary video information frames to identify the priority of content types. Detailed Implementation

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Example, refer to Figure 1 and Figure 2 This paper presents an edge computing-based button-triggered adaptive control method for wireless screen projection, comprising the following steps: Step 1: In response to the physical button trigger signal in the wireless projection system, the wireless projection receiver reads the original extended display identification data of the connected display device through the physical interface, and parses out the display device's maximum refresh rate parameters, optimal resolution parameters, and variable refresh rate support parameters.

[0021] Step 2: The wireless projection receiver generates recombined extended display identifier data based on the highest refresh rate parameter and the variable refresh rate support parameter. It sets the timing descriptor containing the highest refresh rate parameter as the preferred timing and instructs the wireless projection transmitter to trigger a hot-plug detection signal through the wireless link. It then controls the signal source device to read the recombined extended display identifier data and output the video stream.

[0022] Step 3: The wireless projection transmitter parses the auxiliary video information frames and signal format features in the video stream, identifies the content type priority of the video stream, and packages the content type priority with the video stream encoding before transmitting it to the wireless projection receiver.

[0023] Step 4: The wireless screen mirroring receiver receives the data packet and processes it according to the content type priority and signal format characteristics. When the content type priority meets the first preset condition, the wireless screen mirroring receiver sends an automatic low latency mode command to the display device and transmits the video stream.

[0024] When the content type priority meets the second preset condition and the video stream frame rate is less than the maximum refresh rate parameter, the wireless screen projection receiver starts the dynamic frame interpolation algorithm to perform frame rate compensation and then outputs the video stream.

[0025] When the signal format characteristics indicate a variable refresh rate signal, the wireless screen projection receiver executes a dynamic synchronization processing strategy.

[0026] Wired interfaces include HDMI, DVI, Type C (alt mode), and DisplayPort.

[0027] This invention addresses the limitations of existing wireless screen mirroring, where the signal source is restricted by the default original extended display identifier data of the display device, preventing it from actively enabling high refresh rates. It also addresses the issue that a single processing strategy cannot simultaneously achieve low latency in games and high smoothness in videos. At the wireless screen mirroring receiver, the invention parses and generates reconstructed extended display identifier data, forcibly setting the timing descriptor containing the highest refresh rate parameter as the preferred timing. This is combined with a hot-plug detection signal to induce the signal source to reread the original extended display identifier data, thereby actively activating high refresh rate output. Simultaneously, by utilizing auxiliary video information frames to identify content priority, it achieves differentiated processing for low latency in game scene pass-through, dynamic frame interpolation compensation in video scenes, and dynamic synchronization in VRR scenes. This allows users to overcome limitations with just a button press, obtaining a screen mirroring experience that combines forced high refresh rate and scene adaptation.

[0028] In a specific embodiment, the physical button trigger signal in step 1 is generated by the physical button on the wireless projection transmitter or the wireless projection receiver.

[0029] When the physical button is located at the wireless projection transmitter, the wireless projection transmitter will encode the trigger status into a control command and send it to the wireless projection receiver via the wireless network. After parsing the control command, the wireless projection receiver will read the original extended display identifier data of the connected display device.

[0030] When the physical button is located on the wireless projection receiver, the wireless projection receiver responds and reads the original extended display identifier data of the connected display device.

[0031] The control command transmission between the wireless projection transmitter and receiver is implemented based on a customized private TCP / IP communication protocol. During initial deployment, the wireless projection transmitter and receiver pair up based on their respective MAC addresses, establishing a private communication channel with corresponding IP addresses. Devices that are not paired or have not established a connection cannot exchange data through this private communication channel. The control command packet generated by button triggering carries the unique identifier information of the paired device. After receiving the control command, the wireless projection receiver first verifies whether the unique identifier information in the command packet matches that of the paired wireless projection transmitter. Only after successful verification does it execute the subsequent original extended display identifier data reading operation, thereby preventing the button signals of other wireless projection devices in adjacent deployment environments from being mistakenly triggered. Simultaneously, the handshake, checksum, and timeout retransmission mechanisms inherent in the TCP protocol ensure the integrity and reliability of the control commands during wireless transmission.

[0032] This invention addresses the issue of insufficient convenience of button operation in wireless projection devices under different installation environments by designing a dual-end physical button triggering mechanism. Regardless of whether the physical button trigger signal is generated at the wireless projection transmitter or receiver, it can transmit control commands or respond directly through the wireless network, flexibly triggering the original extended display identifier data reassembly process. This solves the problem of users having difficulty touching the device to switch modes in specific scenarios, significantly improving the convenience of human-computer interaction and the flexibility of system deployment.

[0033] In a specific embodiment, the process of generating the recombinant extended display identifier data in step 2 includes: The wireless screen mirroring receiver parses the Video Electronics Standards Association (VESA) data block and Consumer Electronics Association (CEA) data block from the original extended display identifier data and extracts a detailed timing descriptor list.

[0034] The wireless screen projection receiver obtains the maximum transmission bandwidth capability of the wireless screen projection system, performs an intersection operation on the detailed timing descriptor list and the maximum transmission bandwidth capability, filters out candidate timing descriptors, and selects the target timing descriptor containing the highest refresh rate parameter from the candidate timing descriptors.

[0035] In one specific embodiment of the present invention, the intersection of the detailed timing descriptor list and the maximum transmission bandwidth capability is performed, and coding feasibility verification logic based on a fixed bandwidth upper limit is adopted, specifically including: Set a maximum transmission bandwidth for the wireless screen mirroring receiver. , Based on the physical layer stability requirements of the wireless communication module, this represents the maximum output bitrate allowed by the system to ensure smooth video stream transmission, unaffected by fluctuations in instantaneous link negotiation rate. The maximum compression bitrate is set to 30Mbps.

[0036] Iterate through each detailed timing descriptor in the original extended display identifier data, and calculate the target compression ratio required to compress the detailed timing descriptor to the upper limit of the transmission bandwidth. The mathematical expression for the target compression ratio is: ; in, For the target compression ratio, The upper limit of transmission bandwidth is represented by the original uncompressed data volume in the denominator. Indicates the total number of horizontal pixels. This represents the total number of pixels vertically. For refresh rate, This represents the number of bits for color depth, typically 24.

[0037] Obtain the maximum compression capability parameter of the current hardware encoder. That is, the minimum compression ratio that the encoder can achieve while ensuring that the image is recognizable, calculated from each detailed timing descriptor. and Compare them.

[0038] like If the current hardware encoder is capable of compressing the corresponding timing video stream to within the transmission bandwidth limit by adjusting encoding parameters, such as quantization parameter QP or discarding high-frequency details, then the corresponding detailed timing descriptor is retained in the candidate timing descriptor list.

[0039] like If the system determines that even with the strongest compression, the data volume will still exceed the system's transmission capacity or the image quality will be severely distorted, then the detailed timing descriptor will be removed from the list. The dynamic parameter configuration is as follows: when the signal source device outputs the video stream according to the selected target timing, the wireless projection transmitter will enable the bitrate control mechanism to adjust the encoder's quantization parameters according to the complexity of the video stream content, ensuring that the final output video bitrate is always maintained within the upper limit of the transmission bandwidth, thereby achieving high resolution and high refresh rate transmission within a limited bandwidth.

[0040] The bitrate control mechanism is executed frame-by-frame by the hardware encoder at the wireless projection transmitter, supporting both fixed bitrate (CBR) and variable bitrate (VBR) modes. In CBR mode, the hardware encoder strictly locks the output bitrate of each frame within the transmission bandwidth limit. In VBR mode, the hardware encoder adjusts the quantization parameter (QP) in real-time based on the complexity of the current frame. When the complexity increases sharply, the hardware encoder actively increases the quantization parameter to discard high-frequency details, limiting the instantaneous peak bitrate within the transmission bandwidth limit. When the complexity decreases, the hardware encoder correspondingly decreases the quantization parameter to restore image details. At the software level, the encoder uses preset parameters such as maximum bitrate (maxrate), buffer size (bufsize), keyframe interval (GOP), and encoding profile (H.264 profile) to adapt to the transmission characteristics of different wireless communication modules. The accompanying wireless communication module adopts Wi-Fi 4 (802.11n) and above standards. Its physical layer transmission bandwidth is much higher than the transmission bandwidth limit, providing ample bandwidth margin for the transmission of encoded video streams and avoiding screen mirroring stuttering caused by bandwidth overflow.

[0041] The wireless screen projection receiver moves the target timing descriptor to the first address of the detailed timing descriptor list and encapsulates the reordered detailed timing descriptor list into recombined extended display identifier data.

[0042] In a specific embodiment of the present invention, generating recombined extended display identifier data based on the highest refresh rate parameter specifically includes: The wireless screen mirroring receiver reads the original extended display identifier data, which is 128 or 256 bytes long, locates the base data block at addresses 0x00 to 0x7F, and the extended data block at addresses 0x80 to 0xFF, which is usually the Consumer Electronics Association standard extended block. It parses the 18-byte detailed timing descriptor starting at address 0x36 in the base data block, as well as the video data block in the data block set in the extended data block.

[0043] Extract the data structure containing the target detailed timing descriptor with the highest refresh rate parameter, completely overwrite the target detailed timing descriptor to the position of the preferred detailed timing descriptor in the base data block, i.e., address 0x36 to 0x47, and sequentially shift or fill the descriptor at the original position or other secondary descriptors into the subsequent detailed timing descriptor storage slots, such as address 0x48.

[0044] Because changes to the extended display identifier data structure will invalidate the original checksum, the wireless projection receiver must recalculate the checksum for both the base data block and the extended data block, specifically: The checksum is equal to 256 minus the modulo of the sum of the first 127 bytes in the data block. The calculated new checksum is written to address 0x7F of the base data block and address 0xFF of the extended data block to ensure that the reassembled data conforms to the specifications set by the Video Electronics Standards Association (VESA) and can be correctly recognized by the signal source device without errors. If, during the reordering process, there is no remaining storage space to accommodate all the original resolutions, the lower refresh rate timings are discarded first, in ascending order of refresh rate, to ensure the integrity of the data structure.

[0045] This invention addresses the problem of blindly forcing high refresh rates potentially exceeding the wireless network's transmission bandwidth, leading to black screens or stuttering during screen mirroring. It introduces a bandwidth intersection operation mechanism. Before generating the recombined extended display identifier data, a detailed list of timing descriptors is extracted and compared with the maximum transmission bandwidth capability of the wireless screen mirroring system. This ensures that the selected target timing descriptors meet both the high refresh rate requirement and are within the carrying capacity of the wireless link. By moving the compliant timing descriptors to the first address position, the risk of screen mirroring failure due to bandwidth overflow is eliminated at the physical level, guaranteeing connection stability in high refresh rate mode.

[0046] In a specific embodiment, the method by which the wireless projection transmitter triggers the hot-plug detection signal in step 2 is as follows: Adjust the voltage value of the hot-plug detection pin on the connection interface between the wireless projection transmitter and the signal source device, reduce the voltage value to below the preset low level judgment threshold, and after maintaining the preset reset time threshold, increase the voltage value to above the preset high level judgment threshold.

[0047] In a preferred embodiment of the present invention, the hot-plug detection signal is triggered by strictly controlling the electrical characteristics of pin 19 (HPD pin) of the HDMI interface, specifically as follows: The wireless projection transmitter integrates a GPIO control circuit connected to the HPD pin. When it receives a command indicating that the original extended display identifier data reconstruction is complete from the receiver, the transmitter first pulls the HPD pin low, controlling the GPIO output to be low and forcibly pulling the voltage below 0.8V. The logic low-level threshold is typically between 0V and 0.5V. This low-level state is maintained for a first preset duration. To ensure that various signal source devices detect the interrupt signal, the first preset duration is set to 200ms. This duration is based on the HPD interrupt pulse width requirements in the HDMI specification and is sufficient to trigger the connection interrupt handling procedure at the source end.

[0048] To ensure the signal source device fully detects the electrical disconnection from the display device, the wireless projection transmitter pulls the HPD pin voltage low while simultaneously pulling the DDC bus low and disabling the terminating impedance of the TMDS differential signal. This synchronized low-pull of the three signals constitutes a complete simulated disconnection sequence, ensuring the signal source device's detection logic perfectly matches the state when the HDMI cable is actually physically disconnected. Because this operation strictly adheres to the electrical specifications and hot-plug detection timing requirements set by the HDMI Association, all HDMI-certified signal source devices can correctly respond to this simulated plug-in / plug-out signal, ensuring compatibility with various signal source devices.

[0049] Furthermore, after the reset time ends, the transmitter controls the GPIO to output a high level or restore the pull-up resistor connection, rapidly raising the HPD pin voltage to above 2.4V. The logic high level judgment threshold is 2.4V to 5.0V. The rising edge signal will notify the signal source device that the new display has been connected, thereby triggering the source operating system to reread the recombined extended display identifier data in the DDC channel and complete the update of the video output format.

[0050] This invention addresses the problem that signal source devices cannot be triggered to re-read extended display identifier data without physically plugging or unplugging cables. This solution innovatively employs electrical simulation logic, precisely adjusting the voltage value of the hot-plug detection pin. It first lowers the voltage to below a preset low-level judgment threshold and maintains it at a preset reset time threshold, then raises it to above a preset high-level judgment threshold. This perfectly simulates the physical plugging and unplugging action of cables at the circuit level, successfully tricking the signal source device into re-handshaking and reading the re-read extended display identifier data in the connected state, achieving near-lossless and seamless hot-switching of display modes.

[0051] In a specific embodiment, the content type priority in step 3 includes game mode priority and non-game mode priority.

[0052] The wireless screen projection transmitter determines the priority of the current content type by parsing the logical state of the content type flag bits CN0 and CN1 in the auxiliary video information frame.

[0053] When the logical state of CN0 and CN1 indicates that the content type is graphics or game, it is determined to be game mode priority.

[0054] When the logical state of CN0 and CN1 indicates that the content type is movie or photo, it is determined to be non-game mode priority.

[0055] MEMC and AI interpolation are more suitable for smooth gameplay and movie playback. For low-latency gaming scenarios, interpolation or frame multiplication is unsuitable, as frame multiplication essentially involves redundant detection and doesn't help reduce latency. However, sending ALLM (Auto Low Latency Mode) via HDMI Tx to the screen can disable MEMC and PQ, reducing frame latency across the entire wireless projection path. Therefore, the source can send an IT / CN0 / CN1 AVIinfo frame to Tx. Tx determines the content type; if it identifies it as "Game," it instructs Rx to send an AVIinfo frame indicating "Game" and sends ALLM to the screen (if the screen supports this mode), minimizing latency. For other modes, it checks if the input source is high refresh rate; if so, it projects directly to the monitor. Otherwise, it performs interpolation on Rx to convert to high refresh rate before outputting to the monitor.

[0056] like Figure 3 As shown, in a specific embodiment of the present invention, the specific logic for identifying the content type priority of the video stream is as follows: The wireless screen projection transmitter performs protocol layer parsing on the video stream input from the signal source device, extracts the auxiliary video information frame AVIinfoframe, locates the 5th data byte of the auxiliary video information frame, and reads the content type flag bits defined therein, namely CN1 bit and CN0 bit.

[0057] The wireless projection transmitter decodes and determines the binary values ​​of bits CN1 and CN0 according to the mapping table defined in the CEA-861 standard protocol: When the parsed values ​​are CN1=1 and CN0=1, the content type is indicated as "Game". When the parsed values ​​are CN1=0 and CN0=0, the content type is indicated as "Graphics". Given that game scenes have extremely high real-time operation requirements, and graphics scenes (such as PC desktop operations) are also sensitive to mouse movement lag, this embodiment categorizes game and graphics scenes into a game mode priority, i.e., a low-latency priority mode. Under game mode priority, the Automatic Low-Latency Mode (ALLM) instruction will be triggered, notifying the display terminal to disable Motion Estimation and Motion Compensation (MEMC) and Image Quality Enhancement (PQ) algorithms to minimize frame transmission latency.

[0058] When the parsed values ​​are CN1=1 and CN0=0, the content type is indicated as "Cinema". When the parsed values ​​are CN1=0 and CN0=1, the content type is indicated as "Photo". Given that movie and photo scenes prioritize smoothness and color reproduction, and that users are not sensitive to millisecond-level input latency, this embodiment categorizes movie and photo scenes as non-game mode priority, i.e., image quality priority mode. Under non-game mode priority, if the input frame rate is low, the receiver is allowed to enable dynamic frame interpolation algorithms or frequency multiplication to improve visual smoothness.

[0059] This invention addresses the problem of mismatched processing strategies caused by the difficulty of automatically and accurately distinguishing between game and video scenes in existing technologies. It utilizes auxiliary video information frames in the HDMI protocol to lock content attributes by parsing the metadata values ​​of CN0 and CN1 bits, accurately determining the priority of game mode and non-game mode. This abandons the fuzzy recognition based on traditional screen statistics and provides a precise decision basis for subsequent differentiated edge computing processing.

[0060] In a specific embodiment, in step 4, the first preset condition corresponds to the game mode priority.

[0061] Under the first preset condition, when the wireless screen projection receiver is coordinating the adjustment of the screen projection strategy, it disables the internal frame buffer queue and image post-processing algorithm, and locks the output clock to synchronize with the input video stream clock.

[0062] Specifically, when the content type priority is determined to be game mode priority, the wireless projection transmitter simultaneously switches the transmission protocol of the video slice stream from TCP to UDP to eliminate the impact of TCP timeout retransmission and head-of-line blocking on frame transmission latency. At this time, the wireless projection receiver strictly maps the frame buffer according to the slice sequence number index. If individual slices are lost due to network fluctuations, the wireless projection receiver immediately abandons waiting for the retransmission of the lost slice and reuses the slice data from the corresponding spatial position in the previous frame to fill the output, thus maintaining the continuity of the image while maintaining sub-frame level transmission latency.

[0063] This invention addresses the pain point of unresponsive operation caused by traditional image processing pipelines in gaming scenarios. Under a first preset condition, it forcibly disables the internal frame buffer queue and image post-processing algorithm, and locks the output clock to be synchronized with the input video stream clock, thus constructing a clean hardware pass-through channel. This eliminates the input latency introduced by buffering and image enhancement algorithms in conventional screen projection, ensuring real-time response and control feel in high refresh rate game screen projection.

[0064] In a specific embodiment, in step 4, the second preset condition corresponds to the non-game mode priority.

[0065] Under the second preset condition, the wireless projection receiver uses motion estimation and motion compensation algorithms to calculate the motion vectors of the two frames before and after the image, synthesizes the intermediate interpolated frame based on the motion vector, and outputs the original frame and the interpolated frame alternately to match the highest refresh rate parameter of the display device.

[0066] This invention addresses the issue of screen tearing or juddering when low frame rate videos are played directly on high refresh rate displays. Under a second preset condition, it utilizes edge computing to activate a dynamic frame interpolation algorithm. Based on motion vector synthesis, it synthesizes intermediate interpolated frames to compensate the low frame rate source to match the highest refresh rate parameter of the display device. This effectively eliminates the visual stuttering caused by frame rate mismatch, allowing ordinary video streams to exhibit a silky smooth visual effect when wirelessly projected.

[0067] In a specific embodiment, step 3, the encoding and packaging process of the video stream by the wireless projection transmitter includes: The wireless projection transmitter decodes the received minimized transmission differential signal to separate the original image data and information frame data.

[0068] The original image data is compressed using a preset encoder, and the content type priority is embedded as metadata in the wireless transmission protocol header and sent synchronously with the compressed image data.

[0069] This invention addresses the problem that the receiving end has difficulty in simultaneously obtaining the content type while decoding video to quickly switch strategies. In the preprocessing stage at the transmitting end, the parsed content type priority is embedded as metadata into the wireless transmission protocol header, so that the content type priority is strictly synchronized with the compressed image data. This ensures that the receiving end can know the processing strategy the moment it receives the data packet, avoiding strategy switching delays or screen flicker caused by signaling lag, and achieving microsecond-level synchronization between audio-visual streams and control streams.

[0070] In a specific embodiment, the dynamic synchronization processing strategy in step 4 includes: The wireless projection receiver receives video data segments that have been sliced ​​and encoded by the wireless projection transmitter. The wireless projection receiver decodes the video data segments and fills them into the frame buffer, and monitors the fullness of the frame buffer in real time.

[0071] If the data in the frame buffer is detected to meet the output requirements, the decoded image is output to the display device according to the dynamic output frequency of the signal source device.

[0072] If insufficient data is detected in the frame buffer due to wireless transmission jitter, the display interface is controlled to continue outputting the previous frame of image data until a new video data segment arrives and is decoded.

[0073] This invention addresses the problem that wireless network jitter can easily lead to interruptions or black screens in variable refresh rate signal transmission. It establishes a frame buffer monitoring mechanism. When wireless transmission jitter is detected, resulting in insufficient data, the intelligent control of the display interface maintains the output of the previous frame image data to keep it alive until the new slice-encoded video data segment arrives. This anti-jitter strategy masks the impact of network fluctuations on the display at the physical layer, effectively solving the stability problem of VRR projection in wireless environments.

[0074] In a specific embodiment, a button-triggered wireless screen projection adaptive control method based on edge computing is executed based on a distributed edge computing architecture, specifically including: The wireless projection transmitter acts as an edge-aware node and employs a sub-frame-level slicing encoding strategy. It decomposes a frame of image from a variable refresh rate signal into N horizontal strip slices. After capturing a slice, it immediately compresses and sends it without waiting for the complete frame to be captured.

[0075] The wireless projection receiver acts as an edge rendering node, establishing a pipelined parallel mechanism for decoding and display. While decoding the currently received slice and writing it into the frame buffer, it simultaneously triggers the reading and display scanning of the ready slice data.

[0076] The wireless projection transmitter and receiver employ a dual-channel transmission architecture. Signaling data requiring reliable transmission, such as control commands and extended display identifiers, is transmitted via a TCP protocol channel. Video slice stream data is transmitted via a UDP protocol channel to reduce transmission latency and avoid the head-of-line blocking effect of TCP affecting the real-time performance of subframe-level slice transmission.

[0077] When receiving video segments, the wireless screen mirroring receiver parses the segment sequence index carried in the H.264 bitstream header of each segment data and maps each segment to the corresponding spatial location in the frame buffer according to the sequence index. If the wireless screen mirroring receiver detects that a segment sequence number is missing, it triggers error masking processing, reuses the segment data of the corresponding spatial location of the previous frame in the frame buffer to fill the display, and avoids screen tearing or partial screen distortion caused by the loss of a single segment.

[0078] When the network condition between the wireless projection transmitter and the wireless projection receiver deteriorates to the point that normal video transmission cannot be maintained, both ends monitor the link status through a heartbeat detection mechanism. If no response is received from the other end for several consecutive heartbeat cycles, the network connection is determined to be interrupted. The wireless projection receiver outputs a disconnection prompt message on the display device and returns to the main interface to be reconnected, so as to prevent the user from continuously watching abnormal screens.

[0079] This invention addresses the problem that the high latency of traditional full-frame transmission mode cannot meet the requirements of e-sports-level screen projection. This solution innovatively adopts a distributed edge computing architecture. The transmitting end acts as an edge sensing node to execute a sub-frame level slice encoding strategy, and the receiving end acts as an edge rendering node to establish a pipeline parallel mechanism. It decomposes a frame image into N slices and transmits and decodes them immediately without waiting for the full frame to be captured. This reduces the end-to-end latency from the frame level to the slice level, breaking through the physical latency bottleneck of wireless screen projection.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. An edge-computing-based adaptive control method for key-triggered wireless screen projection, characterized in that, The method comprises the following steps: Step 1: In response to a physical key trigger signal in the wireless projection system, the wireless projection receiving end reads the original extended display identification data of the connected display device through the physical interface, and parses the highest refresh rate parameter, the optimal resolution parameter and the variable refresh rate support parameter of the display device; Step 2: The wireless projection receiving end generates recombined extended display identification data based on the highest refresh rate parameter and the variable refresh rate support parameter, sets the time sequence descriptor containing the highest refresh rate parameter as the preferred time sequence, and instructs the wireless projection transmitting end to trigger a hot plug detection signal through a wireless link, so that the signal source device reads the recombined extended display identification data and outputs a video stream; Step 3: The wireless projection transmitting end parses the auxiliary video information frame and signal format feature in the video stream, identifies the content type priority of the video stream, and transmits the content type priority and the video stream after being encoded and packaged to the wireless projection receiving end; Step 4: The wireless projection receiving end receives the data packet and processes the video stream according to the content type priority and the signal format feature, and when the content type priority meets a first preset condition, the wireless projection receiving end sends an automatic low delay mode instruction to the display device and transmits the output video stream; When the content type priority meets a second preset condition and the frame rate of the video stream is less than the highest refresh rate parameter, the wireless projection receiving end starts a dynamic frame insertion algorithm to compensate the frame rate and then outputs the video stream; When the signal format feature indicates a variable refresh rate signal, the wireless projection receiving end executes a dynamic synchronization processing strategy.

2. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 1, wherein, The physical key trigger signal in step 1 is generated by an entity key on the wireless projection transmitting end or the wireless projection receiving end; When the entity key is located on the wireless projection transmitting end, the wireless projection transmitting end encodes the trigger state as a control instruction and sends it to the wireless projection receiving end through a wireless network, and the wireless projection receiving end reads the original extended display identification data of the connected display device after parsing the control instruction; When the entity key is located on the wireless projection receiving end, the wireless projection receiving end reads the original extended display identification data of the connected display device in response.

3. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 1, wherein, The specific process of generating the recombined extended display identification data in step 2 comprises: The wireless projection receiving end parses the video electronics standards association data block and the consumer electronics association data block in the original extended display identification data, and extracts a detailed time sequence descriptor list; The wireless projection receiving end obtains the maximum transmission bandwidth capability of the wireless projection system, performs intersection operation on the detailed time sequence descriptor list and the maximum transmission bandwidth capability, selects a candidate time sequence descriptor, and selects a target time sequence descriptor containing the highest refresh rate parameter from the candidate time sequence descriptor, The wireless projection receiving end moves the target time sequence descriptor to the first address bit of the detailed time sequence descriptor list, and encapsulates the reordered detailed time sequence descriptor list as the recombined extended display identification data.

4. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 1, wherein, The specific way of triggering the hot plug detection signal by the wireless projection transmitting end in step 2 is: Adjusting a voltage value of a hot plug detection pin on a connection interface between the wireless projection transmitting end and the signal source device, lowering the voltage value below a preset low-level judgment threshold, and raising the voltage value above a preset high-level judgment threshold after maintaining a preset reset time threshold.

5. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 1, wherein, The content type priority in the step 3 includes a game mode priority and a non-game mode priority. The wireless projection transmitting end determines the content type priority currently belonging to by analyzing logic states of content type flag bits CN0 and CN1 in the auxiliary video information frame. When the logic states of the CN0 and CN1 indicate that the content type is graphics or game, the game mode priority is determined. When the logic states of the CN0 and CN1 indicate that the content type is movie or photo, the non-game mode priority is determined.

6. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 5, wherein, In the step 4, the first preset condition corresponds to the game mode priority. Under the first preset condition, the wireless projection receiving end disables an internal frame buffer queue and an image post-processing algorithm, and locks output clock synchronization with input video stream clock when cooperatively performing projection strategy adjustment.

7. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 5, wherein, In the step 4, the second preset condition corresponds to the non-game mode priority. Under the second preset condition, the wireless projection receiving end calculates a motion vector of two frames of images by using a motion estimation and motion compensation algorithm, synthesizes an intermediate interpolation frame based on the motion vector, and alternately outputs original frames and interpolation frames to match a highest refresh rate parameter of a display device.

8. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 1, wherein, In the step 3, the encoding and packaging process of the wireless projection transmitting end on the video stream includes: The wireless projection transmitting end decodes the received minimized transmission differential signal to separate original image data and information frame data; The preset encoder is used to compress the original image data, and the content type priority is embedded as metadata in a wireless transmission protocol header and is synchronously transmitted with the compressed image data.

9. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 8, wherein, The dynamic synchronization processing strategy in the step 4 specifically includes: The wireless projection receiving end receives video data segments sliced and encoded by the wireless projection transmitting end, decodes and fills the video data segments into a frame buffer, and monitors a filling state of the frame buffer in real time; If it is monitored that data of the frame buffer meets output requirements, the decoded image is output to a display device according to a dynamic output frequency of a signal source device; If it is monitored that data of the frame buffer is insufficient due to wireless transmission jitter, a display interface is controlled to maintain output of last frame image data until new video data segments arrive and are decoded.

10. The edge computing-based adaptive control method for key trigger type wireless projection according to claim 9, wherein, The method is executed based on a distributed edge computing architecture and specifically includes: The wireless projection transmitting end, as an edge perception node, adopts a sub-frame level slicing encoding strategy to split one frame of image of the variable refresh rate signal into N horizontal strip slices, compresses and transmits the slice immediately after the slice is captured, and does not need to wait for complete frame capture; The wireless projection receiving end, as an edge rendering node, establishes a pipeline parallel mechanism of decoding and display, simultaneously triggers reading and display scanning of ready slice data while decoding and writing the currently received slice into a frame buffer.

Images