Screen direction switching and picture transmission system based on USB (Universal Serial Bus) channel

Abstract

The invention relates to the field of computer peripherals and display control. The screen direction switching and picture transmission system based on the USB channel comprises a host end processing module, a screen end display module, a USB control module, a host-display bidirectional feedback module and a direction sensing module, the host end processing module is connected with the screen end display module through a USB channel, the direction sensing module is used for sensing the real-time direction of the screen end display module, and the USB control module is used for direction parameter interaction between the host end processing module and the screen end display module; the host end processing module is used for collecting original picture data and obtaining preprocessed original picture data according to the original picture data; and the screen end display module is used for receiving the preprocessed original picture data transmitted by the USB control module. Through multi-sensor fusion and hardware acceleration, high-precision real-time sensing and low-delay direction switching of the screen posture are achieved, and the problem that manual adjustment delay is high in a traditional scheme is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer peripherals and display control, in particular to a screen direction switching and picture transmission system based on a USB channel. BACKGROUND

[0002] In the field of intelligent display, the screen direction switching and picture transmission system based on a USB channel is widely used in office equipment, industrial monitoring, vehicle-mounted display, etc.

[0003] Traditional screens lack built-in posture sensing capability, and direction switching needs to rely on user manual operation of host software or physical buttons, with a response delay usually exceeding 200 ms, which cannot meet the needs of dynamic scenarios such as mobile office and vehicle navigation. Although some solutions introduce a single sensor such as an accelerometer, they are significantly disturbed by environmental noise, and the data processing relies on CPU software filtering, resulting in a direction solving error exceeding 5°, and often causing black edges or incomplete display after picture rotation. In addition, when multiple screens are expanded, the direction of each screen needs to be manually configured one by one, lacking a cooperative mechanism, resulting in low operation efficiency. SUMMARY

[0004] The present application provides a screen direction switching and picture transmission system based on a USB channel to solve the problems of high manual adjustment delay and poor transmission reliability in traditional solutions.

[0005] In a first aspect, the present application provides a screen direction switching and picture transmission system based on a USB channel, comprising:

[0006] a host processing module, a screen display module, a USB control module, a host-display bidirectional feedback module, and a direction sensing module;

[0007] The host processing module and the screen display module are connected through the USB control module, the direction sensing module is used to sense the real-time direction of the screen display module, and the USB control module is used for direction parameter interaction between the host processing module and the screen display module; the host processing module is used to collect original picture data and obtain preprocessed original picture data according to the original picture data; the screen display module is used to receive the preprocessed original picture data transmitted by the USB control module and obtain an adjusted display direction according to the processed original picture data, and the host-display bidirectional feedback module is used to monitor the state of the screen display module in real time and feed back to the host processing module.

[0008] Optionally, the direction sensing module comprises:

[0009] a sensor acquisition unit, configured to acquire screen posture data in real time through a built-in sensor unit, wherein the screen posture data comprises three-axis acceleration values, three-axis angular velocity values, and magnetic field intensity values;

[0010] a data filtering unit configured to filter the triaxial acceleration value, eliminate the influence of static gravity component, and extract dynamic motion characteristics;

[0011] an angle calculation unit configured to integrate the triaxial angular velocity value to derive the screen rotation angle, and calibrate the direction in combination with the magnetic field data;

[0012] a parameter fusion unit configured to obtain a fused direction parameter according to the triaxial acceleration value, the triaxial angular velocity value, and the magnetic field intensity value;

[0013] an update frequency obtaining unit configured to obtain a data update frequency according to the fused direction parameter;

[0014] an instruction obtaining unit configured to encapsulate a standardized instruction package according to the data update frequency, and send the instruction package to a host processing module through a communication interface;

[0015] a bidirectional confirmation unit configured to establish a bidirectional confirmation mechanism, and receive a response signal returned by the host processing module, and automatically retransmit if a timeout occurs.

[0016] Optionally, the USB control module comprises:

[0017] a channel establishing unit configured to establish a bidirectional communication channel between the host and the screen;

[0018] a bandwidth monitoring unit configured to monitor the USB bus bandwidth occupation state in real time, and dynamically allocate data transmission channel resources;

[0019] a protocol matching unit configured to obtain an optimal transmission protocol version according to the picture resolution and frame rate parameters;

[0020] an error recovery unit configured to generate a trigger error recovery instruction when a transmission exception is detected based on the optimal transmission protocol version, wherein the trigger error recovery instruction comprises data retransmission and link reestablishment, and the transmission is performed based on the trigger error recovery instruction;

[0021] a multi-device cooperation unit configured to obtain a multi-device cooperative working mode, and establish a fair bandwidth competition mechanism.

[0022] Optionally, the host processing module comprises:

[0023] a data receiving unit configured to obtain preset parameters according to original picture data;

[0024] a picture preprocessing unit configured to obtain lossless compression encoding according to the preset parameters;

[0025] a data sending unit configured to send the marked lossless compression encoding to the screen device through the USB channel.

[0026] A data verification unit is configured to perform double verification according to the lossless compression encoding.

[0027] Optionally, the screen-side display module comprises:

[0028] A data packet receiving unit is configured to receive compressed picture data packets through a dedicated interface and perform integrity verification;

[0029] A data decompression unit is configured to decompress and decode the data packets to restore original picture information;

[0030] A direction adjustment unit is configured to adjust the picture display direction and scale according to the received direction parameters;

[0031] A color correction unit is configured to perform color correction and dynamic range adjustment on the picture to optimize the display effect;

[0032] An image enhancement unit is configured to embed an image enhancement algorithm to improve the picture definition and contrast;

[0033] A touch mapping unit is configured to establish a touch coordinate mapping mechanism to achieve accurate touch positioning after direction switching;

[0034] A picture rendering unit is configured to complete picture rendering through a display controller to drive screen hardware;

[0035] An exception handling unit is configured to establish an exception handling mechanism to automatically diagnose and recover from display exceptions.

[0036] Optionally, the host-display bidirectional feedback module comprises:

[0037] A state monitoring unit is configured to collect screen-side temperature, signal strength, and direction parameter accuracy indicators in real time;

[0038] A quality evaluation unit is configured to calculate the SSIM value and end-to-end delay of picture transmission;

[0039] An exception diagnosis unit is configured to automatically grade when signal discontinuity or picture tearing is detected;

[0040] An adaptive adjustment unit is configured to dynamically adjust the host-side encoding parameters according to the feedback data;

[0041] A log feedback unit is configured to send device status reports to the host through a USB control channel.

[0042] Optionally, the multi-device cooperation unit comprises:

[0043] A device topology unit is configured to automatically identify multiple screen devices and physical layouts connected by the host through USB;

[0044] A synchronization control unit is configured to generate multi-screen direction switching data according to the multi-screen device and the physical layout;

[0045] A cross-screen picture processing unit is configured to implement seamless picture splicing and cross-screen interaction between the multi-screens.

[0046] Optionally, the host processing module captures original picture data in the graphic card frame buffer through a virtual graphic card driver interface within a preset period after initialization, and adaptively matches the resolution to a maximum supported value of the current display mode.

[0047] The host processing module parses the color gamut range and luminance parameters of the target display device according to EDID (Extended Display Identification Data) extended display identification data transmitted by the screen, calculates a conversion matrix from RGB to DCI-P3 color gamut, and converts to obtain a visually lossless range.

[0048] When the direction switching instruction code is detected, the host processing module calls a GPU hardware acceleration unit to perform a rotation transformation operation, completes the rotation processing within a preset time, and automatically crops a black border area generated by rotation to obtain preprocessed original picture data.

[0049] Optionally, the screen display module receives the preprocessed original picture data packet through a USB control channel, and parses a direction control identifier in a data packet header.

[0050] The screen display module compares the parsed direction identifier with a current display direction state register, and triggers a direction switching flag if the two are inconsistent.

[0051] When the direction switching flag is activated, the screen display module drives a display controller to perform the following associated operations in a vertical blanking period, wherein the associated operations include resetting a picture scanning direction, updating a frame buffer start address, and reconstructing a touch coordinate mapping matrix.

[0052] Compared with the related art, the screen direction switching and picture transmission system based on the USB channel provided in the application has at least the following technical effects:

[0053] The application realizes high-precision real-time sensing of the screen posture and low-delay direction switching through multi-sensor fusion and hardware acceleration, solves the problem of high delay in manual adjustment in the traditional scheme, improves data transmission reliability and bandwidth utilization through USB channel optimization and a bidirectional feedback mechanism, guarantees stable transmission of high-resolution pictures, ensures multi-device color consistency and smooth interaction through adaptive color gamut conversion and cross-screen picture processing, eliminates picture tearing and touch misalignment, and realizes automatic topology discovery and synchronization control through a multi-device collaborative unit.

[0054] The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the application will be apparent from the description of illustrative embodiments of the application and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0056] Figure 1 is a block diagram of a USB channel-based screen direction switching and picture transmission system according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] In order to make the objects, technical solutions and advantages of the application clearer, the application will be described and illustrated below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and should not be used to limit the application. Based on the embodiments provided in the application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the application.

[0058] It is obvious that the drawings in the following description are only some examples or embodiments of the application, and for those of ordinary skill in the art, the application can be applied to other similar scenarios without creative efforts based on the drawings. In addition, it can be understood that although the efforts made in the development process can be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the application, some design, manufacture or production changes based on the technical content disclosed in the application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the application.

[0059] Reference to "an embodiment" or "one embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the described embodiments of the application are merely example structures selected for the purposes of illustration and explanation and are presented to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the application. Obviously, one of ordinary skill in the art will readily recognize that the application can be practiced without these specific details.

[0060] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. Unless otherwise defined, the terms "one" and "a" or "an" used in the present application shall not be construed as being limited to a singular aspect, but rather can include a plurality or a group. The terms "comprises", "comprising", "includes", "including" and "have" or "has" or "having" or "contain" or "containing" used in the present application shall be construed to be open-ended, i.e., to mean including but not limited to. For example, a process, method, object, or device that comprises a list of steps or elements is not necessarily limited to only those steps or elements but can include other steps or elements not expressly listed or inherent to such process, method, object, or device. The terms "connected", "coupled", or "pathway" used in the present application shall not be construed as being necessarily limited to a direct connection or coupling between items, but rather can include an indirect connection or coupling between items through one or more intermediary items. The term "plurality" means two or more. The term "and / or" describes associated objects in association with the associated objects, which means that there are three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", etc. are merely used to distinguish similar objects, and do not represent a specific order for the objects.

[0061] The embodiment of the present application provides a screen direction switching and picture transmission system based on a USB channel. Figure 1 is a system diagram according to an exemplary embodiment. As shown in Figure 1 , it comprises:

[0062] a host processing module, a screen display module, a USB control module, a host-display bidirectional feedback module and a direction sensing module;

[0063] The host processing module and the screen display module are connected through the USB control module, the direction sensing module is used for sensing the real-time direction of the screen display module, and the USB control module is used for the direction parameter interaction between the host processing module and the screen display module; the host processing module is used for collecting original picture data and obtaining preprocessed original picture data according to the original picture data; the screen display module is used for receiving the preprocessed original picture data transmitted by the USB control module and obtaining an adjusted display direction according to the processed original picture data, the host-display bidirectional feedback module is used for monitoring the state of the screen display module in real time and feeding back to the host processing module, the host processing module captures the frame buffer data of a display card at initialization, converts a color gamut according to a screen EDID parameter, and calls a GPU to rotate and crop a picture at direction switching; and the screen display module resets a scanning direction and updates a touch mapping matrix in a vertical blanking interval (VBI).

[0064] In summary, the application is composed of a host processing module, a screen display module, a USB control module, a host-display bidirectional feedback module and a direction sensing module, high-precision picture rotation and low-delay transmission are realized through a USB3.0+ channel. The direction sensing module collects nine-axis sensor data (three-axis acceleration / angular velocity / magnetic field intensity) in real time, generates standardized direction parameters (0° / 90° / 180° / 270°) through Kalman filtering and fusion calculation, and transmits them to the host end through the special instruction channel of the USB control module. The host processing module intercepts the original picture of the graphics card frame buffer, and calls the GPU to perform rotation transformation matrix operation after responding to the direction instruction:

[0065]

[0066] wherein (x, y) represents the pixel coordinates of the original picture (with the picture center as the origin), (x 1 , y 1 ) represents the pixel coordinates after rotation, φ represents the rotation angle detected by the direction sensing module, sinφ and cosφ represent the trigonometric values of the rotation angle;

[0067] and automatically crops the black border area based on the angle φ (effective size calculation: W eff = W orig *

[0068] |cosφ|-H orig *|sinφ|), wherein W eff represents the effective width after rotation, W orig represents the original picture width, and H orig represents the original picture height. The processed picture is encoded by H.265 and transmitted to the screen end through the USB data channel.

[0069] The screen display module completes key operations during the vertical blanking period: parses the data packet header direction identifier, when it is inconsistent with the register state, drives the display controller to reset the scanning direction, updates the frame buffer address mapping, and reconstructs the touch coordinate affine matrix:

[0070]

[0071] wherein (X screen , Y screen ) represents the mapped screen logical coordinates, (X touch , Y touch ) represents the touch input coordinates, and (S x , S y ) represents the screen scaling factor. The inverse transformation of the rotation matrix is adopted for touch coordinate mapping, ensuring that the touch point accurately corresponds to the pixel of the picture after rotation.

[0072] Then, the synchronization is performed by color correction through a 3D-LUT table, and the host-display bidirectional feedback module monitors the screen temperature, signal strength and transmission quality in real time. The USB control module implements a three-level fault tolerance mechanism: 1ms interval retransmission when the direction instruction CRC fails, ESRGAN super-resolution reconstruction is enabled when 5 consecutive frames are lost, and the local cache is switched when the connection is interrupted for more than 200ms.

[0073] In one embodiment, the direction perception module comprises: a sensor acquisition unit for acquiring screen posture data in real time through a built-in sensor unit, wherein the screen posture data comprises three-axis acceleration values, three-axis angular velocity values and magnetic field intensity values; a data filtering unit for filtering the three-axis acceleration values, eliminating the influence of static gravity components and extracting dynamic motion features; an angle calculation unit for integrating the three-axis angular velocity values to derive the screen rotation angle, and combining the magnetic field data for direction calibration; a parameter fusion unit for obtaining fused direction parameters according to the three-axis acceleration values, three-axis angular velocity values and magnetic field intensity values; an update frequency acquisition unit for obtaining a data update frequency according to the fused direction parameters; an instruction acquisition unit for encapsulating and obtaining a standardized instruction package according to the data update frequency, and sending it to the host processing module through a communication interface; and a bidirectional confirmation unit for establishing a bidirectional confirmation mechanism, receiving a response signal returned by the host processing module, and automatically retransmitting if it times out.

[0074] In summary, the direction perception module of the present application realizes real-time high-precision perception, data processing and reliable transmission of screen three-dimensional space posture through a multi-sensor fusion architecture and hardware acceleration processing flow, provides accurate posture data support for a screen direction switching system based on a USB channel, and solves the problem of manual operation or wireless transmission delay in traditional non-intelligent display direction adjustment. The module integrates multi-axis sensors to synchronously acquire multi-dimensional posture data such as screen acceleration, angular velocity and magnetic field intensity, eliminates environmental noise and static gravity interference through multi-stage filtering processing, extracts dynamic motion features, calculates real-time angles through quaternion integration and Kalman filtering algorithm, generates standardized direction parameters through a hardware-accelerated parameter fusion unit, contains logical direction mapping and confidence information, and ensures accurate conversion from physical posture to digital instructions.

[0075] The sensor acquisition unit realizes full-dimension attitude data synchronous acquisition through a built-in multi-axis sensor, and improves stability in different environments with the aid of a temperature compensation algorithm; the data filtering unit purifies original data through gravity compensation, band-pass filtering and dynamic feature extraction, laying a foundation for subsequent calculation; the angle calculation unit adopts quaternion integration and extended Kalman filtering to avoid gimbal lock and realize full-angle high-precision calculation; the parameter fusion unit generates logic parameters that can directly drive screen switching through FPGA parallel processing, reducing the processing pressure of the host end; the update frequency acquisition unit improves the sampling frequency and pre-buffers key data when detecting a sharp attitude change, ensuring data integrity in dynamic scenes; the instruction acquisition unit encapsulates parameters in a special protocol format, transmits them through a USB interrupt channel with priority, and ensures low latency; and the bidirectional confirmation unit ensures the reliability of instruction transmission through a sliding window and a retransmission mechanism.

[0076] The module adapts to different working modes through dynamic power management, is compatible with wide temperature and humidity and vibration environments, and reserves an expansion interface to support function upgrade. The units work cooperatively to build a full-process hardware acceleration link from data acquisition to instruction output, solve the problems of insufficient precision of a single sensor, high delay of software filtering and poor reliability of wireless transmission in traditional solutions, significantly reduce the full-process delay of screen direction switching, improve angle calculation precision, and provide an automatic and low-delay screen direction switching solution for industrial control, intelligent office and other scenes.

[0077] In one embodiment, the USB control module comprises: a channel establishment unit configured to establish a bidirectional communication channel between the host end and the screen end; a bandwidth monitoring unit configured to monitor the USB bus bandwidth occupation state in real time and dynamically allocate data transmission channel resources; a protocol matching unit configured to obtain an optimal transmission protocol version according to picture resolution and frame rate parameters; an error recovery unit configured to generate a trigger error recovery instruction when a transmission exception is detected based on the optimal transmission protocol version, wherein the trigger error recovery instruction comprises data retransmission and link reconstruction, and transmission is performed based on the trigger error recovery instruction; and a multi-device cooperation unit configured to support a multi-device cooperative working mode and establish a fair bandwidth competition mechanism.

[0078] In summary, the USB control module of the present application realizes intelligent management of the full process of USB communication between the host end and the screen end through the cooperative operation of multiple units, and aims to solve the problems of low bus resource scheduling efficiency, insufficient multi-device cooperation capability and poor transmission reliability in the screen direction switching and picture transmission scenarios. The module establishes a bidirectional communication channel, dynamically monitors and allocates bandwidth, intelligently matches transmission protocols, efficiently handles transmission exceptions and supports fair competition of multiple devices, thereby building a complete technical link from channel establishment to error recovery, ensuring stable and low-latency transmission of high-resolution pictures and key instructions in the USB channel.

[0079] The channel establishment unit in the module divides the control end and the batch end into two types of communication channels based on the USB protocol standard, respectively used for transmitting direction instructions and picture data, and ensures the real-time of instruction transmission through fixed priority allocation to avoid being blocked by large amount of picture. The bandwidth monitoring unit senses the bus load state in real time, adopts the sliding window algorithm to predict the bandwidth demand, and when the bus utilization rate exceeds the threshold, adjusts the picture compression ratio or suspends the transmission of low priority devices to flexibly allocate resources, ensuring the smooth transmission of the main screen key data. The protocol matching unit calculates the required bandwidth according to the picture resolution, frame rate and other parameters, automatically switches the USB2.0 / 3.0 protocol version and optimizes the transmission parameters, and improves the bandwidth utilization and transmission efficiency in different scenes.

[0080] The error recovery unit relies on the USB layer CRC verification mechanism to verify the integrity of the transmission data in real time, triggers a hierarchical recovery strategy for the data packets that fail the verification, and significantly improves the transmission reliability through the sliding window retransmission and link reconstruction mechanism. The multi-device cooperation unit supports multiple screens to be connected through USB HUB, establishes a fair bandwidth competition and polling scheduling mechanism, and can dynamically allocate resources according to the device priority to ensure the stable operation of each device in the multi-screen cooperation scene.

[0081] The module design strictly follows the USB protocol specification, and solves the problems of instruction delay caused by fixed bandwidth allocation, low efficiency caused by rigid protocol matching, and transmission conflict caused by unordered multi-device competition through the standardized configuration of control end and batch end, adaptive adjustment of protocol parameters, and hardware-level implementation of multi-device scheduling strategy. Through deep cooperation of each unit, a closed-loop control of "monitoring-decision-execution" is formed, the direction instruction transmission delay is controlled at a very low level, the effective transmission rate of picture data is improved, and multi-screen devices can realize real-time and stable direction switching and picture display in industrial monitoring, design drawing and other scenes.

[0082] In one embodiment, the host processing module includes: a data receiving unit configured to obtain preset parameters according to original picture data; a picture preprocessing unit configured to obtain lossless compression coding according to the preset parameters; a data sending unit configured to send the marked lossless compression coding to the screen device through the USB channel; and a data verification unit configured to perform double verification according to the lossless compression coding.

[0083] In summary, the host processing module of the application realizes the full-process optimization processing of the original picture data through the hardware acceleration architecture of multi-unit cooperation, aims to solve the core problems of picture format compatibility, processing delay, transmission bandwidth and data reliability in the screen direction switching scene, and builds an efficient technical link from data reception to reliable transmission to ensure the real-time and accurate display of high-resolution pictures in the USB channel.

[0084] The module first realizes adaptive compatibility of multi-format original picture data (such as RGB, YUV, etc.) through a data receiving unit, realizes seamless access and pipeline processing of data from different sources through double-buffer design and format identification analysis, and avoids compatibility problems caused by format differences. The data decoding unit uses hardware engine for fast decompression of compressed data, extracts key parameters such as resolution and frame rate, and provides basis for subsequent processing, while supporting multi-thread parallel processing of multi-layer pictures to improve processing efficiency in complex scenarios.

[0085] The picture preprocessing unit realizes real-time geometric transformation of the picture based on direction switching instructions and decoding parameters through hardware-accelerated cropping, scaling and rotating circuits. The rotation function can quickly adjust the pixel arrangement according to the direction parameters, and combined with the bilinear interpolation algorithm to ensure the clarity of the scaled picture, solving the problem of high delay and quality loss in traditional software processing. The compression and encoding unit adopts a lossless prediction encoding algorithm, dynamically adjusts the compression strategy according to the USB bus state, effectively reduces the transmission data volume under the premise of ensuring picture quality, and balances the transmission efficiency and bandwidth occupation.

[0086] The synchronization marking unit injects high-precision timestamps and synchronization header information for each data packet to ensure the frame synchronization accuracy of the host and the screen, especially during direction switching, the forced synchronization marking is used to realize timely updating of display parameters. The data sending unit relies on the channel mechanism of the USB control module to transmit key frames and synchronization data preferentially, and avoids bus congestion through flow control to improve transmission stability. The data verification unit uses the dual mechanism of CRC32 verification and frame sequence number continuity verification, combined with the ACK / NACK signaling of the USB layer, to build an end-to-end data integrity protection system, significantly reducing the transmission error rate.

[0087] The module design focuses on hardware acceleration and parameter linkage, for example, the preprocessing unit dynamically adjusts the processing strategy according to the decoding parameters, the compression and encoding unit cooperates with the bandwidth monitoring unit to optimize the transmission efficiency, forming a closed-loop pipeline of "receiving-decoding-processing-compression-synchronization-sending-verification". Through this deep cooperation of multiple units, the module realizes low-delay processing (preprocessing delay is significantly lower than traditional solutions), intelligent compression (bandwidth demand is effectively reduced) and high-reliability transmission (accuracy reaches industry-leading level) of picture data, solving the efficiency bottleneck and compatibility limitations caused by relying on CPU software processing in traditional solutions.

[0088] In one embodiment, the screen-side display module comprises: a data packet receiving unit for receiving compressed picture data packets through a dedicated interface and performing integrity checking; a data decompression unit for decompressing and decoding the data packets to restore the original picture information; a direction adjusting unit for adjusting the picture display direction and scale according to the received direction parameters; a color correction unit for color correction and dynamic range adjustment of the picture to optimize the display effect; an image enhancement unit for built-in image enhancement algorithm to improve the picture definition and contrast; a touch mapping unit for establishing a touch coordinate mapping mechanism to realize accurate touch positioning after direction switching; a picture rendering unit for completing picture rendering through a display controller to drive the screen hardware; and an exception handling unit for establishing an exception handling mechanism to automatically diagnose and recover display exceptions.

[0089] In summary, the screen-side display module of the present application integrates multiple hardware processing units to build a full-process processing link from compressed data packet reception to picture rendering, aiming to solve the core problems of picture delay, color deviation, touch misalignment and display abnormalities of traditional displays during direction switching, and realize high-fidelity, low-delay presentation and accurate interaction of host pictures on the screen side.

[0090] The module first receives compressed picture data through the data packet receiving unit using the USB bulk endpoint, relies on the CRC-32 check and double buffering mechanism to ensure the integrity and continuity of the transmitted data, and avoids data interruption caused by bus fluctuations. The data decompression unit uses a hardware acceleration engine to quickly decompress compressed data in H.264 format, synchronously analyzes metadata such as resolution and direction parameters, and provides accurate parameter support for subsequent processing, significantly improving the processing efficiency of complex pictures.

[0091] The direction adjusting unit uses a hardware rotation matrix and a bilinear interpolation algorithm to complete real-time picture pixel rearrangement and scaling according to the received direction parameters, solves the high delay problem of traditional software processing, and ensures smooth transition of the picture during direction switching. The color correction unit uses a 3DLUT hardware module and a dynamic range adjustment algorithm to automatically calibrate the color matrix and contrast according to the change of ambient light, improves the picture display consistency under different lighting conditions, and avoids overexposure or underexposure. The image enhancement unit optimizes the detail performance of compressed pictures through edge sharpening, noise suppression and other algorithms, effectively improves the subjective clarity of low-code rate videos.

[0092] The touch mapping unit realizes accurate mapping of the touch point after screen rotation by establishing a dynamic coordinate conversion matrix and a five-point calibration algorithm, and ensures real-time correspondence of the touch operation and the display content in combination with a sampling rate synchronization mechanism, thereby improving the user interaction experience. The picture rendering unit supports multi-layer parallel rendering and dynamic backlight control by relying on an integrated GPU-level rendering pipeline and hardware synthesis technology, thereby ensuring stable output of high frame rate pictures and reducing frame rate fluctuation in a multi-task scenario.

[0093] The abnormality processing unit realizes automatic diagnosis and rapid recovery of display faults by triggering a three-level recovery mechanism (retransmission of the current frame, switching of a backup buffer, and hardware reset) through real-time monitoring of pixel mutation and frame rate abnormalities, thereby reducing the cost of manual intervention. The units form a closed-loop processing link of "receiving-decompression-adjustment-optimization-interaction-rendering-fault tolerance" through parameter linkage and pipeline operation, for example, the direction parameter simultaneously drives picture rotation and touch coordinate conversion, and the ambient light data links color correction and backlight adjustment, thereby ensuring the coordinated and efficient operation of the modules in different scenarios.

[0094] The module breaks through the bottleneck of traditional schemes in real-time performance and reliability through hardware acceleration and intelligent algorithm design, controls the total picture display delay at a very low level, significantly reduces the touch coordinate error, and quantitatively improves the color restoration accuracy and abnormal recovery efficiency.

[0095] In one embodiment, the host-display bidirectional feedback module comprises: a state monitoring unit for collecting screen end temperature, signal strength, and direction parameter accuracy index in real time; a quality evaluation unit for calculating the SSIM value and end-to-end delay of picture transmission; an abnormality diagnosis unit for automatically grading when signal discontinuity or picture tearing is detected; an adaptive adjustment unit for dynamically adjusting the host end encoding parameters according to the feedback data; and a log back unit for sending a device state report to the host through a USB control channel.

[0096] In summary, the host-display bidirectional feedback module of the present application integrates multi-dimensional state perception and dynamic adjustment units to build a real-time closed-loop feedback link between the host end and the screen end, aiming to solve the core problems of unknown display state, parameter adjustment lag, and passive abnormality processing in traditional one-way transmission systems, and to realize whole-process monitoring and active optimization of picture transmission quality in the screen direction switching scenario.

[0097] The module collects multiple types of key data on the screen end in real time through the state monitoring unit, including physical indicators such as temperature, USB signal strength, and direction parameter accuracy, and relies on sensors and USB PHY layer registers to achieve accurate data acquisition and periodic packaging transmission (period ≤ 50 ms), providing the host end with a real-time running state image of the screen end. The quality evaluation unit calculates the SSIM (Structural Similarity) value and other quantitative indicators such as end-to-end delay to establish a multi-parameter fusion quality evaluation model, converting factors such as picture clarity, transmission delay, and signal stability into a comprehensive score Q, providing data support for subsequent adjustments and avoiding the ambiguity and lag of traditional subjective evaluation.

[0098] The abnormality diagnosis unit classifies and judges the feedback data based on preset rules, and triggers differentiated response mechanisms for different severity of abnormal events such as signal discontinuity and picture tearing: first-level abnormalities (such as signal collapse) directly start link reconstruction, second-level abnormalities (such as direction deviation exceeding the limit) dynamically adjust encoding parameters, and third-level abnormalities (such as slight temperature rise) only record alarms. This hierarchical processing mechanism ensures that the system can quickly respond to key failures in complex scenarios, while avoiding excessive adjustments that interfere with normal operation.

[0099] The adaptive adjustment unit, as the execution core of the feedback link, dynamically adjusts the encoding strategy of the host end based on the quality evaluation results and abnormality diagnosis levels. For example, when picture quality degradation (Q value decrease) is detected, the H.264 encoding level is automatically upgraded to enhance detail transmission; if the USB bandwidth is insufficient, the token bucket algorithm is used to redistribute the code rate to prioritize key frame transmission. The adjustment period and delay are controlled within the acceptable range of the system, ensuring that parameter adjustment is synchronized with real-time picture demand, effectively reducing the stall rate and picture quality loss.

[0100] The log back unit asynchronously transmits device state data through the USB control channel, covering historical state curves, abnormal event records, and parameter adjustment trajectories, providing a complete data trace chain for system maintenance. Technical personnel can quickly locate periodic failures (such as picture tearing caused by bandwidth fluctuations) by analyzing logs, optimize transmission strategies, reduce manual troubleshooting costs, and improve maintenance efficiency.

[0101] The units of the module form a complete closed loop through data sharing and logical linkage: for example, the temperature data of the state monitoring unit can trigger the adaptive adjustment unit to reduce encoding complexity to reduce screen end power consumption, the SSIM value of the quality evaluation unit can be used as an important basis for abnormality diagnosis, and the log back unit provides historical data support for full-link optimization. This collaborative mechanism enables the system to real-time perceive environmental changes and dynamically adapt to USB transmission link fluctuations, maintaining picture transmission stability and high fidelity in complex scenarios such as vehicle vibration and industrial electromagnetic interference.

[0102] In one embodiment, the multi-device cooperation unit comprises: a device topology unit for automatically identifying the multi-screen devices and the physical layout connected by the host through USB; a synchronization control unit for generating multi-screen direction switching data according to the multi-screen devices and the physical layout; and a cross-screen picture processing unit for realizing seamless picture splicing and cross-screen interaction between the multi-screens.

[0103] In summary, the multi-device cooperation unit of the present application, through the deep cooperation of the device topology unit, the synchronization control unit and the cross-screen picture processing unit, builds an automatic cooperation system between the host and the multi-screen devices, aiming to solve the core problems of complex manual configuration, asynchronous direction switching and poor cross-screen interaction experience in traditional multi-screen display systems, and realizes plug-and-play, direction synchronization and seamless picture splicing of multi-screen devices.

[0104] The device topology unit, through the extension design of the USB enumeration mechanism, automatically reads the screen ID, resolution and other custom descriptor information when the device is connected, calculates the physical distance in combination with the USB signal attenuation characteristics, generates a two-dimensional topology matrix of the multi-screen device, and dynamically maps the spatial layout between the devices (such as side-by-side, stacked, etc.). This unit supports hot plug detection, and when a new screen is connected, it can quickly update the topology structure and trigger picture rearrangement logic, avoiding the tedious process of traditional manual dragging and setting, and significantly improving the deployment efficiency of multi-screens. For example, when the host is connected to multiple screens through a USB HUB, the topology unit can automatically identify the position relationship between the main screen and the extended screen according to the signal strength difference, without the need for manual adjustment by the user.

[0105] The synchronization control unit realizes real-time cooperation of multi-screen direction switching based on master-slave synchronization mode and broadcast signaling mechanism. By default, the first connected screen is the main screen, and its direction parameter is broadcast to the slave screen through the USB control channel with high frequency signaling. The slave screen can choose immediate switching or smooth transition strategy according to the picture content type (static / dynamic), ensuring the balance between direction adjustment and visual experience. At the same time, the unit supports independent direction mode, allowing the slave screen to set the display direction independently according to application requirements, meeting the differentiated needs of main screen horizontal display of documents and slave screen vertical display of tables in split-screen office scenarios, and improving the flexibility of multi-task processing.

[0106] The cross-screen picture processing unit combines multiple screens into a single logical display space through virtual extension desktop technology, supports cross-screen window dragging and content extension. The picture cutting algorithm intelligently partitions the host picture based on a topology matrix, uses feathering processing at the screen edge to eliminate the splicing gap, and realizes accurate positioning of cross-screen operation through a touch coordinate mapping model, ensuring pixel-level matching of the touch point and the display content. In addition, the unit avoids cross-screen picture tearing by synchronizing the vertical synchronization signals (VSYNC) of each screen, ensures smooth display of dynamic content (such as video and graphical interface) between multiple screens, and improves the continuity of user cross-device interaction.

[0107] In one embodiment, the host processing module captures the original picture data in the video card frame buffer through the virtual video card driver interface within a preset period after initialization, and adaptively matches the resolution to the maximum supported value of the current display mode. The host processing module parses the color gamut range and brightness parameters of the target display device according to the EDID extended display identification data transmitted by the screen end, calculates the conversion matrix from RGB to DCI-P3 color gamut, and converts to obtain the visually lossless range. When the direction switching instruction code is detected, the host processing module calls the GPU hardware acceleration unit to perform rotation transformation operation, completes rotation processing within a preset time, and automatically crops the black border area generated by rotation to obtain preprocessed original picture data.

[0108] In summary, the host processing module of the present application cooperates with the hardware driver level capability and intelligent algorithm to realize efficient capture of original picture data, low delay optimization of cross-device color gamut adaptation and direction switching, solve the problems of low picture acquisition efficiency, large color display deviation and rotation processing lag in traditional solutions, and build a high-fidelity, low-delay host picture output link.

[0109] After the module is initialized, it directly accesses the video card frame buffer through the virtual video card driver interface to capture the original picture data in real time. This interface is based on DirectX / OpenGL underlying technology and can synchronously obtain the original picture output by the video card, support adaptive matching of the resolution to the maximum supported value of the current display mode, and avoid the frame rate loss and resolution stretching blur problems of traditional software screenshot tools. For example, when the host is connected to screens with different resolutions, the module automatically adapts to the maximum resolution declared in the screen EDID, realizes pixel-level scaling through the Lanczos interpolation algorithm, improves the picture detail clarity, and ensures smooth edges of text and images without jaggies.

[0110] The EDID (Extended Display Identification Data) based on screen-side transmission is expanded, and the module parses the color gamut range (such as sRGB, DCI-P3), brightness parameter (such as nit value), and gamma curve of the target device, dynamically calculates the conversion matrix of RGB to the target color gamut, constructs the color space mapping relationship between the source color gamut and the target color gamut, and introduces the brightness mapping curve to adjust the channel gain, thereby realizing the visual lossless color conversion. For example, when the sRGB host picture is output to the OLED screen of the DCI-P3 color gamut, the module accurately adjusts the red primary color coordinate, so that the color reproduction error ΔE is significantly reduced, the color distortion caused by the traditional fixed matrix scheme is avoided, and the display authenticity of skin color, landscape and other scenes is improved.

[0111] When the direction switching instruction is detected, the module calls the GPU hardware acceleration unit to perform rotation transformation, and uses the parallel computing capability to complete the picture rotation in a very short time. In combination with the physical direction parameter of the screen, the module automatically calculates the size change of the picture after rotation, cuts off the black border area through the hardware cropping engine, and retains the effective display content. For example, in the scene of switching between horizontal screen and vertical screen, the GPU shader realizes single-frame processing of 90° rotation with extremely low delay, and dynamically fills or copies the adjacent pixels to eliminate the black border. Compared with the traditional CPU software processing scheme, the total delay of direction switching is greatly reduced, the proportion of the complete display area of the picture is significantly improved, and the smooth transition of dynamic content such as games and videos is ensured.

[0112] The module forms a complete processing link of “collection-adaptation-optimization” through efficient capture of the virtual graphics card interface, intelligent color gamut conversion of the EDID driver, and rotation and cropping of the GPU hardware acceleration. The technical implementation closely depends on the deep interaction between the hardware bottom layer capability and the device parameter, solves the core pain points of device compatibility and user experience in multi-screen display, is suitable for scenes such as design drawing, multi-screen game, video editing, which have high requirements on picture quality and real-time performance, promotes the upgrading of host picture output to intelligence and device independence, and improves the consistency and operation efficiency of multi-device collaborative display.

[0113] In one embodiment, the screen-side display module receives a preprocessed original picture data packet through a USB control channel, and parses a direction control identifier in a data packet header; the screen-side display module compares the parsed direction identifier with a current display direction state register, and if the two are inconsistent, triggers a direction switching flag; when the direction switching flag is activated, the screen-side display module drives the display controller to perform the following associated operations in the vertical blanking period, wherein the associated operations include resetting the picture scanning direction, updating the frame buffer start address, and reconstructing the touch coordinate mapping matrix.

[0114] In summary, the screen end display module of the application cooperates with the hardware level through the USB control channel, realizes the direction identifier analysis of the original picture data packet and the real-time switching of the display direction, aims to solve the problems of high direction switching delay, incomplete picture display and inaccurate touch coordinates in the traditional scheme, and constructs a low delay and high consistency display direction switching process.

[0115] The module first receives the preprocessed picture data packet sent by the host end through the USB control channel, and parses the direction control identifier in the header. The identifier carries the target display direction information (such as horizontal screen and vertical screen), which is quickly parsed through hardware circuit to ensure real-time acquisition of direction instructions. Subsequently, the module compares the parsed direction identifier with the built-in current display direction state register, and quickly determines whether to trigger direction switching by using hardware logic such as XOR operation, to avoid response lag caused by software level judgment delay.

[0116] When the direction identifier is detected to be inconsistent, the module activates the direction switching flag and drives the display controller to perform a series of hardware level related operations within the vertical blanking period (VBI) of the display device. The vertical blanking period is a period of time in the screen refresh cycle when no effective picture is displayed. The module uses this period of time to complete the adjustment of key parameters, avoiding affecting normal picture output. The specific operations include: resetting the picture scanning direction, adjusting the scanning register of the display controller to change the output order of the pixel data, so as to match the new display direction and ensure that there is no stretching or black edge residue in the picture; updating the frame buffer start address, re-mapping the storage and reading path of the pixel data through the design of the dual-port RAM and the hardware address conversion circuit, to realize the fast positioning and reading of the picture data after direction switching; reconstructing the touch coordinate mapping matrix, dynamically generating a new coordinate conversion relationship through the built-in 3x3 matrix register, to ensure that the touch point can be accurately mapped to the logical coordinates after the display direction changes, maintaining the continuity and accuracy of the operation.

[0117] These hardware level operations are processed in parallel and synchronized, which significantly improves the efficiency and reliability of direction switching. For example, the resetting of the scanning direction and the updating of the frame buffer address are realized by special circuits without the need for software intervention of the CPU, greatly reducing the processing delay. The matrix reconstruction of the touch coordinates is calculated in real time by the hardware multiplier to ensure the real-time response of the touch. The whole direction switching process is completed in the vertical blanking period, avoiding the visual stuttering or tearing caused by picture refresh interruption in the traditional scheme, and ensuring the smoothness of the display.

[0118] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific and detailed manner, but should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these are all within the scope of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. A screen direction switching and picture transmission system based on a USB channel, characterized by, The application relates to a host-terminal processing module, a screen-terminal display module, a USB control module, a host-terminal-display bidirectional feedback module and a direction sensing module. The host-terminal processing module and the screen-terminal display module are connected through the USB control module, the direction sensing module is used for sensing the real-time direction of the screen-terminal display module, the USB control module is used for the direction parameter interaction between the host-terminal processing module and the screen-terminal display module, the host-terminal processing module is used for collecting original picture data and obtaining pretreated original picture data according to the original picture data, the screen-terminal display module is used for receiving the pretreated original picture data transmitted by the USB control module and obtaining an adjusted display direction according to the pretreated original picture data, and the host-terminal-display bidirectional feedback module is used for monitoring the state of the screen-terminal display module in real time and feeding back to the host-terminal processing module. The direction sensing module comprises a sensor collecting unit, a parameter fusion unit, an update frequency obtaining unit and an instruction obtaining unit.

2. The USB channel-based screen orientation switching and picture transmission system according to claim 1, wherein, The sensor collecting unit is used for collecting screen posture data in real time through a built-in sensor unit, wherein the screen posture data comprises three-axis acceleration values, three-axis angular velocity values and magnetic field intensity values. The parameter fusion unit is used for obtaining fusion direction parameters according to the three-axis acceleration values, the three-axis angular velocity values and the magnetic field intensity values. The update frequency obtaining unit is used for obtaining data update frequency according to the fusion direction parameters. The instruction obtaining unit is used for encapsulating and obtaining a standardized instruction package according to the data update frequency and sending the standardized instruction package to the host-terminal processing module through a communication interface. The bidirectional confirmation unit is used for establishing a bidirectional confirmation mechanism, receiving a response signal returned by the host-terminal processing module and automatically retransmitting when a timeout occurs. The USB control module comprises a channel establishing unit, a bandwidth monitoring unit, a protocol matching unit, an error recovery unit and a multi-device cooperation unit.

3. The USB channel based screen orientation switching and picture transfer system according to claim 1, wherein, The channel establishing unit is used for establishing a bidirectional communication channel between the host terminal and the screen terminal. The bandwidth monitoring unit is used for monitoring the USB bus bandwidth occupation state in real time and dynamically allocating data transmission channel resources. The protocol matching unit is used for obtaining an optimal transmission protocol version according to picture resolution and frame rate parameters. The error recovery unit is used for generating a trigger error recovery instruction when transmission abnormity is detected based on the optimal transmission protocol version, wherein the trigger error recovery instruction comprises data retransmission and link reestablishment, and transmission is carried out based on the trigger error recovery instruction. The multi-device cooperation unit is used for obtaining a multi-device cooperative working mode and establishing a fair bandwidth competition mechanism. The host-terminal processing module comprises a data receiving unit, a picture preprocessing unit, a data sending unit and a data checking unit.

4. The USB channel based screen orientation switching and picture transfer system according to claim 1, wherein, The data receiving unit is used for obtaining preset parameters according to original picture data. The picture preprocessing unit is used for obtaining lossless compression coding according to the preset parameters. The data sending unit is used for sending the marked lossless compression coding to the screen-terminal device through a USB channel. The data checking unit is used for carrying out double verification according to the lossless compression coding. The screen-terminal display module comprises a data packet receiving unit, a data decompression unit, a direction adjusting unit and a color correction unit.

5. The USB channel based screen orientation switching and picture transfer system according to claim 1, wherein, The data packet receiving unit is used for receiving compressed picture data packets through a special interface and carrying out integrity checking. The data decompression unit is used for decompressing and decoding the data packets and restoring original picture information. The direction adjusting unit is used for adjusting picture display direction and proportion according to received direction parameters. The color correction unit is used for color correction and dynamic range adjustment of the picture and optimizes display effect. ​ An image enhancement unit is configured to embed an image enhancement algorithm to improve picture definition and contrast; A touch mapping unit is configured to establish a touch coordinate mapping mechanism to achieve accurate touch positioning after direction switching; A picture rendering unit is configured to complete picture rendering by driving screen hardware through a display controller; An exception handling unit is configured to establish an exception handling mechanism to automatically diagnose and recover display exceptions.

6. The USB channel based screen orientation switching and picture transfer system according to claim 1, wherein, The host-display bidirectional feedback module includes: A state monitoring unit is configured to collect screen end temperature, signal strength, and direction parameter accuracy indicators in real time; A quality assessment unit is configured to calculate the SSIM value and end-to-end delay of picture transmission; An exception diagnosis unit is configured to automatically grade when signal discontinuity or picture tearing is detected; An adaptive adjustment unit is configured to dynamically adjust host end encoding parameters according to feedback data; A log return unit is configured to send device status reports to the host through a USB control channel.

7. The USB channel based screen orientation switching and picture transfer system according to claim 3, wherein, The multi-device coordination unit includes: A device topology unit is configured to automatically identify multiple screen devices and physical layouts connected by the host through a USB connection; A synchronization control unit is configured to generate multi-screen direction switching data according to the multiple screen devices and physical layouts; A cross-screen picture processing unit is configured to achieve seamless picture splicing and cross-screen interaction between multiple screens.

8. The USB channel based screen orientation switching and picture transfer system according to claim 4, wherein, The host end processing module captures original picture data in the video card frame buffer through a virtual video card driver interface within a preset period after initialization, and adaptively matches the resolution to the maximum supported value of the current display mode; The host end processing module parses the color gamut range and brightness parameters of the target display device according to the EDID extended display identification data transmitted by the screen end, calculates the conversion matrix from RGB to DCI-P3 color gamut, and converts to obtain the visual lossless range; When the direction switching instruction code is detected, the host end processing module calls the GPU hardware acceleration unit to perform a rotation transformation operation, completes the rotation processing within a preset time, and automatically crops the black border area generated by rotation to obtain preprocessed original picture data.

9. The USB channel based screen orientation switching and picture transfer system according to claim 5, wherein, The screen end display module receives the preprocessed original picture data packet through the USB control channel, and parses the direction control identifier in the data packet header; The screen end display module compares the parsed direction identifier with the current display direction state register, and if they are inconsistent, triggers the direction switching flag; When the direction switching flag is activated, the screen end display module drives the display controller to perform the following associated operations during the vertical blanking period, wherein the associated operations include resetting the picture scanning direction, updating the frame buffer start address, and reconstructing the touch coordinate mapping matrix.

Images