Driving system, display device, distortion correction method, and computer program product

Abstract

A drive system is provided. The drive system includes one or more first processors and one or more second processors. The one or more first processors are configured to receive an image, and are configured to transmit the image to the one or more second processors. In the first display mode, the one or more second processors are configured to perform a first distortion correction on the image. In the second display mode, the one or more second processors are configured to perform a second distortion correction on the image. The first display mode is different from the second display mode; and the first distortion correction is different from the second distortion correction.

Description

TECHNICAL FIELD

[0001] The present application relates to display technology, and in particular, to a driving system, a display device, a distortion correction method and a computer program product. BACKGROUND

[0002] Head-up displays (HUDs) in vehicles are transparent displays that present data without requiring the user to look away from their usual point of view. HUD technology has entered the automotive industry to improve safety and convenience for drivers. HUDs in vehicles project important information directly onto the windshield in the driver's line of sight. This information can include speed, navigation directions, engine warnings, and other relevant data. The main benefit of an HUD is that it allows the driver to view the information while keeping their eyes on the road. This reduces the need to look down at the car's dashboard, thereby minimizing distraction and improving safety. SUMMARY

[0003] In one aspect, the present disclosure provides a driving system comprising one or more first processors and one or more second processors; wherein the one or more first processors are configured to receive an image, and are configured to send the image to the one or more second processors; in a first display mode, the one or more second processors are configured to perform a first distortion correction on the image; in a second display mode, the one or more second processors are configured to perform a second distortion correction on the image; the first display mode is different from the second display mode; and the first distortion correction is different from the second distortion correction.

[0004] Optionally, the first display mode is a partitioned display mode in which the driving system is configured to drive the display panel to display different scenes in different areas of the display panel; the different areas comprise a dashboard display area, a main console display area, and an auxiliary display area; and the second display mode is a full screen display mode in which the driving system is configured to drive the display panel to display a single scene substantially on the entire display panel.

[0005] Optionally, the one or more second processors are further configured to divide the image into a plurality of image portions comprising at least a first image portion.

[0006] Optionally, in the first display mode, the one or more second processors are further configured to re-scale at least the first image portion into a first re-scaled image portion; the re-scaling is performed at least partially along a first direction; the first image portion is stretched along the first direction; and the plurality of image portions are arranged along a second direction.

[0007] Optionally, the one or more second processors are further configured to add a black patch to the image.

[0008] Optionally, the rescaled compression coefficients are as follows:

[0009]

[0010] wherein V factor denotes a compression coefficient; R1 denotes a resolution of the image along a first direction; d2 denotes a maximum distortion applied to the first image portion to form the first rescaled image portion.

[0011] Optionally, the one or more second processors are further configured to write the first rescaled image portion and the other image portions into the memory; and store the first rescaled image portion and the other image portions independently in the memory; wherein the one or more second processors are further configured to read the image having the first resolution stored in the memory, and convert the image having the first resolution stored in the memory into an image having a second resolution; the image having the first resolution stored in the memory comprises the first rescaled image portion and the other image portions; and the image having the second resolution comprises a plurality of read image portions, wherein the plurality of read image portions are arranged in a different manner from an arrangement of the first rescaled image portion and the other image portions in the image having the first resolution.

[0012] Optionally, the one or more second processors are further configured to perform distortion correction on the plurality of read image portions read from the memory, and generate a plurality of corrected image portions.

[0013] Optionally, the one or more second processors are further configured to perform the distortion correction based on a plurality of sets of correction parameters.

[0014] Optionally, the one or more second processors are further configured to combine the plurality of corrected image portions, generate a combined image, and send the combined image to the display panel for image display.

[0015] Optionally, in the second display mode, the one or more second processors are further configured to write the plurality of image portions into the memory; and the plurality of image portions are stored independently in the memory; wherein the one or more second processors are further configured to read the image having the first resolution stored in the memory, and convert the image having the first resolution stored in the memory into an image having a second resolution; the image having the first resolution stored in the memory comprises the plurality of image portions; and the image having the second resolution comprises a plurality of read image portions, wherein the plurality of read image portions are arranged in a different manner from an arrangement of the plurality of image portions in the image having the first resolution.

[0016] Optionally, the one or more second processors are further configured to combine the plurality of read image portions, generating a combined image.

[0017] Optionally, the one or more second processors are further configured to perform distortion correction on the combined image, and generate a corrected image.

[0018] Optionally, the one or more second processors are further configured to perform a rotation distortion correction in which directions having less distortion are aligned as a plurality of rows for buffering.

[0019] Optionally, the one or more second processors are further configured to: rotate the corrected image; generate a rotated image; and send the rotated image to the display panel for image display.

[0020] Optionally, the one or more second processors are further configured to: write the corrected image to a memory; and read the corrected image stored in the memory; wherein the rotated corrected image is performed during reading the corrected image stored in the memory.

[0021] Optionally, the one or more first processors comprise a system on chip, and the one or more second processors comprise a field programmable gate array.

[0022] In another aspect, the present disclosure provides a display apparatus comprising the driving system described herein, and a display panel configured to receive the corrected image from the driving system.

[0023] In another aspect, the present disclosure provides a distortion correction method comprising: receiving, by one or more first processors, an image; sending, by the one or more first processors, the image to one or more second processors; performing, by the one or more second processors, a first distortion correction on the image in a first display mode; and performing, by the one or more second processors, a second distortion correction on the image in a second display mode; wherein the first display mode is different from the second display mode; and the first distortion correction is different from the second distortion correction.

[0024] In another aspect, the present disclosure provides a computer program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions executable by a processor to cause the processor to perform: receiving, by one or more first processors, an image; sending, by the one or more first processors, the image to one or more second processors; performing, by the one or more second processors, a first distortion correction on the image in a first display mode; and performing, by the one or more second processors, a second distortion correction on the image in a second display mode; wherein the first display mode is different from the second display mode; and the first distortion correction is different from the second distortion correction. BRIEF DESCRIPTION OF DRAWINGS

[0025] According to various disclosed embodiments, the following figures are examples for illustration purposes only and are not intended to limit the scope of the present invention.

[0026] Figure 1 is a schematic diagram illustrating a structure of a display device in some embodiments according to the present disclosure.

[0027] Figure 2 is a schematic diagram illustrating a display device in a scene in some embodiments according to the present disclosure.

[0028] Figure 3 illustrates a first image and a second image in a scene in some embodiments according to the present disclosure.

[0029] Figure 4 illustrates a full screen display mode in a display device in some embodiments according to the present disclosure.

[0030] Figure 5 illustrates a partitioned display mode in a display device in some embodiments according to the present disclosure.

[0031] Figure 6 is a schematic diagram illustrating a structure of a display device in some embodiments according to the present disclosure.

[0032] Figure 7 is a schematic diagram illustrating a plurality of image portions segmented by a video segmentation module in some embodiments according to the present disclosure.

[0033] Figure 8 is a schematic diagram illustrating at least an image portion re-scaled by a video rescaler module in some embodiments according to the present disclosure.

[0034] Figure 9 is a schematic diagram illustrating a second image in some embodiments according to the present disclosure.

[0035] Figure 10 is a schematic diagram illustrating an image stored in a memory in some embodiments according to the present disclosure.

[0036] Figure 11 is a schematic diagram illustrating an image read from a memory in some embodiments according to the present disclosure.

[0037] Figure 12 is a schematic diagram illustrating a process of generating a second image in some embodiments according to the present disclosure.

[0038] Figure 13 is a schematic diagram illustrating a maximum distortion of an image along a first direction in some embodiments according to the present disclosure.

[0039] Figure 14 This is a schematic diagram illustrating the maximum distortion of an image along a second direction according to some embodiments of the present disclosure.

[0040] Figure 15 This is a schematic diagram illustrating the structure of a display device according to some embodiments of the present disclosure.

[0041] Figure 16 This is a schematic diagram illustrating images stored in a memory according to some embodiments of the present disclosure.

[0042] Figure 17 This is a schematic diagram illustrating an image read from a memory according to some embodiments of the present disclosure.

[0043] Figure 18 This is a schematic diagram illustrating the process of generating a second image according to some embodiments of the present disclosure.

[0044] Figure 19 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure.

[0045] Figure 20 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure.

[0046] Figure 21 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure. Detailed Implementation

[0047] This disclosure will now be described in more detail with reference to the following embodiments. It should be noted that the following description of some embodiments presented herein is for illustrative and descriptive purposes only. It is not exhaustive or limited to the precise forms disclosed.

[0048] This disclosure provides, in particular, a driving system, a display device, a distortion correction method, and a computer program product that substantially eliminates one or more problems caused by the limitations and disadvantages of the prior art. In one aspect, the invention provides a driving system. In some embodiments, the driving system includes one or more first processors and one or more second processors. Optionally, the one or more first processors are configured to receive an image and are configured to send the image to the one or more second processors. Optionally, in a first display mode, the one or more second processors are configured to perform a first distortion correction on the image. Optionally, in a second display mode, the one or more second processors are configured to perform a second distortion correction on the image. Optionally, the first display mode differs from the second display mode. Optionally, the first distortion correction differs from the second distortion correction.

[0049] Figure 1This is a schematic diagram illustrating the structure of a display device according to some embodiments of the present disclosure. (Reference) Figure 1 In some embodiments, the display device includes one or more first processors P1, one or more second processors P2, and a display panel DP. In some embodiments, one or more first processors P1 include a system-on-a-chip (SoC), and one or more second processors P2 include a field-programmable gate array (FPGA). A SoC integrates a system solution or multiple functions onto a single microchip. A SoC provides a comprehensive solution to a specific application or set of requirements, rather than simply integrating basic system components such as processors, memory, and input / output controls. This can involve embedding specialized components or capabilities such as sensors, data processing algorithms, power management systems, and wireless communication modules all onto a single chip. A field-programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer or designer after manufacturing, hence "field-programmable." FPGAs are very versatile and widely used across various technology sectors because they can be programmed to perform a wide range of digital functions. Unlike traditional chips, which have fixed functions once manufactured, FPGAs can be reprogrammed to suit different needs and applications. This is achieved through a dedicated hardware description language (HDL), such as VHDL or Verilog.

[0050] Figure 2 This is a schematic diagram illustrating a display device in a scenario according to some embodiments of the present disclosure. Reference Figure 2 In some embodiments, the display panel DP is a head-up display (HUD) used in the vehicle. The display panel DP is the core component of the HUD system, located below the windshield WS of the vehicle, producing a virtual image VI that passes through the windshield WS and enters the driver's viewpoint VP at the driver's position. This arrangement is strategic for optimal visibility and minimal driver distraction. The HUD system projects information onto the windshield WS, producing a virtual image VI that appears to float in the driver's line of sight. This image is typically a reflection of the output of the display panel. The virtual image VI is positioned in such a way that it is easily visible to the driver without requiring the driver to take their focus off the road. This allows the driver to see important information (such as speed, navigation direction, and warning signals) while keeping their eyes on the road ahead. HUD technology improves safety and convenience by reducing the need for the driver to look down at traditional instrument clusters or infotainment systems. It is a feature increasingly found in modern vehicles designed to seamlessly integrate information into the driving experience.

[0051] Figure 3 A first image and a second image are shown in a scene according to some embodiments of the present disclosure. (Refer to...) Figure 3The first image IM1 represents the shape of the image input to the display panel, while the second image IM2 represents the shape of the image derived from the first image IM1 using an inverse distortion correction process. Due to the curvature of the windshield, inverse distortion correction needs to be performed on the first image IM1 input to the display panel to ensure that the image entering the driver's eye is a standard rectangle. Vehicle windshields are typically curved, which distorts the projected image from the HUD system. When viewed by the driver, this curvature can cause the image to appear distorted or warped. To counteract this distortion, the image projected by the HUD needs to undergo inverse distortion correction. This means that the original image from the display module is pre-distorted to compensate for the curvature of the windshield. The purpose of this correction is to ensure that the final image perceived by the driver maintains a standard rectangular shape, thus preserving the clarity and readability of the displayed information.

[0052] This disclosure provides a driving system, display device, and image display method to achieve an effective distortion correction method. The driving system, display device, and image display method can operate or be implemented in a first display mode or a second display mode. In some embodiments, the distortion correction method implemented in the first display mode differs from the distortion correction method implemented in the second display mode. In some embodiments, the first display mode is a partitioned display mode, wherein the display panel is configured to display different scenes in different areas of the display panel. In the first display mode, the screen is divided into different areas, each potentially displaying different content or requiring different levels of distortion correction. This mode allows for more targeted and effective correction. In some embodiments, the second display mode is a full-screen display mode, wherein the display panel is configured to display a single scene substantially across the entire display panel. In the second display mode, the entire screen outputs as a single unit. In some embodiments, the second display mode may require a standard distortion correction method applied to the entire screen. The inventors of this disclosure have found that by implementing different distortion correction methods separately in different display modes, the demand for logic resources used for distortion correction can be effectively reduced, especially in high-resolution display systems. In particular, FPGAs are excellent for handling complex parallel processing tasks, but their resources are limited. Effective strategies help ensure that FPGAs can perform distortion correction without being overloaded, thereby maintaining system performance and reliability.

[0053] Figure 4 A full-screen display mode in a display device according to some embodiments of the present disclosure is shown. (Refer to...) Figure 4 In full-screen mode, the display panel is configured to show a single scene across essentially the entire display panel. Standard distortion correction methods are applied across the entire screen. For example... Figure 4 As shown, the maximum distortion in full-screen mode is represented as d1.

[0054] Figure 5 This illustrates a partitioned display mode in a display device according to some embodiments of the present disclosure. (Refer to...) Figure 5 In zoned display mode, the display panel is divided into different zones, each configured to display different content or require a different level of distortion correction. In one example, the display panel in zoned display mode is divided into three zones: Zone 1, Zone 2, and Zone 3. The maximum distortion in Zone 1 is denoted as d3, and the maximum distortion in Zone 2 or Zone 3 is denoted as d2. Figure 4 and Figure 5 As shown, d2 and d3 are both smaller than d1. The inventors of this disclosure have discovered that by performing partitioned distortion correction on the input image, the maximum distortion in each region can be effectively reduced, thereby reducing the number of data lines stored in the cache, since less information needs to be processed and stored when the correction is applied more effectively. This method reduces the storage resources required for distortion correction. This efficiency is crucial in complex systems, such as vehicle displays, where space and processing power are extremely valuable.

[0055] In some embodiments, the three areas include an instrument panel display area configured to display instrument information; a main console display area configured to display navigation information; and an auxiliary display area configured to display (e.g., for passengers) auxiliary information.

[0056] The automotive industry is increasingly integrating high-resolution displays into vehicles to enhance the user experience, including providing drivers with critical information, supporting navigation, and offering entertainment options. However, optimizing the display of various information while ensuring optimal viewing angles for both the driver and passengers within the confined space of a vehicle and the limitations of available power remains a pressing challenge.

[0057] The inventors of this disclosure have discovered that the drive system and display device according to this disclosure can be specifically tailored to automotive environments where space and energy resources are limited, and the need to provide drivers and passengers with abundant information is crucial. This situation is particularly well-suited for multi-zone displays within vehicles to address the challenges encountered in automotive setups. The drive system and display device according to this disclosure solve this challenge by providing two different display modes. In an automotive context, these modes are specifically designed to maximize the use of limited space, optimize energy consumption, and provide drivers and passengers with rich and clear information.

[0058] Figure 6 This is a schematic diagram illustrating the structure of a display device according to some embodiments of the present disclosure. (Reference) Figure 6In some embodiments, the display device includes one or more first processors P1, one or more second processors P2, and a display panel DP. In some embodiments, the one or more first processors P1 include a system-on-a-chip, and the one or more second processors P2 include a field-programmable gate array (FPGA).

[0059] In some embodiments, one or more first processors P1 are configured to receive an image and to send the image to one or more second processors P2.

[0060] In some embodiments, one or more second processors P2 are configured to receive images from one or more first processors P1. In some embodiments, the one or more second processors P2 include a data receiving module HDMI_RX, which is configured to receive images transmitted from one or more first processors P1 via a data interface HDMI. In some embodiments, the data interface HDMI is a High Definition Multimedia Interface, and the data receiving module HDMI_RX is a High Definition Multimedia Data Receiving Module. In a particular example, the data receiving module HDMI_RX is configured to receive 8K×4K (e.g., 7680×4320) data signals.

[0061] In some embodiments, one or more second processors P2 further include a video segmentation module VIDEO_SPLIT, which is configured to divide an image into multiple image parts. The main function of the video segmentation module VIDEO_SPLIT is to acquire a single video input and segment or divide it into multiple outputs. This can be useful in various scenarios, such as displaying a single video source on multiple screens or segmenting a video feed to process different parts of an image individually.

[0062] Figure 7 This is a schematic diagram illustrating multiple image portions segmented by a video segmentation module according to some embodiments of the present disclosure. Reference Figure 3 , Figure 6 and Figure 7 In some embodiments, the video segmentation module VIDEO_SPLIT is configured to segment the first image IM2 into multiple image parts (including a first image part VID1, a second image part VID2, and a third image part VID3).

[0063] Figure 8 This is a schematic diagram showing at least some embodiments of an image rescaled by a video scaler module according to this disclosure. Reference Figure 3 , Figure 6 and Figure 8In some embodiments, one or more second processors P2 include a video scaler module (VSCALER) configured to rescale at least one of a plurality of image portions. Optionally, the VSCALER module is configured to change the size of the image portion. Optionally, the VSCALER module is configured to change the resolution of the image portion. Optionally, the VSCALER module is configured to change the aspect ratio of the image portion.

[0064] like Figure 7 and Figure 8 As shown, in some embodiments, the video scaler module VSCALER is configured to rescale a first image portion VID1 to a first rescaled image portion VID1-1. The rescaling is performed at least partially along a first direction DR1. The first image portion VID1 is stretched along the first direction DR1. Optionally, multiple image portions are arranged along a second direction DR2. The first direction DR1 and the second direction DR2 intersect each other and are different from each other. In one example, the first direction DR1 is perpendicular to the second direction DR2.

[0065] After stretching, a black padding is added to the image (a black padding operation), resulting in the first rescaled image portion VID1-1. The formula for calculating the compression factor of the Video Scaler module VSCALER is:

[0066]

[0067] Among them, V factor R1 represents the compression factor of the video scaler module VSCALER; R2 represents the resolution of the image along the first direction DR1; and d2 represents the maximum distortion applied to the first image portion VID1 to form the first rescaled image portion VID1-1. In a particular example, the resolution of the image along the first direction DR1 is 1040 pixels.

[0068] Figure 9 This is a schematic diagram illustrating a second image according to some embodiments of the present disclosure. Reference Figure 9 The maximum distortion applied to the first image portion is denoted as d2.

[0069] In some embodiments, the first rescaled image portion VID1-1 and other image portions (e.g., the second image portion VID2 and the third image portion VID3) are written to memory. In a particular example, reference... Figures 6 to 9The first rescaled image portion VID1-1 and other image portions (e.g., the second image portion VID2 and the third image portion VID3) are written to memory DDR3 via the Advanced Extensible Interface AXI_ARBITRATE and the ARM architecture-based processing unit ARM CORE. The first rescaled image portion VID1-1 and other image portions (e.g., the second image portion VID2 and the third image portion VID3) are stored independently in memory DDR3. In some embodiments, one or more second processors P2 include multiple Write Direct Memory Access (e.g., WDMA1, WDMA2, and WDMA3) configured to write the first rescaled image portion VID1-1 and other image portions (e.g., the second image portion VID2 and the third image portion VID3) to memory DDR3.

[0070] In some embodiments, the resolution of the display panel differs from the resolution of the image input from one or more first processors. For example, the display panel has a non-standard resolution setting. In a particular example, the image input from one or more first processors has a resolution of 8K×4K, while the display panel has a resolution of 10K×1K. Therefore, image segmentation needs to be performed on the image input from one or more first processors.

[0071] In some embodiments, one or more second processors P2 are configured to receive an image with a first resolution from one or more first processors P1 via a data interface, and are configured to convert the image into an image with a second resolution. In a particular example, the first resolution is 8K×4K, and the second resolution is 10K×1K.

[0072] In some embodiments, one or more second processors P2 are configured to read an image stored in memory with a first resolution and convert the image stored in memory with the first resolution into an image with a second resolution. Figure 10 This is a schematic diagram illustrating images stored in a memory according to some embodiments of the present disclosure. Figure 11 This is a schematic diagram illustrating an image read from a memory according to some embodiments of the present disclosure. Reference Figure 10 and Figure 11 In some embodiments, one or more second processors P2 are configured to read an image stored in memory (including a first rescaled image portion VID1-1, a second image portion VID2, and a third image portion VID3) and stitch multiple read image portions (including the first rescaled image portion VID1-1, the second image portion VID2, and the third image portion VID3) in different arrangements, thereby converting an image stored in memory with a first resolution into an image with a second resolution.

[0073] Reference Figure 10 The image stored in the memory has a first resolution of 7680×4320. Each of the multiple stored image portions (including the first rescaled image portion VID1-1, the second image portion VID2, and the third image portion VID3) stored in the memory has a resolution of 3498×1040. Multiple read image portions read from the memory and stitched together in different arrangements have a second resolution of 10494×1040.

[0074] refer to Figure 6 In some embodiments, one or more second processors P2 include multiple read direct memory accesses (e.g., RDMA_1, RDMA_2, and RDMA_3) configured to read multiple stored image portions (e.g., a first rescaled image portion VID1-1, a second image portion VID2, and a third image portion VID3) from memory DDR3 via an advanced extensible interface AXI_ARBITRATE and an ARM architecture-based processing unit ARM CORE.

[0075] In some embodiments, one or more second processors P2 are further configured to perform distortion correction on a plurality of read image portions read from memory, thereby generating a plurality of corrected image portions. Figure 12 This is a schematic diagram illustrating the process of generating a second image according to some embodiments of the present disclosure. (See reference) Figure 12 In some embodiments, one or more second processors P2 are further configured to perform distortion correction on a plurality of read image portions (including a first rescaled image portion VID1-1, a second image portion VID2, and a third image portion VID3) read from memory, and generate a plurality of corrected image portions (e.g., including a first corrected image portion VID1-2, a second corrected image portion VID2-2, and a third corrected image portion VID3-2).

[0076] In some embodiments, one or more second processors P2 are configured to perform distortion correction on multiple read image portions read from memory based on multiple sets of correction parameters, respectively used to correct multiple read image portions. In a particular example, the multiple sets of correction parameters are multiple tables of correction parameters, each of which is configured to correct a corresponding read image portion among the multiple read image portions read from memory.

[0077] In some embodiments, multiple sets of calibration parameters are stored on external non-volatile memory, such as an external secure digital card. In some embodiments, upon system power-up, multiple sets of calibration parameters are loaded from the external non-volatile memory into memory (e.g., via the advanced extensible interface AXI_ARBITRATE and the ARM CORE-based processing unit). Figure 6 On DDR3).

[0078] In some embodiments, one or more second processors P2 further include multiple direct memory accesses (e.g., DMA1, DMA2, and DMA3) configured to read multiple sets of correction parameters stored in memory DDR3.

[0079] In some embodiments, one or more second processors P2 further include multiple distortion correction modules (e.g., DISTORTION_1, DISTORTION_2, and DISTORTION_3). In some embodiments, multiple sets of correction parameters stored in memory DDR3 are read and sent to the multiple distortion correction modules for use in each corresponding video stream. Distortion correction is performed independently for each of the three video streams. Distortion correction involves adjusting each video stream according to the correction parameters to correct any distortion in the image.

[0080] In some embodiments, reference Figure 6 and Figure 12 One or more second processors P2 further include a video merging module VID_MERGE, which is configured to combine multiple corrected image portions (e.g., including a first corrected image portion VID1-2, a second corrected image portion VID2-2, and a third corrected image portion VID3-2) to generate a combined image CI. In some embodiments, one or more second processors P2 are configured to send the combined image CI to a display panel DP for display.

[0081] Figure 13 This is a schematic diagram illustrating the maximum distortion of an image along a first direction according to some embodiments of the present disclosure. For example... Figure 13 As shown, the maximum distortion of the image along the first direction DR1 is represented as d1. Figure 14 This is a schematic diagram illustrating the maximum distortion of an image along a second direction according to some embodiments of the present disclosure. For example... Figure 14As shown, the maximum distortion of the image along the second direction DR2 is denoted as d2. The maximum distortion d1 of the image along the first direction DR1 is greater than the maximum distortion d2 of the image along the second direction DR2. The inventors of this disclosure have discovered that a rotation distortion correction scheme can be used to effectively correct image distortion, wherein the maximum distortion d1 of the image along the first direction DR1 is greater than the maximum distortion d2 of the image along the second direction DR2. In the rotation distortion correction scheme, the image is rotated such that the direction with less distortion is aligned to multiple rows for buffering. After distortion correction is completed, the image is then rotated back to its original orientation. This process reduces the number of data rows that need to be buffered during distortion correction, thereby reducing the system's storage resource requirements.

[0082] Figure 15 This is a schematic diagram illustrating the structure of a display device according to some embodiments of the present disclosure. (Reference) Figure 15 In some embodiments, the display device includes one or more first processors P1, one or more second processors P2, and a display panel DP. In some embodiments, the one or more first processors P1 include a system-on-a-chip (SoC), and the one or more second processors P2 include a field-programmable gate array (FPGA). In some embodiments, the one or more first processors P1 are configured to receive an image and are configured to transmit the image to the one or more second processors P2.

[0083] In some embodiments, one or more second processors P2 are configured to receive images from one or more first processors P1. In some embodiments, the one or more second processors P2 include a data receiving module HDMI_RX, which is configured to receive images transmitted from one or more first processors P1 via a data interface HDMI. In some embodiments, the data interface HDMI is a High Definition Multimedia Interface, and the data receiving module HDMI_RX is a High Definition Multimedia Data Receiving Module. In a particular example, the data receiving module HDMI_RX is configured to receive 8K×4K (e.g., 7680×4320) data signals.

[0084] In some embodiments, one or more second processors P2 further include a video segmentation module VIDEO_SPLIT, which is configured to divide an image into multiple image portions, such as a first image portion, a second image portion, and a third image portion. The primary function of the video segmentation module VIDEO_SPLIT is to acquire a single video input and segment or divide it into multiple outputs. This can be useful in various scenarios, such as displaying a single video source on multiple screens or segmenting a video signal to process different parts of an image individually.

[0085] In some embodiments, multiple image portions (e.g., a first image portion, a second image portion, and a third image portion) are written to memory. In a particular example, the first image portion, the second image portion, and the third image portion are written to DDR3 memory via the Advanced Extensible Interface AXI_ARBITRATE and an ARM architecture-based processing unit ARM CORE. The first image portion, the second image portion, and the third image portion are stored independently in DDR3 memory. In some embodiments, one or more second processors P2 include multiple Write Direct Memory Access (WDMA1, WDMA2, and WDMA3) configured to write the first image portion, the second image portion, and the third image portion to DDR3 memory.

[0086] In some embodiments, the resolution of the display panel differs from the resolution of the image input from one or more first processors. For example, the display panel has a non-standard resolution setting. In a particular example, the image input from one or more first processors has a resolution of 8K×4K, while the display panel has a resolution of 10K×1K. Therefore, image segmentation needs to be performed on the image input from one or more first processors.

[0087] In some embodiments, one or more second processors P2 are configured to receive an image with a first resolution from one or more first processors P1 via a data interface, and are configured to convert the image into an image with a second resolution. In a particular example, the first resolution is 8K×4K, and the second resolution is 10K×1K.

[0088] In some embodiments, one or more second processors P2 are configured to read an image stored in memory with a first resolution and convert the image stored in memory with the first resolution into an image with a second resolution. Figure 16 This is a schematic diagram illustrating images stored in a memory according to some embodiments of the present disclosure. Figure 17 This is a schematic diagram illustrating an image read from a memory according to some embodiments of the present disclosure. Reference Figure 16 and Figure 17 In some embodiments, one or more second processors P2 are configured to read images stored in memory (including a first image portion VID1, a second image portion VID2, and a third image portion VID3) and stitch multiple read image portions (including the first image portion VID1, the second image portion VID2, and the third image portion VID3) in different arrangements, thereby converting an image stored in memory with a first resolution into an image with a second resolution.

[0089] Reference Figure 16The image stored in the memory has a first resolution of 7680×4320. Each of the multiple stored image portions (including a first rescaled image portion VID1-1, a second image portion VID2, and a third image portion VID3) stored in the memory has a resolution of 3498×1040. Multiple read image portions, read from the memory and merged in different arrangements, have a second resolution of 10494×1040.

[0090] refer to Figure 15 In some embodiments, one or more second processors P2 include multiple read direct memory accesses (e.g., RDMA_1, RDMA_2, and RDMA_3) configured to read multiple stored image portions (e.g., first image portion VID1, second image portion VID2, and third image portion VID3) from memory DDR3 via the advanced extensible interface AXI_ARBITRATE and the ARM architecture-based processing unit ARM CORE.

[0091] In some embodiments, reference Figure 15 One or more second processors P2 also include a video merging module VID_MERGE, which is configured to combine multiple read image portions (e.g., including a first image portion VID1, a second image portion VID2, and a third image portion VID3) read from memory to generate a combined image CI.

[0092] Figure 18 This is a schematic diagram illustrating the process of generating a second image according to some embodiments of the present disclosure. (See reference) Figure 18 In some embodiments, one or more second processors P2 are also configured to perform distortion correction on the combined image CI to generate a corrected image CTI.

[0093] In some embodiments, one or more second processors P2 are configured to write the corrected image CTI to memory. In some embodiments, one or more second processors P2 also include write direct memory access (e.g., WDMA_4), which is configured to write the corrected image CTI to memory DDR3, for example, via the advanced extensible interface AXI_ARBITRATE and the ARM architecture-based processing unit ARM CORE.

[0094] In some embodiments, one or more second processors P2 are configured to read a corrected image CTI stored in memory DDR3. In some embodiments, one or more second processors P2 are configured to rotate the corrected image CTI during reading of the corrected image CTI stored in memory DDR3, thereby generating a rotated image RI. In some embodiments, one or more second processors P2 also include read direct memory access (e.g., RDMA_4) configured to read the corrected image CTI.

[0095] In some embodiments, one or more second processors P2 are configured to send a rotated image RI to a display panel DP. In some embodiments, one or more second processors P2 also include an embedded display port EDP, which is configured to send the rotated image RI from read direct memory access to the display panel.

[0096] In some embodiments, one or more second processors P2 further include a determining module configured to determine a display mode selected from a first display mode and a second display mode. Various suitable algorithms may be used to determine the display mode.

[0097] In some embodiments, the determining module is configured to receive user input and, upon receiving user input, determine the display mode. The drive system may allow the user to manually select the display mode according to their needs. This can be achieved through the vehicle's user interface or settings menu. The user can choose between zoned display or rotational distortion correction mode based on a specific environment.

[0098] In some embodiments, the determining module is configured to receive one or more sensor inputs and, upon receiving a sensor input, determine a display mode. Various suitable sensors can be used to generate the one or more sensor inputs. Examples of suitable sensors include vehicle speed sensors, GPS, or tilt sensors, which are configured to detect the vehicle's operating status or external environmental conditions. Based on the one or more sensor inputs, the drive system can automatically switch display modes. For example, a first display mode may be preferred during high-speed driving, while a second display mode may be more suitable during parking or low-speed driving.

[0099] In some embodiments, the determining module is configured to determine the display mode based on predefined conditions and rules. This can be based on factors such as time, location, and weather conditions. For example, different display modes may be needed during the day and at night.

[0100] In some embodiments, the determination module is configured to determine the display mode based on intelligent algorithms. Using intelligent algorithms and machine learning techniques, the system can make decisions based on historical data and real-time conditions. This approach may involve more complex algorithms and data analysis.

[0101] In some embodiments, the determining module is configured to determine the display mode based on user preferences. The driving system can automatically select a mode based on user preferences and habits. Users can set their preferred modes in the system, and the system will automatically switch modes according to these preferences.

[0102] In another aspect, this disclosure provides a display device. In some embodiments, the display device includes a driving system and a display panel configured to receive a calibrated image from the driving system. In some embodiments, the driving system includes one or more first processors and one or more second processors as described herein. In some embodiments, the display panel is a heads-up display panel.

[0103] On the other hand, this disclosure provides a distortion correction method. Figure 19 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure. (Reference) Figure 19 In some embodiments, the distortion correction method includes receiving an image via one or more first processors; sending the image to one or more second processors via the one or more first processors; performing first distortion correction on the image via the one or more second processors in a first display mode; and performing second distortion correction on the image via the one or more second processors in a second display mode. Optionally, the first display mode is different from the second display mode. Optionally, the first distortion correction is different from the second distortion correction.

[0104] In some embodiments, the first display mode is a partitioned display mode, wherein the driving system is configured to drive the display panel to display different scenes in different areas of the display panel. In some embodiments, the second display mode is a full-screen display mode, wherein the driving system is configured to drive the display panel to display a single scene substantially across the entire display panel.

[0105] In some embodiments, the method further includes dividing the image into a plurality of image portions comprising at least a first image portion by one or more second processors.

[0106] Figure 20 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure. (Reference) Figure 20 In some embodiments, the method further includes, in a first display mode, rescaling at least a first image portion to a first rescaled image portion by one or more second processors. Optionally, the rescaling is performed at least partially along a first direction. Optionally, the first image portion is stretched along the first direction. Optionally, multiple image portions are arranged along a second direction.

[0107] In some embodiments, the method further includes adding a black patch to the image via one or more second processors.

[0108] In some embodiments, the rescaling compression factor is as follows:

[0109]

[0110] Among them, V factor R1 represents the compression factor; R2 represents the resolution of the image along the first direction; and d2 represents the maximum distortion applied to the first image portion to form the first rescaled image portion.

[0111] In some embodiments, the method further includes writing the first rescaled image portion and other image portions into memory via one or more second processors. Optionally, the first rescaled image portion and other image portions are stored independently in memory. In some embodiments, the method further includes reading an image with a first resolution stored in memory via one or more second processors, and converting the image with the first resolution stored in memory into an image with a second resolution via one or more second processors. Optionally, the image with the first resolution stored in memory includes the first rescaled image portion and other image portions. Optionally, the image with the second resolution includes a plurality of read image portions, wherein the plurality of read image portions are arranged in a manner different from the arrangement of the first rescaled image portion and other image portions in the image with the first resolution.

[0112] In some embodiments, the method further includes performing distortion correction on a plurality of read image portions read from memory by one or more second processors, and generating a plurality of corrected image portions.

[0113] In some embodiments, the method further includes performing distortion correction based on multiple sets of correction parameters via one or more second processors.

[0114] In some embodiments, the method further includes combining multiple corrected image portions by one or more second processors to generate a combined image, and sending the combined image to a display panel for image display.

[0115] Figure 21 This is a flowchart illustrating a distortion correction process according to some embodiments of the present disclosure. (Reference) Figure 21In some embodiments, the method further includes, in a second display mode, writing multiple image portions into memory by one or more second processors. Optionally, the multiple image portions are stored independently in memory. In some embodiments, the method further includes reading an image with a first resolution stored in memory by one or more second processors, and converting the image with the first resolution stored in memory into an image with a second resolution. Optionally, the image with the first resolution stored in memory includes multiple image portions. Optionally, the image with the second resolution includes multiple read image portions, wherein the multiple read image portions are arranged in a manner different from the arrangement of the multiple image portions in the image with the first resolution.

[0116] In some embodiments, the method further includes combining multiple read image portions by one or more second processors and generating a combined image.

[0117] In some embodiments, the method further includes performing distortion correction on the combined image via one or more second processors and generating a corrected image.

[0118] In some embodiments, the method further includes performing rotational distortion correction via one or more second processors, in which directions with less distortion are aligned to a plurality of rows for buffering.

[0119] In some embodiments, the method further includes rotating and correcting the image via one or more second processors; generating a rotated image; and sending the rotated image to a display panel for image display.

[0120] In some embodiments, the method further includes writing the corrected image to a memory via one or more second processors; and reading the corrected image stored in the memory. Optionally, rotating the corrected image is performed during the reading of the corrected image stored in the memory.

[0121] In some embodiments, one or more first processors include a system-on-a-chip, and one or more second processors include a field-programmable gate array.

[0122] In another aspect, this disclosure provides a computer program product comprising a non-transient tangible computer-readable medium having computer-readable instructions thereon. In some embodiments, the computer-readable instructions are executable by one or more processors to cause one or more processors to: receive an image via one or more first processors; transmit the image to one or more second processors via one or more first processors; perform a first distortion correction on the image via one or more second processors in a first display mode; and perform a second distortion correction on the image via one or more second processors in a second display mode. Optionally, the first display mode differs from the second display mode. Optionally, the first distortion correction differs from the second distortion correction.

[0123] In some embodiments, the first display mode is a partitioned display mode, wherein the driving system is configured to drive the display panel to display different scenes in different areas of the display panel. In some embodiments, the second display mode is a full-screen display mode, wherein the driving system is configured to drive the display panel to display a single scene substantially across the entire display panel.

[0124] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: dividing an image into a plurality of image portions including at least a first image portion by one or more second processors.

[0125] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: in a first display mode, rescaling at least a first image portion to a first rescaled image portion by one or more second processors. Optionally, the rescaling is performed at least partially along a first direction. Optionally, the first image portion is stretched along the first direction. Optionally, multiple image portions are arranged along a second direction.

[0126] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to perform further actions: adding a black patch to the image by one or more second processors.

[0127] In some embodiments, the rescaling compression factor is as follows:

[0128]

[0129] Among them, V factor R1 represents the compression factor; R2 represents the resolution of the image along the first direction; and d2 represents the maximum distortion applied to the first image portion to form the first rescaled image portion.

[0130] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: writing a first rescaled image portion and other image portions into memory by one or more second processors. Optionally, the first rescaled image portion and other image portions are stored independently in memory. In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: reading an image with a first resolution stored in memory by one or more second processors, and converting the image with the first resolution stored in memory into an image with a second resolution by one or more second processors. Optionally, the image with the first resolution stored in memory includes the first rescaled image portion and other image portions. Optionally, the image with the second resolution includes a plurality of read image portions, wherein the plurality of read image portions are arranged in a manner different from the arrangement of the first rescaled image portion and other image portions in the image with the first resolution.

[0131] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: distortion correction on a plurality of read image portions read from memory by one or more second processors, and to generate a plurality of corrected image portions.

[0132] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: distortion correction based on multiple sets of correction parameters by one or more second processors.

[0133] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: combining multiple corrected image portions to generate a combined image by one or more second processors, and sending the combined image to a display panel for image display.

[0134] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: in a second display mode, writing multiple image portions into memory by one or more second processors. Optionally, the multiple image portions are stored independently in memory. In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: reading an image stored in memory with a first resolution by one or more second processors, and converting the image stored in memory with the first resolution into an image with a second resolution. Optionally, the image stored in memory with the first resolution includes multiple image portions. Optionally, the image with the second resolution includes multiple read image portions, wherein the multiple read image portions are arranged in a manner different from the arrangement of the multiple image portions in the image with the first resolution.

[0135] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: combining multiple read image portions by one or more second processors, and generating a combined image.

[0136] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: distortion correction on the combined image by one or more second processors, and to generate a corrected image.

[0137] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: rotational distortion correction by one or more second processors, in which the direction with less distortion is aligned to multiple lines for buffering.

[0138] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: rotating and correcting an image by one or more second processors; generating a rotated image; and sending the rotated image to a display panel for image display.

[0139] In some embodiments, computer-readable instructions may be executed by one or more processors to cause one or more processors to further perform: writing a corrected image to memory by one or more second processors; and reading the corrected image stored in memory. Optionally, rotating the corrected image may be performed during the reading of the corrected image stored in memory.

[0140] In some embodiments, one or more first processors include a system-on-a-chip, and one or more second processors include a field-programmable gate array.

[0141] All or some steps of the methods disclosed above, the functional modules / units in the system, and the devices can be implemented as software, firmware, hardware, or a suitable combination thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division between physical components. For example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable storage medium, which may include computer storage media (or non-transient media) and communication media (or transient media). The term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data, as is known to those skilled in the art. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Additionally, communication media are typically embodied as computer-readable instructions, data structures, program modules, or other data in modulated data signals, such as carrier waves or other transmission mechanisms, and include any information transmission medium as is known to those skilled in the art.

[0142] For illustrative and descriptive purposes, the foregoing description of embodiments of the invention has been provided. It is not exhaustive, nor is it intended to limit the invention to the precise forms or exemplary embodiments disclosed. Therefore, the foregoing description should be considered illustrative rather than restrictive. Clearly, many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described to explain the principles of the invention and its best mode of practical application, thereby enabling those skilled in the art to understand the various embodiments of the invention and the various modifications suitable for the particular use or implementation contemplated. The scope of the invention is intended to be defined by the appended claims and their equivalents, wherein, unless otherwise stated, all terms are to be interpreted in their broadest reasonable sense. Therefore, the terms “the invention,” “the present invention,” etc., do not necessarily limit the scope of the claims to the specific embodiments, and references to exemplary embodiments of the invention do not imply limitation of the invention, nor should such limitation be inferred. The invention is defined only by the spirit and scope of the appended claims. Furthermore, these claims may involve the use of “first,” “second,” etc., followed by nouns or elements. These terms should be understood as nomenclature and should not be construed as limiting the number of elements modified by these nomenclatures unless a specific number has been given. Any advantages and benefits described may not apply to all embodiments of the invention. It should be understood that changes to the described embodiments can be made by those skilled in the art without departing from the scope of the invention as defined by the appended claims. Furthermore, the elements and components in this disclosure are not intended for public distribution, whether or not they are expressly recited in the appended claims.

Claims

1. A drive system, comprising one or more first processors and one or more second processors; in, The one or more first processors are configured to receive an image and to send the image to the one or more second processors; In the first display mode, the one or more second processors are configured to perform a first distortion correction on the image; In the second display mode, the one or more second processors are configured to perform a second distortion correction on the image; The first display mode is different from the second display mode; as well as The first distortion correction is different from the second distortion correction.

2. The drive system according to claim 1, wherein, The first display mode is a partitioned display mode, in which the driving system is configured to drive the display panel to display different scenes in different areas of the display panel; the different areas include the instrument panel display area, the main console display area, and the auxiliary display area; as well as The second display mode is a full-screen display mode, in which the driving system is configured to drive the display panel to display a single scene substantially across the entire display panel.

3. The drive system according to claim 1, wherein, The one or more second processors are further configured to divide the image into a plurality of image portions, including at least the first image portion.

4. The drive system according to claim 3, wherein, In the first display mode, the one or more second processors are further configured to rescale at least the first image portion to a first rescaled image portion; Rescaling is performed at least partially along the first direction; Stretch the first image portion along the first direction; as well as The plurality of image portions are arranged along a second direction.

5. The drive system according to claim 4, wherein, The one or more second processors are also configured to add black patches to the image.

6. The drive system according to claim 4, wherein, The rescaling compression factor is as follows: Among them, V factor R1 represents the compression factor; R2 represents the resolution of the image along the first direction; and d2 represents the maximum distortion applied to the first image portion to form the first rescaled image portion.

7. The drive system according to claim 4, wherein, The one or more second processors are also configured to write the first rescaled image portion and other image portions into memory; as well as The first rescaled image portion and the other image portions are stored independently in the memory; The one or more second processors are further configured to read an image with a first resolution stored in the memory, and to convert the image with the first resolution stored in the memory into an image with a second resolution; The image stored in the memory, having the first resolution, includes the first rescaled image portion and the other image portions; and The image having the second resolution includes a plurality of read image portions, wherein the plurality of read image portions are arranged in a manner different from the arrangement of the first rescaled image portion and the other image portions in the image having the first resolution.

8. The drive system according to claim 7, wherein, The one or more second processors are further configured to perform distortion correction on the plurality of read image portions read from the memory and generate a plurality of corrected image portions.

9. The drive system according to claim 8, wherein, The one or more second processors are further configured to perform the distortion correction based on multiple sets of correction parameters.

10. The drive system according to claim 8, wherein, The one or more second processors are further configured to combine the plurality of corrected image portions to generate a combined image, and to send the combined image to a display panel for image display.

11. The drive system according to claim 3, wherein, In the second display mode, the one or more second processors are further configured to write the plurality of image portions into memory; and The plurality of image portions are stored independently in the memory; The one or more second processors are further configured to read an image with a first resolution stored in the memory, and to convert the image with the first resolution stored in the memory into an image with a second resolution; The image stored in the memory, having the first resolution, includes the plurality of image portions; and The image having the second resolution includes a plurality of read image portions, wherein the plurality of read image portions are arranged in a manner different from the arrangement of the plurality of image portions in the image having the first resolution.

12. The drive system according to claim 11, wherein, The one or more second processors are further configured to combine the plurality of read image portions to generate a combined image.

13. The drive system according to claim 12, wherein, The one or more second processors are further configured to perform distortion correction on the combined image and generate a corrected image.

14. The drive system according to claim 13, wherein, The one or more second processors are also configured to perform rotational distortion correction, in which directions with less distortion are aligned to multiple rows for buffering.

15. The drive system according to claim 13, wherein, The one or more second processors are further configured to: Rotate the corrected image; Generate rotated images; and The rotated image is sent to the display panel for image display.

16. The drive system according to claim 15, wherein, The one or more second processors are further configured to: Write the corrected image to memory; and Read the corrected image stored in the memory; During the reading of the corrected image stored in the memory, the corrected image is rotated.

17. The drive system according to any one of claims 1 to 16, wherein, The one or more first processors include a system-on-a-chip, and the one or more second processors include a field-programmable gate array.

18. A display device comprising a driving system according to any one of claims 1 to 17, and a display panel configured to receive a calibrated image from the driving system.

19. A distortion correction method, comprising: Images are received via one or more first processors; The image is sent to one or more second processors via the one or more first processors; In the first display mode, the image undergoes a first distortion correction via the one or more second processors; as well as In the second display mode, the image undergoes a second distortion correction via the one or more second processors; Wherein, the first display mode is different from the second display mode; and The first distortion correction is different from the second distortion correction.

20. A computer program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform: Images are received via one or more first processors; The image is sent to one or more second processors via the one or more first processors; In the first display mode, a first distortion correction is performed on the image by the one or more second processors; and In the second display mode, the image undergoes a second distortion correction via the one or more second processors; in, The first display mode is different from the second display mode; and The first distortion correction is different from the second distortion correction.

Images