System and method for simulating vehicle display

Abstract

A computer-implemented method is performed by data processing hardware of a display calibration system in communication with a camera system of a motor vehicle having a vehicle display. The method includes calculating a first gamma value for a vehicle display based on at least one display specification or vehicle statistics. Further, the method includes receiving a display lookup table, the display lookup table being modified based on the calculated first gamma value to generate a transform lookup table. Image data from the camera system is received and the transform lookup table is applied to the image data to generate transformed image data. Simulated image data is generated based on the transformed image data, and the simulated image data is presented at a display of the display calibration system. The presented display corresponds to a vehicle display.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for simulating a vehicle display, such as a vehicle camera display screen for an infotainment system. Background Art

[0002] The information provided in this section is for the purpose of generally presenting the background of the present disclosure. To the extent described in this section, the work of the presently named inventors, as well as aspects of the description that may not qualify as prior art at the time of filing, are neither explicitly nor implicitly admitted to be prior art with respect to the present disclosure.

[0003] Modern vehicles typically include one or more camera systems equipped on the exterior of the vehicle to assist the driver in viewing the environment surrounding the vehicle. The vehicle camera system generates image data and transmits such image data to one or more displays or graphical user interfaces (e.g., an infotainment system) contained within the passenger compartment of the vehicle. Vehicle displays are typically unique to a particular vehicle design and / or manufacturer. Vehicle display image parameters may differ from image parameters of conventional displays, such as laptop or desktop monitors. Therefore, vehicle displays typically must be uniquely calibrated or tuned to accurately depict image data generated by the vehicle camera system. Calibrating a vehicle display may include an iterative process of adjusting the camera system image parameters and then viewing the image output on the vehicle display to verify that the displayed image meets the desired image quality threshold. This calibration process may require multiple iterations of adjustment and verification. Summary of the invention

[0004] One aspect of the present disclosure provides a computer-implemented method that, when executed by data processing hardware of a display calibration system in communication with a camera system of a motor vehicle having a vehicle display, causes the data processing hardware to perform operations including calculating a first gamma value for a vehicle display based on at least one of display specifications or vehicle statistics, receiving a display lookup table, modifying the display lookup table based on the calculated first gamma value to generate a transformed lookup table, receiving image data from the camera system, applying the transformed lookup table to the image data to generate transformed image data, generating simulated image data based on the transformed image data, and presenting the simulated image data at a display of the calibration system, whereby the presented display corresponds to the vehicle display.

[0005] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, the vehicle statistics include at least one of an effective display duration and a vehicle life cycle value, and the display specifications include a display white point drop percentage lookup table. In some examples, operations include receiving a display gamma lookup table and a display white point lookup table for a vehicle display. In some configurations, modifying the display lookup table includes modifying at least one of the display gamma lookup table and the display white point lookup table based on the calculated first gamma value.

[0006] In some additional even further embodiments, the operations include modifying a camera lookup table based on the modified display lookup table. In some examples, the operations include calculating pixel values ​​of the image data and separating the image data into a red layer, a blue layer, and a green layer before generating the simulated image data. In some configurations, the operations include concatenating the red layer, the blue layer, and the green layer to generate the simulated image data after generating the transformed image data.

[0007] Another aspect of the present disclosure provides a calibration system for verifying image data received from a camera system of a vehicle for presentation on a vehicle display. The calibration system includes a display, data processing hardware, and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include calculating a first gamma value for a vehicle display based on at least one of display specifications or vehicle statistics, receiving a display lookup table, modifying the display lookup table based on the calculated first gamma value to generate a transform lookup table, receiving image data from a camera system, applying the transform lookup table to the image data to generate transformed image data, generating simulated image data based on the transformed image data, and presenting the simulated image data at a display of the calibration system. The presented display corresponds to a vehicle display.

[0008] This aspect of the disclosure may include one or more of the following optional features. In some embodiments, the vehicle statistics include at least one of an effective display duration and a vehicle life cycle value. The display specifications include a display white point drop percentage lookup table. In some examples, the operation includes receiving a display gamma lookup table and a display white point lookup table for a vehicle display. In some configurations, modifying the display lookup table includes modifying at least one of the display gamma lookup table and the display white point lookup table based on the calculated first gamma value.

[0009] In some embodiments, the operations include modifying a camera lookup table based on the modified display lookup table. In some examples, the operations include calculating pixel values ​​of the image data and separating the image data into a red layer, a blue layer, and a green layer before generating the transformed image data. In some configurations, the operations include stitching the red layer, the blue layer, and the green layer to generate the simulated image data after generating the transformed image data.

[0010] Yet another aspect of the present disclosure provides a user device for verifying image data received from a camera system of a vehicle for presentation on a vehicle display. The user device includes a display, data processing hardware in communication with the display, and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. These operations include calculating a first gamma value for a vehicle display based on at least one of a display specification or vehicle statistics, receiving a display lookup table, modifying the display lookup table based on the calculated first gamma value to generate a transform lookup table, applying the transform lookup table to image data to generate transformed image data, generating simulated image data based on the transformed image data, and presenting the simulated image data at a display of the user device. The presented display corresponds to a vehicle display.

[0011] This aspect of the disclosure may include one or more of the following optional features. In some embodiments, the vehicle statistics include at least one of an effective display duration and a vehicle life cycle value, and the display specifications include a display white point drop percentage lookup table. In some examples, the operations include receiving a display gamma lookup table and a display white point lookup table for a vehicle display. In some configurations, modifying the display lookup table includes modifying at least one of the display gamma lookup table and the display white point lookup table based on the calculated first gamma value.

[0012] In some implementations, the operations include modifying a camera lookup table based on the modified display lookup table. In some examples, the operations include calculating pixel values ​​of the image data and separating the image data into a red layer, a blue layer, and a green layer before generating the transformed image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are for illustrative purposes only of selected configurations and are not intended to limit the scope of the present disclosure.

[0014] Figure 1 is a schematic diagram of an example of a vehicle calibration environment according to the principles of the present disclosure; and

[0015] Figure 2 is an example sequence of operations performed by a vehicle calibration environment to calibrate a vehicle display.

[0016] Corresponding reference characters indicate corresponding parts throughout the several views. DETAILED DESCRIPTION

[0017] Example configurations will now be described more fully with reference to the accompanying drawings. Example configurations are provided so that the present disclosure will be thorough and will fully convey the scope of the present disclosure to those of ordinary skill in the art. Specific details, such as examples of specific components, devices, and methods, are set forth to provide a thorough understanding of the configurations of the present disclosure. It will be apparent to those of ordinary skill in the art that specific details need not be employed, that example configurations may be implemented in many different forms, and that the specific details and exemplary configurations should not be construed as limiting the scope of the present disclosure.

[0018] The terms used herein are only for describing specific exemplary configurations, not for limiting. As used herein, the singular articles "a", "an" and "the" may also be intended to include plural forms, unless the context clearly indicates otherwise. The terms "comprise", "include", "contain" and "have" are inclusive, thus specifying the presence of features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts and / or groups thereof. The method steps, processes and operations described herein should not be interpreted as necessarily requiring them to be performed in the specific order discussed or shown, unless specifically identified as an execution order. Additional or alternative steps may be adopted.

[0019] When an element or layer is referred to as being "on another element or layer," "engaged to," "connected to," "attached to," or "coupled to" another element or layer, it may be directly on, engaged, connected, attached to, or coupled to another element or layer, or there may be intermediate elements or layers. Conversely, when an element is referred to as being "directly on another element or layer," "directly engaged to," "directly connected to," "directly attached to," or "directly coupled to" another element or layer, there may be no intermediate elements or layers. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.). As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0020] The terms first, second, third, etc. can be used here to describe various elements, components, regions, layers and / or parts. These elements, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish each element, component, region, layer or part. Terms such as "first", "second" and other numerical terms do not imply order or sequence unless the context clearly indicates. Therefore, the first element, component, region, layer or part discussed below can be referred to as the second element, component, region, layer or part, without departing from the teaching of the example configuration.

[0021] In this application, including the definitions below, the term module may be replaced by the term circuit. The term "module" may refer to or be part of an application specific integrated circuit (ASIC), or include an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; a memory (shared, dedicated, or group) that stores code executed by the processor; other suitable hardware components that provide the functionality; or a combination of some or all of the above, such as in a system on a chip.

[0022] The term code used above may include software, firmware and / or microcode, and may refer to a program, a routine, a function, a class and / or an object. The term shared processor includes a single processor that executes some or all of the code from multiple modules. The term group processor includes a processor that executes some or all of the code from one or more modules in combination with an additional processor. The term shared memory includes a single memory that stores some or all of the code from multiple modules. The term group memory includes a memory that stores some or all of the code from one or more modules in combination with an additional memory. The term memory may be a subset of the term computer-readable medium. The term computer-readable medium does not include transient electrical signals and electromagnetic signals propagated through the medium and can therefore be considered to be tangible non-transitory memory. Non-limiting examples of non-transitory memory include tangible computer-readable media, including non-volatile memory, magnetic memory, and optical memory.

[0023] The apparatus and methods described in this application may be implemented in part or in whole by one or more computer programs executed by one or more processors. The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include and / or rely on stored data.

[0024] Software applications (i.e., software resources) may refer to computer software that enables a computing device to perform tasks. In some examples, software applications may be referred to as "applications," "apps," or "programs." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0025] Non-transitory memory can be a physical device used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program state information) for use by a computing device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0026] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0027] Various implementations of the systems and techniques described herein can be realized in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system comprising at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from a storage system, and to send data and instructions to the storage system.

[0028] The processes and logic flows described in this specification can be performed by one or more programmable processors, also known as data processing hardware, executing one or more computer programs to perform functions by operating on input data and generating outputs. These processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include or be operably connected to one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from the large-capacity storage device or to transmit data to it, or both. However, a computer does not need to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated in special purpose logic circuitry.

[0029] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen, and an optional keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0030] refer to Figure 1, an example display calibration environment 100 is provided according to the present disclosure. The display calibration environment 100 is configured for calibrating the display of image data generated by one or more cameras 14 of a vehicle camera system 12 and transmitted to a graphical user interface or vehicle display 16 of the vehicle 10. In the illustrated example, the display calibration environment 100 is schematically represented and includes a calibration device 110, which includes a controller 112 (e.g., data processing hardware), memory hardware 114, and a display 116. In some examples, the calibration device 110 may include a computer system, such as a laptop. In other examples, the display calibration system 120 may be at least partially hosted on a cloud computing system and accessed over a network.

[0031] As discussed in more detail below, the display calibration environment 100 includes a display calibration system 120 executed or hosted on the calibration device 110. The display calibration system 120 includes a plurality of calibration modules 122, 122a-122f, each of which is operable to perform one or more operations of the display calibration system 120. Figure 1 As shown, the calibration module 122 includes a lifetime gamma calculator 122a configured to calculate a gamma value of the vehicle display 16 based on a lifetime value of the vehicle 10 or the vehicle display 16, a display lookup table (LUT) modifier 122b, a camera LUT modifier 122c, an image separation module 122d, an image transformation module 122e, and an image stitching module 122f—the operation of each of which will be described in more detail below. The calibration device 110 may further include or communicate with one or more storage devices 130, 140, which include display specifications 130 and vehicle statistics 140.

[0032] refer to Figure 2 , provides an example of the operation of the display calibration environment 100. Although the operation of the display calibration environment 100 is described sequentially herein, it should be understood that the various operations may be performed in parallel or in an order different from that described herein.

[0033] At initial operation 202, the lifecycle gamma calculator 122a obtains each of the vehicle lifecycle value 220, the effective display duration value 222, and the white point drop percentage lookup table (LUT) 224 for calculating the gamma value of the display. For clarity, white point drop refers to the total brightness of the vehicle display 16 when all display pixels are set to display white. In this example, the vehicle lifecycle value 220 can be a total number of ignition cycles associated with the vehicle 10, which can be obtained from the vehicle statistics memory 140 of the vehicle 10. In other examples, the vehicle lifecycle value 220 can include a duration associated with vehicle operation, such as an engine running duration. The effective display duration value 222 includes a duration corresponding to the total duration that the vehicle display 16 has been effective (ie, powered on). At operation 202, using the vehicle lifecycle value 220, the effective display duration value 222, and the white point drop percentage LUT 226, the lifecycle gamma calculator 122a calculates the gamma value of the vehicle display 16. In other words, the gamma value of the vehicle display 16 can depend on the age or use of the vehicle display 16, and can change based on age or use. By calculating the current gamma value of the vehicle display 16 , the vehicle display system 120 provides an accurate baseline for generating the simulated display 116 of the image data 15 at the calibration device 110 .

[0034] In another operation 204, the display calibration system 120 obtains various display parameter values ​​228, 230 from the display specification memory 130 and modifies one or more lookup tables (LUTs) associated with the vehicle display 16. In some cases, the display specification memory 130 may be stored locally in the display calibration environment 100 (e.g., in the calibration device 110). In other examples, the display specification memory 130 may be integrated into the vehicle display 16 and obtained therefrom. For example, the vehicle display 16 may be provided with memory hardware that is pre-configured with display specifications by the vehicle display manufacturer. In operation 204, the display calibration system 120 obtains the white point LUT 228 and the gamma LUT 230 of the vehicle display 16. The white point LUT 228 and the gamma LUT 230 are pre-defined specifications that depend on the specific vehicle display 16. For example, a first model of vehicle display 16 may have a first white point LUT 228 and a first gamma LUT 230, while a different model of vehicle display 16 may have a different second white point LUT 228 and a second gamma LUT 230. The display LUT modifier 122b evaluates the white point LUT 228 and the gamma LUT 230 and the calculated display gamma 203 to generate one or more modified display LUTs 205. In some examples, the modified display LUT 205 may include a piecewise gamma transform LUT 205.

[0035] In operation 206, the display calibration system 120 receives the modified display LUT 205 and generates a modified camera LUT 207 based on the modified display LUT 205. For example, the camera LUT modifier 122c obtains the predefined camera LUT 18 based on the specifications of the camera 14, which may be provided by the camera manufacturer. In operation 208, the display calibration system 120 obtains image data 15 from one or more vehicle cameras 14. The image data 15 includes specifications related to the camera gamma LUT 232.

[0036] In operation 210, the image separation module 122d of the display calibration system 120 receives the acquired image data 15 and performs the step of determining the individual pixel values ​​of the image data 15. The image separation module 122d further separates the image data 15 into a red (R) layer, a green (G) layer, and a blue (B) layer, thereby creating separated image data 211. The separated image data 211 including the pixel values ​​and the R, G, and B layers is forwarded to the transformation module 122e, which performs a piecewise linear transformation using the calculated piecewise gamma transformation LUT 205 provided in operation 204. When transforming the separated image data 211, the transformation module 122e transmits the transformed image data 213 to the image stitching module 122f, which recombines the transformed R, G, and B layers in operation 214 to generate simulated image data 215 associated with the image data 15. The simulated image data 215 is then presented to the display 116 of the calibration device 110 in operation 216.

[0037] In addition to integrated camera and display gamma and real-time integration of black point, white point, and contrast, a target image for any display can be simulated on an auxiliary display with different display specifications. This approach provides an accurate simulation of any target display image, enabling precise control of image contrast, brightness, and color reproduction in the absence of physical hardware (such as a vehicle display). The method also provides a comprehensive solution for simulating any target display image, providing an effective means to precisely adjust image quality at the system level before implementation into the vehicle 10. Therefore, by simulating the image presentation of the vehicle display 16 via the display 116 of the calibration device 110, engineers can calibrate and verify image quality at a remote location independent of the vehicle 10. On a wide variety of vehicles 10 and vehicle display 16 types, the display calibration system 120 provides a single interface for calibrating a wide variety of displays without the need to visit each vehicle 10, thereby significantly reducing the time associated with verification and vehicle integration.

[0038] Many embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the present disclosure. Therefore, other embodiments are also within the scope of the following claims.

[0039] The foregoing description has been provided for the purpose of illustration and description. It is not intended to be exhaustive or to limit the present disclosure. Individual elements or features of a particular configuration are generally not limited to that particular configuration, but where applicable, they are interchangeable and can be used in a selected configuration even if not specifically shown or described. This can also be varied in a variety of ways. Such variations should not be considered a departure from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

Claims

1. A computer-implemented method, when performed by data processing hardware of a display calibration system in communication with a camera system of a motor vehicle having a vehicle display, the method causing the data processing hardware to perform operations comprising: calculating a first gamma value for a vehicle display based on at least one of display specifications or vehicle statistics; receiving a display lookup table; modifying a display lookup table based on the calculated first gamma value to generate a transform lookup table; receiving image data from a camera system; applying the transformed lookup table to the image data to generate transformed image data; generating simulated image data based on the transformed image data; as well as The simulated image data are presented at a display of the calibration system, whereby the presented display corresponds to the vehicle display.

2. The method according to claim 1, wherein: The vehicle statistics include at least one of an effective display duration and a vehicle life cycle value, and the display specifications include a display white point drop percentage lookup table.

3. The method according to claim 2, wherein: The operations also include receiving a display gamma lookup table and a display white point lookup table for the vehicle display.

4. The method according to claim 3, wherein: Modifying the display lookup table includes modifying at least one of the display gamma lookup table and a display white point lookup table based on the calculated first gamma value.

5. The method according to claim 4, wherein: The operations also include modifying a camera lookup table based on the modified display lookup table.

6. The method according to claim 5, wherein: The operation also includes, before generating the analog image data, calculating pixel values ​​of the image data and separating the image data into a red layer, a blue layer, and a green layer.

7. The method according to claim 6, wherein: The operation also includes, after generating the transformed image data, splicing the red layer, the blue layer, and the green layer to generate simulated image data.

8. A calibration system for verifying image data received from a camera system of a vehicle for presentation on a vehicle display, the calibration system comprising: monitor, Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including: calculating a first gamma value for a vehicle display based on at least one of display specifications or vehicle statistics; receiving a display lookup table; modifying a display lookup table based on the calculated first gamma value to generate a transform lookup table; receiving image data from a camera system; applying the transformed lookup table to the image data to generate transformed image data; generating simulated image data based on the transformed image data; as well as The simulated image data are presented at a display of the calibration system, whereby the presented display corresponds to the vehicle display.

9. The system according to claim 8, wherein: The vehicle statistics include at least one of an effective display duration and a vehicle life cycle value, and the display specifications include a display white point drop percentage lookup table.

10. A user device for verifying image data received from a camera system of a vehicle for presentation on a vehicle display, the user device comprising: monitor, data processing hardware that communicates with the display; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including: calculating a first gamma value for a vehicle display based on at least one of display specifications or vehicle statistics; receiving a display lookup table; modifying a display lookup table based on the calculated first gamma value to generate a transform lookup table; applying the transformed lookup table to the image data to generate transformed image data; generating simulated image data based on the transformed image data; as well as The simulated image data is presented at a display of the user device, whereby the presented display corresponds to the vehicle display.

Images