Method for transmitting an image data stream

By separating brightness and color information into distinct partial images with appropriate bit widths, the method ensures efficient data transmission and processing for ADAS and AD applications, addressing the inefficiencies of existing camera interface standards.

DE102019219142B4Active Publication Date: 2024-11-14ZF FRIEDRICHSHAFEN AG
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
DE102019219142
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Filing Date
2019-12-09
Publication Date
2024-11-14
Estimated Expiration
2039-12-09

AI Technical Summary

Technical Problem

Existing camera interface standards, such as CSI-2, do not efficiently support image data formats required for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD) applications, leading to inefficient processing and storage of image data due to differing data format requirements for brightness and color information.

Method used

The method involves preparing image data streams from ADAS and AD cameras to separate brightness and color information into distinct partial images, using bit widths suitable for CSI-2 interface, such as YUV formats with 12 bits for luminance and 8 bits for chrominance, and transmitting these separately to ensure compatibility and efficient processing.

Benefits of technology

This approach allows for rapid and efficient processing of image data by image processing devices, minimizing memory requirements and buffer sizes, while maintaining compliance with standard interfaces.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Method for transferring an image data stream (4) from a camera (1) to an image processing device (2), - wherein the camera (1) provides the image data stream (4) as camera output data, - wherein the image processing device (2) uses a standard interface, - wherein the method includes a data preparation process (S1) during which the camera output data is prepared such that the image data stream (4) after the data preparation process (S1) is compatible with the standard interface, characterized by the fact that the image data stream (4) is transformed during the data preparation process (S1) into a form in which image data of the image data stream (4) are separated according to brightness information and color information, where the standard interface is a CSI, CSI-2 or CSI-3 interface.
Need to check novelty before this filing date? Find Prior Art

Description

Technical field

[0001] The invention relates to a method for transmitting an image data stream according to the preamble of claim 1. The invention further relates to a system for transmitting and / or processing an image data stream, a vehicle, a computer program and a computer-readable medium according to the dependent claims. State of the art

[0002] Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD) systems typically use cameras that transmit their images via high-speed interfaces. Transmitter and receiver modules for these high-speed interfaces often use the CSI-2 camera interface as a standard interface at either the input or output.

[0003] Today's commonly used camera interface standards, such as CSI-2, primarily support image data formats designed for display and viewing. However, cameras for ADAS and AD typically produce images not intended for display or viewing, but rather for analysis by algorithms. The data format requirements for these images differ significantly from what is currently supported by typical camera interface standards like CSI-2. ADAS and AD cameras are expected to output high contrast across a wide dynamic range. Color information is also required, for example, to recognize the colors of traffic signs. However, the color information can be stored at a significantly lower resolution than the contrast information.Very often, brightness and color of images are also evaluated separately by image processing algorithms in ADAS and AD applications.

[0004] Accordingly, it is advisable to use data formats for ADAS and AD applications in which brightness (luminance) and color components (chrominance) are stored separately. One possible such image data format is the YUV format, in which luminance is stored in the Y value and chrominance is stored in the U and V values.

[0005] Common camera interface standards, such as CSI-2, only support YUV formats with 8-bit and 10-bit luminance and chrominance. While unsupported data formats can be packed into so-called "user-defined" data formats and thus transmitted in a standards-compliant manner, the problem is that this approach can lead to data arriving at the video inputs of image processing processors that cannot be efficiently stored and processed. This is because, for example, the incoming image data arrives fragmented at the image processing processors. Image processing processors typically offer various image preprocessing functions or so-called DMAs (Direct Memory Access) to process image data and efficiently transfer it to the image processing processor's memory.These functions can only be used if the data arriving at the image processing processors conforms to the supported standards. For example, if YUV data is packed into user-defined formats, this data cannot necessarily be processed efficiently by the video inputs of the image processing processors, nor can it necessarily be easily transferred to memory in a suitable structure.

[0006] US Patent 5,568,192 A discloses a system and method for processing video signals. According to a preferred embodiment, a plurality of video signals are recorded by a camera in response to an image. The plurality of video signals are converted by the camera into a plurality of raw digital signals. The plurality of raw digital signals are converted by a computer into a plurality of usable digital video signals, and the plurality of raw digital signals are transmitted to the computer via a cable.

[0007] US Patent 2019 / 0116347 A1 discloses digital camera systems and methods that provide direct luminance detection to a digital color camera. The luminance signals are obtained directly from a broadband image sensor channel without interpolation of RGB data. The chrominance signals are obtained from one or more additional image sensor channels capable of detecting a red and / or blue color spectrum. The red and blue signals are directly combined with the luminance image sensor channel signals. The digital camera generates an image in the YCrCb color space by directly combining the outputs of the broadband, red, and blue sensors and outputs this image.

[0008] US Patent 2017 / 0334367 A1 discloses a vision system for a vehicle, comprising a camera mounted on the vehicle with a field of view directed outwards and in front of the vehicle. The camera includes an image sensor and a lens. The image sensor captures image data. The camera includes an image sensor interface mounted on a circuit board compatible with 8-bit MIPI-CSI2 and / or 12-bit parallel. The image sensor interface is connected to an image processor that processes the image data captured by the image sensor.

[0009] With the current state of the art, it is therefore problematic, among other things, to transfer image data for ADAS applications and AD applications in such a way that certain functions of image processing processors can be used efficiently. General description of the invention

[0010] The object of the invention is to eliminate or at least reduce the disadvantages of the prior art.

[0011] The problem is solved by a method for transmitting an image data stream from a camera to an image processing device, wherein the camera provides the image data stream as camera output data, the image processing device uses a standard interface, and the method includes a data preparation process during which the camera output data is prepared such that the image data stream is compatible with the standard interface after the data preparation process. During the data preparation process, the image data stream is converted into a form in which the image data of the image data stream is separated according to brightness information and color information. According to the invention, the standard interface is a CSI, CSI-2, or CSI-3 interface.

[0012] The camera is typically a camera for an ADAS application and / or an AD application. The image processing device is typically an image processing processor or a control unit, or more generally, a technical system capable of processing digital images. An image data stream is generally understood to be a data stream comprising a set of bits, which can be organized in different data formats, with typically several bits encoding one or more pixels of a specific image. It is important to note that the image data stream, as used here, can change its form. For example, the data formats in which the individual bits of the image data stream are organized can be reorganized or repackaged into other data formats during the process.Camera output data refers to data in specific data formats that the camera or camera head outputs. A standard interface is generally understood to be an interface that meets the requirements of a specific information technology standard. CSI-2 is an example of such a standard. A data preparation process is understood to be a process that is executed at some point in the procedure, typically comprising several steps, during which reformatting and / or reorganization of the image data stream can and / or actually occurs. The term "compatible with the standard interface" typically means that the image data stream contains image data that exclusively has data formats compatible with the standard interface.In typical designs, the image data stream after the data processing consists exclusively of image data, which has data formats that are provided for in the standard interface.

[0013] The invention solves the problem by ensuring that, on the one hand, only data formats compatible with the image processing device's standard interface are transmitted to the device, and on the other hand, that an image data organization particularly useful for ADAS and AD applications is achieved, namely a separation of brightness information on the one hand and color information on the other. This ensures particularly efficient and fast processing of the image data stream in the image processing device.

[0014] In typical embodiments, the image data has bit widths other than 8 bits or 10 bits, in particular, for example, 12 bits, and / or is transmitted with bit widths other than 8 bits or 10 bits, in particular, for example, 12 bits. In typical embodiments, the brightness information has different bit widths than the color information and / or the brightness information is transmitted with different bit widths than the color information.

[0015] Using a high-speed interface, preferably a CSI-2 interface, as the standard interface has the advantage of ensuring fast and efficient data transmission and processing.

[0016] In typical implementations, the image data stream is processed in such a way that, after processing, three sub-images are available for each image in the stream, specifically one luminance sub-image and two chrominance sub-images. The luminance sub-images are typically composed exclusively of image data containing brightness information. The chrominance sub-images are typically composed exclusively of image data containing color information. This division into three sub-images has the advantage that, on the one hand, a data structure specifically suited to ADAS and AD cameras is selected, and on the other hand, standards-compliant transmission of the image data stream is generally ensured.

[0017] In typical configurations, the camera output data is in YUV format, where typically the Y values ​​each have a length of 12 bits and / or the U values ​​typically have a length of 8 bits and / or the V values ​​typically have a length of 8 bits, and / or where typically luminance image data of the luminance sub-image has the data format "RAW12" and / or chrominance image data of the chrominance sub-images typically has the data format "RAW8" and / or "Userdef (8Bit)". The "Userdef (8Bit)" data format is an 8-bit data format provided in the CSI-2 standard for users of a CSI-2 interface to use as needed for various specialized tasks. It is, in effect, a kind of "wildcard" data format.Such an interplay between camera output data in YUV format, the lengths of the individual YUV values, and the original data formats of the CSI-2 standard is advantageous because it allows for the standardized transmission of data particularly well-suited to ADAS or AD applications. Alternatively, other bit lengths are also possible; for example, the Y values ​​can have lengths of 14, 16, 18, or 20 bits, and / or the U and / or V values ​​can have lengths of only 6 or 4 bits. Furthermore, the luminance image data can be in the RAW14, RAW16, or RAW20 format. Similarly, the chrominance image data can be in the RAW7 or RAW6 format, for example. In principle, all data formats of the CSI-2 standard can typically be used, as long as the separation of the original camera output data into luminance and color information is maintained.When using data formats with too many bits, it is possible to omit certain bits. The term "image data" generally refers to data that encodes and / or defines one or more pixels of an image or sub-image using a plurality of bits. In advantageous embodiments, during transmission and / or storage of the chrominance image data, this chrominance image data is stored either as separate sub-images or as a single, combined sub-image. In advantageous embodiments, the transmitted image consists of a luminance image and one or two chrominance images. In the case of only one chrominance image, the U-values ​​and V-values, which can also be referred to as color vectors, are transmitted in a single image. The respective color vectors are typically inserted into the image data stream in alternating sequence.

[0018] In typical embodiments, the luminance sub-images, in particular the Y values, and / or the chrominance sub-images, in particular the U values ​​and / or the V values, have bit widths other than 8 bits or 10 bits, in particular, for example, 12 bits, and / or are transmitted with bit widths other than 8 bits or 10 bits, in particular, for example, with 12 bits. In typical embodiments, the Y values ​​have bit widths other than the U values ​​and / or the V values, and / or the Y values ​​are transmitted with bit widths other than the U values ​​and / or the V values.

[0019] When this description refers to the length of a data format or a value in bits, it essentially means bit width. In other words, the two terms "length" and "bit width" are used synonymously.

[0020] In typical embodiments, the luminance sub-images of the image data stream are transmitted during image acquisition and / or the chrominance sub-images of the data stream are typically transmitted in a blanking interval, preferably a vertical blanking interval and / or a horizontal blanking interval. Typically, the image data of the chrominance sub-images are stored while the image is being acquired and the luminance sub-images are being transmitted. This approach has the advantage of minimizing storage requirements because the luminance sub-images typically have a larger bit depth and / or a higher resolution than the chrominance sub-images. In this way, the time spent acquiring the image is used to transmit the image data for the luminance sub-images, and subsequently, a typically available blanking interval is used to send the image data of the chrominance sub-images afterward.In typical embodiments, particularly in cases where the duration of a blanking interval does not match the time required for transmitting the chrominance sub-images, the blanking interval is adjusted with respect to its duration to the time required for transmitting the chrominance sub-images and / or the luminance sub-images are partially buffered.

[0021] In typical configurations, the data processing and / or transmission of the image data stream occurs line by line. Typically, during the acquisition of pixels in a line and / or multiple lines, an image data stream containing luminance sub-images is transmitted. Subsequently, during a horizontal blanking interval, an image data stream containing chrominance sub-images is transmitted. This line-by-line data processing and / or transmission has the advantage of minimizing the buffer size. Even with this line-by-line transmission, it is possible to adjust the horizontal blanking interval to the data being transmitted, as described previously. In other words, the image acquisition speed can be adapted to the amount of data being transferred.

[0022] In typical embodiments, the partial images are transferred sequentially to the image processing device, typically ordered according to data formats and / or typically according to a predefined, repeating sequence. Typically, blocks of luminance image data belonging to luminance partial images are transferred, followed by blocks of chrominance image data relating to chrominance partial images, with this sequence repeating in a predefined pattern. Such a transfer has the advantage that the image processing device can easily distinguish the different blocks according to chrominance and luminance information and store them in corresponding memory areas.

[0023] In typical configurations, the entire image data stream is transmitted over a single virtual channel. Unique identification of the image data, particularly regarding luminance and chrominance information, is typically achieved by using a specific data format for each type of image data. The image processing device can thus recognize the type of image information based on the data formats of the incoming image data. For example, when transmitting over a single virtual channel, all Y-values ​​are transmitted in the "RAW12" data format, all U-values ​​are transmitted in a first "Userdefined" format, and all V-values ​​are transmitted in a second "Userdefined" format. The two "Userdefined" formats for U-values ​​and V-values ​​are different and typically have different IDs.The terms «Userdefined», «Userdef (8 Bit)» and «Userdef» are used synonymously in this description and in the claims.

[0024] Alternatively, it is also possible to transmit the partial images not via a single virtual channel, but via different virtual channels. In this case, the image data arriving at the image processing device can be easily identified using the virtual channels. For example, all Y-values ​​in the data format "RAW12" are transmitted in a first virtual channel, all U-values ​​in the data format "RAW8" are transmitted in a second virtual channel, and all V-values ​​in the data format "RAW8" are transmitted in a third virtual channel.

[0025] However, the sequential transmission of the partial images in a single virtual channel has the advantage that the different virtual channels typically provided by the standard interfaces can be used for different cameras.

[0026] In advantageous embodiments, all luminance image data have the same data format, for example, "RAW12". In advantageous embodiments, all chrominance image data have the same data format, for example, "RAW8". In typical embodiments, all chrominance image data based on U-values ​​have a specific data format, for example, a specific "user-defined" format, and all chrominance image data based on V-values ​​also have a specific data format, for example, a specific "user-defined" format, wherein the two "user-defined" formats for the U-values ​​and V-values ​​are different "user-defined" formats and typically have different IDs.

[0027] In typical configurations, the data processing takes place on the camera side. In other words, the data processing is performed before the image data stream is sent to the image processing device. Alternatively, however, it is also possible for the data processing to take place on the receiver side. In the case of camera-side data processing, the data processing is typically performed either by the camera itself or by a logic module downstream of the camera. In the case of receiver-side data processing, for example, a logic module is placed upstream of the image processing device, which processes the image data of the image data stream so that standard-compliant data formats are provided to the image processing device.

[0028] Camera-side data processing typically occurs either directly within the image sensor or via a corresponding processing module located downstream of the image sensor. In the case of a downstream processing module, various interfaces and data formats can be used for data transmission to this module. Examples of interfaces include CSI2, parallel image sensor interfaces, and similar interfaces. Data formats can be any known format or sensor-specific formats. In typical configurations, the data formats include raw data format and / or RGB and / or YUV and / or similar formats. In typical configurations, the processing module is implemented as a specific integrated component. In typical configurations, the processing module is a configurable logic module, for example.an FPGA (Field Programmable Gate Array) or another integrated circuit into which a logic circuit can be loaded.

[0029] The problem is further solved by a system for transmitting and / or processing an image data stream, comprising means for at least partially carrying out a method according to at least one of the aforementioned embodiments.

[0030] In typical embodiments, the system is integrated into a camera. In typical embodiments, the system includes a logic module and / or is itself a logic module.

[0031] In advantageous embodiments, the system is suitable for at least partially performing, coordinating, and / or controlling a method for transmitting an image data stream from a camera to an image processing device according to at least one of the aforementioned embodiments. For this purpose, the system advantageously comprises suitable components, for example, a data processing component, a storage component, a line averaging component, and / or a blanking interval adjustment component. In typical embodiments, the system comprises one or more data acquisition devices and / or a plurality of communication interfaces for communication with one or more cameras, a vehicle, a vehicle control system, and / or the internet.

[0032] Advantageously, at least some of the aforementioned components are implemented in the system for transmitting and / or processing an image data stream using computer program code. In advantageous embodiments, the system is at least partially part of a vehicle control system and / or a cloud, and / or includes a digital control unit and / or a display and / or means for data input and / or means for data output.

[0033] In one embodiment of the invention, a vehicle is suitable for carrying out a method according to at least one of the aforementioned embodiments. For this purpose, the vehicle typically includes means for carrying out a method according to at least one of the aforementioned embodiments.

[0034] In one embodiment of the invention, a logic module comprises a fixed and / or a configurable logic implementation, wherein the logic module is suitable for implementing a method according to at least one of the aforementioned embodiments in logic.

[0035] In one embodiment of the invention, a computer program comprises instructions which, when executed by a computer, cause the computer to perform one of the aforementioned methods. The computer program can also be referred to as a computer program product.

[0036] In one embodiment of the invention, a computer-readable medium comprises computer program code for carrying out one of the aforementioned methods. The term "computer-readable medium" includes, in particular but not exclusively, hard drives and / or servers and / or memory sticks and / or flash memory and / or DVDs and / or Blu-rays and / or CDs. Additionally, the term "computer-readable medium" also includes a data stream, such as that generated when a computer program and / or a computer program product is downloaded from the internet. Short description of the drawings

[0037] The invention is briefly explained below with reference to drawings, which show: Fig. 1: A schematic representation of a method according to the invention in a first embodiment, Fig. 2: A schematic representation of a method according to the invention in a second embodiment, and Fig. 3: A schematic representation of a method according to the invention in a third embodiment. Description of preferred embodiments

[0038] Fig. Figure 1 shows a schematic representation of a method according to a first embodiment of the invention. Fig. Figure 1 shows a camera 1 and an image processing device 2. An image data stream 4 is transmitted from the camera 1 to the image processing device 2. The camera 1 initially provides the image data stream 4 in the form of camera output data in YUV format. This means that for each image, a plurality of Y values, a plurality of U values, and a plurality of V values ​​are output as camera output data. The Y values ​​are typically referred to as luminance information or brightness information, and the U values ​​and V values ​​are referred to as chrominance information or color information. The Y values ​​have a bit depth of 12 bits, and the U values ​​and V values ​​each have a bit depth of 8 bits.

[0039] The in Fig. The method shown comprises a data processing system 3, which performs a data processing process S1. During this data processing process 1, the camera output data, i.e., the YUV values, are converted into data formats defined in the CSI-2 standard. Within the data processing process S1, the Y-values ​​are converted to the RAW12 data format, the U-values ​​are converted to the RAW8 data format, and the V-values ​​are also converted to the RAW8 data format. Thus, after data processing S1, the image data stream 4 only contains data formats defined in the CSI-2 standard or compatible with this standard. This processed image data stream 4 is then fed to the image processing device 2.Image processing device 2 is thus provided exclusively with data formats compatible with the CSI-2 standard and can therefore process this image data efficiently, for example via Direct Memory Access (DMA). Furthermore, image processing device 2 knows that Y-values, i.e., luminance information, are stored in the RAW12 data format and that U-values ​​and V-values, as chrominance information, are contained in the RAW8 data formats. In this way, image processing device 2 can easily distinguish between image data with luminance information and image data with chrominance information, which simplifies the processing of the image data within the device.

[0040] Fig. Figure 2 shows a schematic representation of a further embodiment of the invention. The Fig. The embodiment shown in 2 is the one described in Fig. The implementation shown in Figure 1 is very similar. The only difference is that the U-values ​​and V-values ​​are not packed into the RAW8 format during the data preparation process S1. Instead, the U-values ​​are packed into a first user-defined format with a length of 8 bits, and the V-values ​​are packed into a second user-defined format, also with a length of 8 bits. The CSI-2 standard provides several such user-defined formats, which can be distinguished by their different IDs.In this way, the image processing device 2 is enabled to distinguish not only between luminance and chrominance information based on the data formats of the image data stream 4 arriving at the image processing device 2, but also between U-values ​​and V-values, since U-values ​​and V-values ​​can each be recognized in a special user-defined data format reserved for them, just as the Y-values ​​can be recognized in the RAW12 data format.

[0041] Fig. Figure 3 shows a schematic representation of a method according to a third embodiment of the invention. Similar to the method already described in Figure 3. Fig. In the embodiment shown in 1, the following applies: Fig. Figure 3 shows the transmission of an image data stream 4 from a camera 1 to an image processing device 2 in a data preparation process S1. The Y-values ​​are stored in RAW12 format, and the U-values ​​and V-values ​​are each stored in RAW8 format. However, the transmission to the image processing device 2 does not subsequently take place in a single virtual channel (as shown in the Fig. 1 and Fig. 2), but in three virtual channels. In this way, the image processing device 2 is enabled to distinguish between Y-values, U-values, and V-values ​​based on the assignment of the received image data to the different channels, because Y-values ​​in RAW12 format are transmitted in the upper channel, U-values ​​in RAW8 format are transmitted in the middle channel, and V-values ​​in RAW8 format are transmitted in the lower channel (where the designations "upper channel", "middle channel", and "lower channel" refer only to the graphical representation in Fig. 3). In particular, it will be used in the case of the Fig. The embodiment shown in Figure 3 enables the image processing device to distinguish between received image data in RAW8 format according to U-values ​​and V-values, because a separate virtual channel is used for each. A disadvantage of the embodiment shown in Figure 3 is that... Fig. However, the difference in the embodiment shown in 3 is that three virtual channels of the CSI-2 interface are required, whereas in the embodiment shown in Fig. 1 and Fig. In the two illustrated examples, only one virtual channel is required in each case. Fig. It should also be noted that the three virtual channels shown are created by time-division multiplexing, meaning that there are not actually three parallel data transmission paths.

[0042] Data processing system 3 is in Fig. 1 to 3 are shown as individual components, but in typical embodiments, they can also be part of camera 1. In all the Fig. In the embodiments shown in Figures 1 to 3, the image information is typically acquired in parallel by camera 1 during tape recording. Therefore, in typical embodiments, image information, such as U-values ​​and / or V-values, is temporarily stored. In typical embodiments, each image recorded by camera 1 is transmitted in three partial images: a luminance image and two chrominance images. The image data for the luminance image is typically supplied by camera 1 as a 12-bit Y-value, and the image data for the chrominance images are supplied by camera 1 as an 8-bit U-value and an 8-bit V-value. The luminance image of each image is typically transmitted during image acquisition. The two chrominance images are temporarily stored and transmitted after the luminance image has been transmitted, preferably during a blanking interval.

[0043] Luminance images, which, as explained, are typically encoded by 12-bit Y-values, typically have a higher resolution and bit depth than chrominance images, which typically have only half or a quarter of the resolution of the luminance images. The bit depth for U-values ​​and V-values ​​is also lower than for Y-values ​​in ADAS and AD cameras. Therefore, the storage and transmission time for U-values ​​and V-values ​​are significantly lower than for Y-values. To minimize the required camera storage, the Y-value components are typically transmitted directly during tape recording. The U-values ​​and V-values ​​are preferably transmitted in the vertical blanking interval. If necessary, the image acquisition is adjusted accordingly, for example, so that the length of the vertical blanking interval is sufficient for transmitting the U-values ​​and V-values.Alternatively, or in combination with this, at least partial buffering of the Y-value can also be performed. If a buffer is designed as a first-in-first-out (FIFO) buffer, possibly with different input and output data rates, typically only portions of the Y-value are dynamically held in memory. This makes it possible to keep the size of the buffer correspondingly small.

[0044] In typical embodiments, the buffering on the camera 1 side is reduced by separating the image data line by line during transmission from the camera 1 to the image processing device 2. CSI-2 allows such line-by-line transmission. This eliminates the need to buffer entire images and then transmit them in the vertical blanking interval; instead, only one or more lines are stored at a time, and the corresponding U-values ​​and V-values ​​are then transmitted in the respective horizontal blanking intervals. Typically, the Y-values ​​continue to be transmitted directly by the camera 1 during image data recording. In typical embodiments, the horizontal blanking interval is adjusted in time, specifically to ensure that it is long enough for the transmission of the U-values ​​and V-values.

[0045] Even in the line-by-line transmission of image data, dynamically determined Y values ​​are temporarily stored in typical designs to optimize the output timing.

[0046] In advantageous embodiments, after each line-wise transmission of Y-values, either a line of U-values ​​or a line of V-values ​​is transmitted alternately, resulting in a uniform data stream. In typical embodiments, image information from two lines is evaluated for the calculation of the U-values ​​and / or the V-values. For this purpose, U-values ​​and / or V-values ​​are typically temporarily stored. In such a case, separating the image data line-wise does not necessarily lead to an increase in the camera's memory, because temporary storage occurs anyway due to the cross-line evaluation of pixel data.

[0047] In principle, it is also possible to use items other than those listed in the Fig. The data formats shown in Figures 1 to 3 can be used. U-values ​​and V-values ​​can, for example, be encoded in RGB data formats, in which case one color component of the RGB data format would not be used. Furthermore, in cases where, for example, 6 bits are sufficient for the U-values ​​and V-values, it is possible to transfer the data from camera 1 to image processing device 2 such that all Y-values ​​are transmitted as RAW12 data formats, all U-values ​​are transmitted as RAW6 data formats, and all V-values ​​are transmitted as RAW7 formats. In this case, the most significant bit (MSB) would not be used in the RAW7 format.

[0048] In typical embodiments, the data preparation process S1 is performed in the camera 1 itself, or preferably in a logic module directly downstream of the camera 1. In other words, in such embodiments, the Y-values, U-values, and V-values ​​are packed into CSI-2-compatible data formats before the actual transmission of the image data stream 4 to the image processing device 2. Alternatively, however, it is also possible to transmit all image data as user-defined (8-bit) data and then perform a reformatting at the input side of the image processing device 2, for example, by means of a component that is connected upstream of the image processing device 2. Such a format conversion can, for example, be easily implemented in a configurable logic module (such as an FPGA).

[0049] In a typical embodiment, one of the methods according to the invention is implemented in a vehicle to make image data transmission between an ADAS camera and / or an AD camera and / or a plurality of such cameras on the one hand and an image processing device on the other hand as efficient as possible. For this purpose, the vehicle typically includes a system according to the invention for transmitting and / or processing an image data stream.

[0050] The invention is not limited to the described embodiments. The scope of protection is defined by the patent claims.

[0051] In principle, all methods described in the description or in the claims can be carried out by devices which include means for carrying out the respective process steps of these methods. Reference sign 1 camera 2 Image processing device 3 Data processing system 4 Image data stream S1 Data Preparation Process YUV image data in YUV format

Claims

[1] Method for transmitting an image data stream (4) from a camera (1) to an image processing device (2), - wherein the camera (1) provides the image data stream (4) as camera output data, - wherein the image processing device (2) uses a standard interface, - wherein the method comprises a data processing process (S1) during which the camera output data are processed such that the image data stream (4) is compatible with the standard interface after the data processing process (S1), characterized by , that the image data stream (4) is brought into a form during the data preparation process (S1) in which image data of the image data stream (4) are separated according to brightness information and color information, where the standard interface is a CSI, CSI-2 or CSI-3 interface. [2] Method according to claim 1, characterized by , that - the image data have bit widths other than 8 bits or 10 bits, in particular for example 12 bits, and / or are transmitted with bit widths other than 8 bits or 10 bits, in particular for example with 12 bits, and / or - the brightness information has different bit widths than the color information and / or the brightness information is transmitted with different bit widths than the color information. [3] Method according to one of the preceding claims, characterized by that in the data processing process (S1) the image data stream (4) is processed in such a way that after the data processing process (S1) there are three partial images for each image of the image data stream (4), in particular one luminance partial image and two chrominance partial images, the two chrominance partial images preferably being transmitted in a common partial image. [4] Method according to one of claims 2 to 3, characterized bythat the camera output data is in YUV format, - where typically the Y values ​​each have a length of 12 bits and / or the U values ​​each have a length of 8 bits and / or the V values ​​each have a length of 8 bits, and / or - whereby luminance image data of the luminance sub-image typically have the data format «RAW12» and / or chrominance image data of the chrominance sub-images typically have the data format «RAW8» and / or «Userdef (8Bit)», and / or - wherein typically the luminance sub-images, in particular the Y values, and / or the chrominance sub-images, in particular the U values ​​and / or the V values, have bit widths other than 8 bits or 10 bits, in particular for example 12 bits, and / or are transmitted with bit widths other than 8 bits or 10 bits, in particular for example with 12 bits, and / or - wherein typically the Y values ​​have different bit widths than the U values ​​and / or the V values ​​and / or wherein typically the Y values ​​are transmitted with different bit widths than the U values ​​and / or the V values. [5] Method according to one of claims 3 to 4, characterized by that the luminance sub-images of the image data stream (4) are transmitted during an image recording and / or that the chrominance sub-images of the data stream are typically transmitted in a blanking interval, preferably a vertical blanking interval and / or a horizontal blanking interval. [6] Method according to one of the preceding claims, characterized by that the data preparation process (S1) and / or the transmission of the image data stream (4) takes place line by line. [7] Method according to one of claims 3 to 6, characterized bythat the partial images are transmitted to the image processing device (2) sequentially, typically ordered according to data formats and / or typically according to a predetermined, repeating sequence. [8] Method according to one of the preceding claims, characterized by that the data preparation process (S1) takes place on the camera side. [9] System for transmitting and / or processing an image data stream (4), comprising means (1, 3) for at least partially carrying out a method according to one of claims 1 to 8. [10] Vehicle suitable for carrying out a method according to one of claims 1 to 8. [11] Logic component comprising a fixed and / or a configurable logic implementation, characterized by that the logic module is suitable for converting a method according to one of claims 1 to 8 into logic. [12] A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 8. [13] Computer-readable medium, characterized by that the computer-readable medium comprises computer program code for carrying out a method according to one of claims 1 to 8.

Citation Information

Patent Citations

  • Vehicle vision system with forward viewing camera

    US20170334367A1

  • Digital cameras with direct luminance and chrominance detection

    US20190116347A1

  • Method and apparatus for processing digital video camera signals

    US5568192A