Method for creating at least one encoding criterion for encoding an image received by an optical sensor, method for encoding an image received by an optical sensor, and information processing device

Abstract

The invention relates to a method (600) for creating at least one encoding criterion (105) for encoding an image (115) received by an optical sensor (110), wherein the method (600) comprises at least a step of reading in (610) the image (115) received by the optical sensor (110) and a step of creating (620) a frequency distribution (122) of occurrences of light signal values (123) at different image points (118) in the image (115). Furthermore, the method (600) comprises a step of assigning (630) code words (135) to the light signal values (123) using the frequency distribution (122) to create the at least one encoding criterion (105) for encoding the image (115) received by the optical sensor (110).

Description

Technical Field

[0001] This invention is based on the methods and information processing apparatus of the type claimed in the independent claims. The subject of this invention is also a computer program. Background Technology

[0002] Image processing systems often present the challenge of storing or processing images depicting very different regions of an environment with minimal storage effort, while still reproducing as much detail as possible with sufficient accuracy. For this reason, very high image resolutions are often used; however, this requires significant storage to store or encode the image, even if the individual regions contain relatively little detail to depict, thus necessitating coarser encoding methods. Summary of the Invention

[0003] Against this backdrop, the present invention introduces a method for creating at least one encoding criterion for encoding images received by an optical sensor, a method for encoding images received by an optical sensor, further introduces an information processing apparatus using at least one of these methods, and finally introduces a corresponding computer program according to the main claim. Advantageous extensions and improvements of the apparatus specified in the independent claim are possible through the measures listed in the dependent claims.

[0004] Based on the scheme proposed here, a method is proposed for creating at least one coding criterion for encoding images received by an optical sensor, the method comprising at least the following steps:

[0005] - Read in the image received by the optical sensor;

[0006] - Create the frequency distribution of light signal values ​​at different points in the image; and

[0007] - Assign codewords to optical signal values ​​using frequency distribution to create at least one coding criterion for encoding images received by an optical sensor.

[0008] An optical sensor can be understood as, for example, a camera. An image point in an image can be understood as, for example, a pixel: at that pixel, brightness or other optical parameters acting at a related location are detected and can be stored or forwarded as optical signal values. Optical signal values ​​can be understood as, for example, brightness, light intensity (e.g., a specific spectral range (color)), polarization, extended spectral channels, etc. A codeword can be understood as a symbol or combination of symbols; for example, using these symbols or combinations of symbols, optical signal values ​​can be stored in a sequential numbering system of codewords. An encoding criterion can be understood as an allocation criterion by which optical signal values ​​or ranges of optical signal values ​​are assigned to corresponding codewords (or vice versa).

[0009] The proposed scheme is based on the understanding that by assigning different light signal values ​​to codewords, it is possible to encode an image or at least a portion of an image very efficiently, taking into account the frequency distribution of signal values ​​appearing on the image (or a portion of an image). This allows for the encoding of image signal values ​​provided by pixels or image points, enabling the best possible consideration of other objects depicted in the image or image portion. Utilizing the fact that other objects are also characterized by increased light intensity at corresponding image points in the image or image portion, information about how many objects or different or distinguishable objects in the image need to be encoded is obtained by calculating the frequency distribution of brightness or other light parameters at each image point in the image. In this way, by subsequently applying the created encoding criteria, image encoding can be achieved very memory-efficiently, allowing for high detail at the resolution of individual objects in the image, and accordingly, the appropriate format of the codewords used can be selected. In this way, for example, an encoding criteria that creates only a small codeword space can be used to encode an image or image portion where a small number of objects or details need to be distinguished, while an encoding criteria that creates a larger codeword space can be used to encode an image or image portion where many objects or details need to be distinguished. Therefore, by using the frequency distribution of optical signal values, the possibility of using coding criteria adapted to each image to be encoded is opened up, thereby enabling memory-efficient encoding of images or portions of images.

[0010] Another advantageous implementation of the proposed scheme here involves assigning codewords to light signal values ​​in the allocation step such that, within a range of high-frequency light signal values ​​in the image, adjacent codewords are assigned light signal values ​​with small differences between the light signal values ​​to be assigned to the codewords. Alternatively, within a range of low-frequency light signal values ​​in the image, adjacent codewords can be assigned light signal values ​​with large differences between the light signal values ​​to be assigned to the codewords. This implementation of the proposed scheme offers the advantage of using codewords with very strong distinguishability for regions in the image that have very similar brightness or light parameters, while encoding regions in the image that have very large brightness differences using codewords with less distinguishable light signal values. The distinguishability of the codewords can here be based on the light signal values ​​assigned to each codeword according to an encoding criterion. In this way, image content can be encoded very efficiently, especially for regions with many different objects, achieving a high level of detail, and therefore, variations in resolution depth can be achieved when encoding the image as required.

[0011] Another particularly advantageous implementation of the scheme presented here involves reading an image, representing a portion of the overall image received by the optical sensor, during the read-in step. Here, at least another portion of the overall image received by the optical sensor may not be read in and may be used to create a frequency distribution. This implementation offers the advantage of being able to create different coding criteria for different segments of the overall image, which improves the efficiency of encoding or storing the overall image; for example, different coding criteria can be used for different parts of the overall image if needed.

[0012] Another possible implementation of the proposed scheme is that, in the reading step, another image received by the optical sensor is read in; in the creation step, a different frequency distribution of the light signal values ​​appearing at different points in the other image is created; and in the allocation step, another codeword is assigned to the light signal values ​​using the different frequency distribution to create a different encoding criterion. For example, the other image may relate to a region of the overall image, which is radially symmetrically located outside the optical center of the overall image. Additionally, in the grid-like structure of the overall image... In the case of subdivision, additional images can also correspond to additional rasters from this overall image. This implementation of the proposed scheme offers the advantage of creating different coding rules for different images / sub-images, and thus enabling the selection of the most advantageous of multiple coding rules for the relevant application scenario, thereby achieving image encoding with the most memory-efficient approach.

[0013] Another particularly advantageous implementation of the scheme presented here involves using additional codewords with a longer codeword length than the codewords in the encoding criteria during the allocation step. In this way, different requirements for level of detail can be advantageously accommodated when encoding images with a large number of objects.

[0014] Furthermore, according to another embodiment of the scheme proposed herein, a frequency distribution can be created in the creation step for a predetermined spectral range of optically visible light and / or a spectral range detectable by a sensor (e.g., ultraviolet and / or NIR (near-infrared) range and / or SWIR (= short-wave infrared) or FIR (faraday infrared) range) and / or light signal values ​​of light parameters detectable by a sensor, and / or wherein, in the allocation step, binary sequences, especially binary sequences with predetermined equal bit lengths, are used as codewords. What can be utilized here is that within certain spectral ranges of optically visible light, which, for example, represent specific colors such as red, green, blue, or similar colors, there exists a particularly large amount of information about the rich details of the image, which can then also be used to determine codewords or encoding criteria. Using binary sequences as codewords, especially equal-length codewords in encoding criteria, provides the advantage of simplified digital or circuit techniques for encoding images using that encoding criteria.

[0015] Another possible implementation of the proposed scheme is that, in the allocation step, the lowest value codeword is assigned to the optical signal value that is near the lowest optical signal value in the frequency distribution within the tolerance range. Alternatively, in the allocation step, the highest value codeword can be assigned to... Codewords are assigned to optical signal values ​​that are near the highest optical signal value in the frequency distribution within a tolerance range. For example, such a tolerance range could include ten to twenty percent of the range near the respective lowest or highest optical signal values ​​from the authoritative distribution. Such an implementation of the scheme proposed herein offers the advantage of using the created codeword space as optimally as possible through codewords or correspondingly assigned optical signal values.

[0016] In order to use the expiration points at each image point as cheaply and memory-efficiently as possible Based on the frequency distribution of the proposed scheme, a method for encoding images received by an optical sensor is also proposed, which includes the following steps:

[0017] - Read in at least one image and an encoding criterion, wherein the encoding criterion represents the allocation of light parameters of image points of the image to one of a plurality of distinguishable codewords, wherein, according to the encoding criterion, at least one first difference in the luminance and / or light parameters allocated between two adjacent codewords differs from a second difference in the luminance and / or light parameters allocated between two further adjacent codewords; and

[0018] - Assign a codeword to at least one of the multiple image points of the image in order to encode the image.

[0019] In the current context, an image can be understood as an overall image sensor signal representing an image of the sensor's surrounding environment detected by the sensor. However, it is also conceivable that the image is given only as a portion of the overall image, and then it should be encoded using a coding criterion. Here, for example, an allocation criterion can be used as the coding criterion, which is obtained using the frequency distribution of brightness or light parameters of individual image points in the image presented here.

[0020] A particularly advantageous implementation involves selecting one of several encoding criteria during the reading step, particularly using the frequency distribution of light signal values ​​occurring at different image points in the image. This implementation offers the advantage of enabling memory-saving encoding selection of appropriate encoding criteria for each image / image portion, wherein, in the allocation step, information is stored regarding which encoding criteria is used to encode the image / image portion, so that the image / image portion can be decoded as quickly and clearly as possible.

[0021] Another particularly advantageous implementation of the scheme presented here involves, in the read-in step, at least one additional image different from the stated image and an additional encoding rule, wherein the encoding rule differs from the additional encoding rule, and wherein, in the allocation step, one additional codeword of each of the additional encoding rules is allocated to at least one additional image point among a plurality of additional image points of the additional image, in order to encode the image. Such an implementation provides the advantage of being able to use different encoding rules for different images or portions or regions of the entire image, thereby allowing the use of advantageous encoding rules for memory-efficient encoding in each case, depending on the conditions in the image or a region of the entire image.

[0022] To minimize optical interruptions between different parts of the overall image during decoding, according to another embodiment of the scheme presented herein, in the allocation step, auxiliary values ​​obtained using codewords and additional codewords can be assigned to image points in edge regions of the image. Additionally or alternatively, according to this embodiment, additional auxiliary values ​​obtained using codewords and additional codewords can be assigned to additional image points in edge regions of other images during the allocation step. In particular, interpolation can be performed using at least one codeword and additional codewords when obtaining auxiliary values ​​and / or additional auxiliary values. Such interpolation can be accomplished, for example, by assuming that for image points further within the image, the weight of the codeword encoded by that image is greater than the weight of the codeword encoded by the adjacent image.

[0023] Alternatively or additionally, according to another embodiment, in the read-in step, at least partially overlapping images and other images can be read in. In this way, decoding of the images and other images or fragments of the whole image can be performed, wherein optical artifacts during decoding of the images or the whole image can be kept as low as possible or completely avoided.

[0024] These methods can be implemented, for example, in software or hardware or a combination of software and hardware, such as in an information processing device.

[0025] The proposed solution also implements an information processing device designed to perform, manipulate, or implement variations of the methods proposed herein within a corresponding device. This embodiment of the invention, in the form of an information processing device, can also quickly and efficiently solve the task on which the invention is based.

[0026] Therefore, the information processing device may have at least one computing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The computing unit may be, for example, a signal processor, a microcontroller, etc., and the storage unit may be flash memory, EEPROM, or magnetic storage. The communication interface may be configured to read or output data wirelessly and / or via wired means, wherein a communication interface capable of reading or outputting wired data may, for example, read the data from or output it to a corresponding data transmission line electrically or optically.

[0027] In the current context, an information processing device can be understood as an electrical device that processes sensor signals and controls and / or data signals based on its output. The information processing device may have an interface that can be configured as hardware and / or software. In the case of a hardware configuration, the interface may be, for example, part of a so-called system ASIC that incorporates various functions of the information processing device. However, the interface may also be a separate integrated circuit or at least partially composed of discrete structural elements. In the case of a software configuration, the interface may be a software module that exists, for example, on a microcontroller along with other software modules.

[0028] Preferably, there is also a computer program product or computer program having program code, which may be stored on a machine-readable carrier or storage medium, such as semiconductor memory, hard disk memory or optical memory, and is used to execute, implement and / or manipulate the steps of a method according to one of the above embodiments, especially when the program product or program is implemented on a computer or device. Attached Figure Description

[0029] Embodiments of the proposed solution are illustrated in the accompanying drawings and explained in more detail in the following description. The drawings show:

[0030] Figure 1 A schematic block diagram of an information processing device is shown for creating encoding criteria for encoding images received by an optical sensor;

[0031] Figure 2 The overall image is shown, as it is, for example, based on Figure 1 The illustration in the image is as shown in the output of a camera, which acts as an optical sensor.

[0032] Figures 3A to 3D The left-hand plot shows the overall image and different portions of the overall image that are considered to be used to obtain the frequency distribution, while the right-hand plot shows the resulting frequency distribution and the corresponding codewords.

[0033] Figure 4 A schematic block diagram of an information processing device for encoding images received by an optical sensor is shown.

[0034] Figure 5 A schematic diagram is shown for preparing a portion or image, as well as another portion or another image, for encoding an image received by an optical sensor;

[0035] Figure 6 A flowchart illustrating an embodiment of a method for creating at least one encoding criterion for encoding images received by an optical sensor; and

[0036] Figure 7 A flowchart is shown of an embodiment of a method for encoding images received by an optical sensor. Detailed Implementation

[0037] In the following description of advantageous embodiments of the invention, the same or similar reference numerals are used for elements shown in different figures that have similar effects, wherein repeated descriptions of these elements are omitted.

[0038] Figure 1 A schematic block diagram of an information processing device 100 is shown, which is used to create encoding criteria 105 for encoding an image 115 received by an optical sensor 110. First, the information processing device 100 includes an interface 117 for reading the image 115 detected by the optical sensor 110, for example, configured as a camera. The detected image 115 is represented, for example, as a two-dimensional arrangement of a plurality of image points 118 (also called pixels), arranged, for example, in rows and columns. These image points 118 then represent objects 119 in the environment detected by the optical sensor 110, wherein these image points 118 depict a representation of the object 119 in the image 115 with a specific brightness (and possibly some predefined color) at a corresponding location.

[0039] The information processing device 100 also includes a creation unit 120, in which a frequency distribution 122 is created, wherein the subdivision of the brightness or light parameter or light signal value 123 appearing in the image point 118 is plotted, for example, on the vertical axis 124 of a graph illustrating the frequency distribution 122, and for example, on the horizontal axis of a graph illustrating the frequency distribution 122, the number 124 of the brightness / light signal value 123 appearing correspondingly in the image point 118 of the image 115 is plotted. Therefore, the frequency distribution 122 can be used to identify which light signal values ​​123 appear in which number 124 at each image point 118 across the entire area of ​​the image 115. Thus, it is readily apparent from the frequency distribution 120 which light signal values ​​123 should be encoded in a very fine-grained manner for detailed illustration or encoding of the image, and which light signal values ​​123 for which a coarser gradation is sufficient without significant information loss, thereby eliminating the need for the required precision to identify the object 119, for example, for subsequent automatic evaluation in highly automated vehicle driving.

[0040] The creation unit 120 can also be understood as the overall data content of the optical signal value being analyzed here; therefore, the creation unit 120 can also be called the analysis unit.

[0041] Furthermore, the information processing device 100 includes an allocation unit 130 configured to allocate codewords 135 to optical signal values ​​123. Codewords 135 can be symbols or combinations of symbols, and a codeword space is created using codewords 135 in which the optical signal values ​​123 can be encoded. For example, codewords 135 can be binary sequences of lengths of, for example, 4 bits, 6 bits, 10 bits, 12 bits, 16 bits, 20 bits, or 24 bits. A more advantageous method is to encode the optical signal values ​​123 in the image 115 using codewords 135 of the same length, such that, for example, all codewords 135 have a length of 4 bits. The allocation criteria from codewords 135 to optical signal values ​​123 then form encoding criteria 105. A more detailed description of the allocation of codewords 135 to optical signal values ​​123 and vice versa is explained below.

[0042] In order to depict the most diverse scenarios of the arrangement of objects 119 in the environment surrounding the optical sensor 110 as accurately as possible, the above process can be repeated at least once, but it is advantageous to repeat it multiple times. Here, at least one additional image 115' different from image 115 can be read from the optical sensor 110. This additional image 115' can also be composed of a corresponding arrangement of additional image points 118', from which an additional frequency distribution 122' can be obtained, wherein additional codewords 135' are assigned to the optical signal value 123 according to an additional encoding criterion 105'.

[0043] Furthermore, the above process is not limited to sampling performed sequentially in time, but can also be obtained by reading out spatially separated sensor elements (such as segmented pixel arrays or multi-channel sensors, or even stacked detectors (such as FOVEON)).

[0044] It is also conceivable that image 115 (or another image 115') is only a portion of the overall image, in which the arrangement of object 119 within the environment of optical sensor 110 is reproduced. Here, for example, image 115 and another image 115' can be received simultaneously, but representing different sub-regions of the overall image. In this way, corresponding encoding criteria 105 (or 105') can be created for different sub-regions of the overall image, thereby causing as little information loss as possible through such encoding, depending on the required level of detail in the depiction or encoding.

[0045] It is also conceivable that the information processing device 100 is set up in a laboratory environment and reads only temporarily stored images 115 or 115' previously received by the optical sensor 110, for example, during driving in a real environment, via the interface 117. Therefore, it is not required that the coding guidelines 105 be created in real time immediately after the image 115 is generated.

[0046] To describe in more detail the process of assigning codeword 135 to optical signal value 123, this document refers to several exemplary images 115 or portions of images 115 in the following figures, where it is not important whether image 115 used to determine the frequency distribution 122 is merely a part of the overall image or the overall image itself. For simplicity, the process is described in more detail only with reference to the read-in image 115, and this should not be construed as a limitation on the following description.

[0047] Figure 2 The overall image 200 is shown, as it is, for example, based on Figure 1 The illustration is as shown in the image 115 output from the camera, which is the optical sensor 110. Multiple objects 119 can be identified here, such as vehicles, tunnel wall doors, tunnel lights, lane markings, and tunnel entrances. Now, for example, if... Figure 2 The overall image 200 shown is divided into three rows and three columns, resulting in nine parts of the image. Each part can be considered as image 115 for obtaining individual frequency distributions 122 and as the basis for obtaining coding criteria 105.

[0048] exist Figure 3A , Figure 3B , Figure 3C and Figure 3D In the diagram, the overall image 200 and the various parts of the image 115, which are considered to be used to obtain the frequency distribution 122, are shown on the left side, while the curve of the frequency distribution 122 obtained therefrom and the corresponding codewords (in this case, the length is 4 bits, thereby creating a codeword space of 16 codewords arranged in sequence) is depicted on the right side.

[0049] exist Figure 3A In this context, image 115 is used here as a portion of the bottom row of the middle column of the overall image 200, where this portion of image 115 shows most of the damp and therefore reflective road in the tunnel within the overall image 200. As a corresponding frequency distribution 122, a very uniform range of brightness as a light parameter is derived in the central portion of the graph in the right-hand partial view, a range that contains almost no particularly high or low brightness values ​​as a light parameter. In other words, light parameters such as brightness in image 115 are very close to each other, so that for high resolution of detail, within the range of high-frequency light signal values ​​123 in image 115, adjacent codewords 135 are assigned light signal values ​​123 with small differences between the light signal values ​​123 to be assigned to codeword 135. In this way, in Figure 3AIn the right-hand diagram, when codewords 135 are connected in ascending order via connecting lines, a very steep transition occurs in graph 300. It can also be understood that codeword 135 is assigned to the lowest value (e.g., within tolerance) optical signal value 132 located near the lowest optical signal value in the frequency distribution. The lowest value codeword can be understood as a codeword that does not have another preceding codeword 135 in the sequential arrangement. Similarly, the highest value codeword 123 is assigned (e.g., within tolerance) optical signal value 123 located near the highest optical signal value 123 in the frequency distribution 122. The highest value codeword can be understood as a codeword that does not have another subsequent codeword 135 in the sequential arrangement. In this way, through the allocation of codewords 135, the dynamic range of the optical signal value 123 can be encoded very efficiently, for example, for highly advantageous reconstruction in the post-processing stage.

[0050] exist Figure 3B In this context, image 115 is used here as a portion of the top row of the right column of the overall image 200, where this portion of image 115 shows most of the ceiling covering in the overall image 200, including the lights above the road in the tunnel. As a corresponding frequency distribution 122, in Figure 3B The right-hand portion of the graph in the illustration shows a highly uneven range of brightness as a light parameter 123, caused by the presence of bright radiating light fixtures. This range includes exceptionally high brightness 123 in addition to moderate brightness. In other words, the brightness as a light parameter in the image 115 is sometimes quite different from each other. Thus, for high-resolution details, in the first, middle (and right) regions of the image 115 with high-frequency light signal values ​​123, adjacent codewords 135 are assigned light signal values ​​123 with small differences between the light signal values ​​123 to be assigned to codewords 135. Between these two ranges of frequently occurring light signal values ​​123, there is another range of light signal values ​​123 that almost never appear in the image 115. For these optical signal values ​​123, fine differentiation is not required. Therefore, generally, within the range of optical signal values ​​123 in image 115 that have low frequencies (compared to other optical signal values ​​123), adjacent codewords 135 are assigned optical signal values ​​123 that have large differences between the optical signal values ​​123 to be assigned to codewords 135. In this way, in Figure 3BIn the right-hand portion of the diagram, when codewords 135 are connected in ascending order via connecting lines, a very steep gradient is observed in the region where optical signal values ​​123 accumulate. In the region between two areas where optical signal values ​​123 frequently occur, the graph 300 exhibits a significantly flatter slope. Furthermore, it can be recognized again that the codeword 135 assigned to the lowest value is (e.g., within tolerance) located near the lowest optical signal value 123 in the frequency distribution 122. Similarly, the codeword 123 assigned to the highest value is (e.g., within tolerance) located near the highest optical signal value 123 in the frequency distribution 122.

[0051] exist Figure 3C In this context, image 115 is used here as a portion of the middle row of the right column of the overall image 200, where this portion of image 115 shows multiple vehicles and doors of object 119 within the overall image 200. As a corresponding frequency distribution 122, it is again observed due to the presence of brightly illuminated tunnel walls. Figure 3C The right-hand portion of the graph in the illustration shows a range of unevenness in brightness, representing the light parameter 123, in the middle and right sections. This unevenness is not as pronounced as in Figure 3 with actively illuminating light fixtures. Therefore, graph 300 is also not as... Figure 3A The graph in the image is as steep as 300, but at least steeper than... Figure 3B The middle part of the figure 300 is steeper.

[0052] exist Figure 3D In this context, image 115 is used here as a portion of the bottom row of the right column of the overall image 200, where this portion of image 115 shows multiple reflected road markings of object 119 in a dark road in other cases within the overall image 200. As a corresponding frequency distribution 122, due to the presence of brightly reflected road markings, in... Figure 3D The right-hand portion of the diagram shows a fairly uniform range of brightness as the light parameter 123, which is again obtained in the middle part of the graph. This uniform range now generally corresponds to... Figure 3A The frequency of the optical signal value 124 is lower than that of the optical signal value 123. Therefore, the graph 300 is also quite steep again, as... Figure 3A The diagram in the image shows that, in each case, the image begins and ends with a lower optical signal value of 123.

[0053] In order to encode image 115 as efficiently as possible while minimizing memory usage, the obtained encoding criterion 105 can now be advantageously used. This will be described in more detail below with the aid of the corresponding information processing device 400 for encoding image 115 received by optical sensor 110.

[0054] Figure 4 A schematic block diagram of an information processing device 400 for encoding an image 115 received by an optical sensor 110 is shown. First, the information processing device 100 includes an interface 117 for reading at least a portion of an image 115 from an optical sensor 110, which is, for example, configured as a camera. The read-in image 115 is again represented, for example, as a two-dimensional arrangement of a plurality of image points 118 (also referred to as pixels) arranged in rows and columns. These image points 118 then represent objects 119 in the environment detected by the optical sensor 110, wherein these image points 118 depict a representation of the object 119 in the image 115 with a specific brightness (and possibly some predefined color) at a corresponding location.

[0055] Furthermore, encoding criterion 105 and possibly at least one additional encoding criterion 105' can be read from memory 405, which is located outside the information processing device 400 (however, the memory may also be located inside the information processing device 400), via interface 117. Encoding criterion 105 or the additional encoding criterion 105' forms an allocation criterion, for example, obtained according to the foregoing description. For example, the encoding criterion may represent the allocation of the brightness or light parameter of image points in a portion of an image to one of a plurality of distinguishable codewords, wherein, according to the encoding criterion 105, at least one first difference in brightness as light parameter 123 allocated between two adjacent codewords 135 differs from a second difference in brightness as light parameter 123 allocated between two other adjacent codewords 135. In this way, encoding criterion 105, which describes the non-uniform allocation from light signal values ​​to codewords 135, can be read in, as it is specifically obtained by taking into account the frequency distribution of different brightness occurrences as light parameters in image 115. In allocation unit 130, for example, each codeword is then allocated to at least one of a plurality of image points of a portion of the image, so as to encode image 115 and obtain encoded image 420. This encoded image 140 can then, for example, in... Figure 4 The image may be further processed, stored, or sent, for example, to a central computer unit for further processing or to other road users in an image processing unit (e.g., a driver assistance system for a vehicle) not shown in the image.

[0056] Furthermore, for example, the frequency of brightness as a light parameter appearing in image 115 can be analyzed in interface 117, and a specific one of multiple coding criteria 105 can be selected based on the obtained analysis results. This possibility provides the advantage of selecting the coding criterion 105 to be used before actual encoding or assigning codewords to image points 118, which is based on a very similar frequency distribution that also exists in the image 115 to be encoded. In this way, by loading or reading in the optimal coding criterion 105 for the current image 115, memory-saving encoding of image 115 can be implemented very quickly and in a low-cost manner in terms of digital or circuitry techniques.

[0057] As already implemented above in determining encoding criteria 105 or 105', the read-in portion of image 115 or 115' can also be a segment (at least partially different) that is received simultaneously with the overall image 200 but depicts other regions of the overall image 200. (Based on a partial appendix from Figure 3) Figure 3A , 3B In 3C and 3D illustrations, when encoding image 115, it is now possible to encode individual parts of the image / overall image 200 (here referred to as image 115 or another image 115') using different encoding criteria 105 or 105' (i.e., encoding criterion 105 and at least one other encoding criterion 105'). In this way, different situations or different required levels of detail in different images 115 or 115' can be fully considered.

[0058] If now the overall image 200 is based on Figure 2 The illustration in the image is divided into multiple (partial) images 115 or 115' and encoded. When decoding such segmented encoded images 420, image artifacts, such as edges at the interfaces of these parts, may occur. These image artifacts should be avoided as much as possible, especially if further automated image processing is to be performed to identify objects or situations so that the scene in front of the optical sensor 100 can be automatically evaluated. To avoid such image artifacts or edges, a further optimized form of encoding of image 115 can be used, as described in more detail below.

[0059] Figure 5 A schematic diagram showing part or image 115 and another part or another image 115' is shown. Figure 2As shown, the image and another image are located in adjacent portions of the overall image 200. If image point 118 of image 115 is now encoded using coding rule 105 and another image point 118' of another image 115' is encoded using another coding rule 105', these image artifacts may appear, for example, in the boundary region 500 where image 115 and another image 115' meet. To avoid this, for example, auxiliary values ​​510 can be read into image point 118 of image 115. These auxiliary values ​​are obtained by using codeword 135 assigned to the relevant image point 118 using coding rule 105 and another codeword 135 assigned to another image point 180' of another image 115' according to another coding rule 105'. For example, the auxiliary value 510 can be obtained with the following interpretation: the closer image point 118 is to the boundary line 500, the greater the weight of the other codeword 135'. Similarly, for example, an additional auxiliary value 520 assigned to another image point 118' of another image 115' can be obtained using the following interpretation: the closer the other image point 118' is to the boundary line 500, the greater the weight of the codeword 135. In this way, the codeword 135 or 135' or auxiliary value 510 or 520 assigned to each image point 118 or 118' can be made as smooth or continuous as possible, thereby avoiding corresponding image artifacts when necessary.

[0060] Alternatively or additionally, image 115 or another image 115' may also be selected or determined in such a way that they at least partially overlap (as it is through) Figure 5 The other image 115' is shown by a dashed line in the image, and thus, for example, for the same image point 118 or 118', codeword 135 can be obtained on one hand, and another codeword 135' can be obtained on the other hand. In this case, for example, the corresponding auxiliary value 510 can also be obtained using codeword 135 and another codeword 135' (e.g., via a message).

[0061] Figure 6 A flowchart illustrating an embodiment of a method 600 for creating at least one encoding criterion for encoding an image received by an optical sensor is shown, wherein the method 600 has at least one step 610 of reading in an image received by an optical sensor. Furthermore, the method 600 includes a step 620 of creating a frequency distribution of the occurrence of light signal values ​​at different image points in the image. Finally, the method 600 includes a step 630 of assigning codewords to light signal values ​​using the frequency distribution to create at least one encoding criterion for encoding the image received by the optical sensor.

[0062] Figure 7A flowchart illustrating an embodiment of a method 700 for encoding an image received by an optical sensor is shown. The method 700 includes a step 710 of reading in at least one portion of an image and an encoding criterion, wherein the encoding criterion represents the allocation of brightness or light parameters of image points of a portion of the image to one of a plurality of distinguishable codewords, wherein, according to the encoding criterion, at least one first difference in brightness or light parameters allocated between two adjacent codewords differs from a second difference in brightness or light parameters allocated between two further adjacent codewords. Furthermore, the method 700 includes a step 720 of assigning each codeword to at least one of a plurality of image points of a portion of the image to encode the image.

[0063] In summary, it should be noted that the proposed scheme, along with at least one encoding criterion for creating images received by an optical sensor and two disclosed methods for encoding images received by an optical sensor, opens up the possibility of reducing the data depth required for transmitting, processing, and storing sensor data from an optical sensor by performing non-lossless reduction of the information. Particular attention may be drawn to the different aspects described below.

[0064] 1.) Information reduction is chosen in such a way that information loss can occur in a variable manner, both in location and time.

[0065] Regarding this point, positional variability can be understood as follows: the information compression function can depend (e.g., in an image sensor) on a two-dimensional array of images 115 detected by the sensor, or (in a 3D sensor) on a "region of interest" within a three-dimensional volume element. A particular form of this image selection, and especially useful for encoding, is an image or portion thereof, such as:

[0066] a. Radially symmetrical about the optical center

[0067] b. Apply individually to a predefined grid, such as top left, top center, top right, etc.

[0068] c. Extending relatively far to the defined coordinates of the rasterized available information, for example to the target coordinates x, y, where the function can then be applied in any defined variable region around the target coordinates. For practical reasons, portions or images 115 are typically selected symmetrically about the target coordinates.

[0069] Furthermore, regarding this point, time-variable generation can be understood as the compression standard at system startup not being fixed, but rather changing from time to time, so that...

[0070] a) To react to rapidly changing situations (such as entering a tunnel), the compression should be changed frame by frame.

[0071] b) Allows adjustment for slow parameter drift (e.g., diurnal variation, temperature drift, contamination or aging drift).

[0072] 2.) The information reduction is chosen in such a way that the impact on subsequent processes is harmless or minimal (i.e., below a certain threshold).

[0073] Compression is cleverly chosen to maximize the use of available hardware resources (which can also be dynamically allocated) under given probing conditions. (See point 1.2)

[0074] As an example, it can be said that the same applies to the human visual system.

[0075] 3.) Information reduction based on the following situations

[0076] a) Statistical parameters of the local environment or the overall image

[0077] In this example, this is luminance; other parameters include chromaticity and depth information (e.g., derived from stereo parallax or TOF (time-of-flight) signals).

[0078] b) The expected target bit depth of the data.

[0079] 4.) Through linear overlay functions By altering / adjusting the information reduction in different grids in this way, discontinuous jumps in relevant parameters (brightness, contrast, color, etc.) will not occur at the edges of the selected grids, which is beneficial for avoiding block artifacts.

[0080] 5.) The entire processing chain occurs "in situ," for example, on the image sensor or the information processing device embedded therein, even before transmission to downstream systems, thereby keeping the system's bandwidth requirements low and reducing the overall system's energy demand.

[0081] If an embodiment includes an "and / or" link between a first feature and a second feature, this should be understood as an embodiment according to one implementation including both the first feature and the second feature, and according to another implementation including either only the first feature or only the second feature.

Claims

1. A method for creating at least one encoding criterion for encoding images received by an optical sensor, wherein, The method comprises at least the following steps: Read in the image received by the optical sensor; Create the frequency distribution of light signal values ​​at different image points in the image; and The method of assigning codewords to optical signal values ​​using the frequency distribution to create at least one encoding criterion for encoding the image received by the optical sensor is characterized in that, in the assignment step, the codewords are assigned to the optical signal values ​​in such a manner that, within the range of high-frequency optical signal values ​​in the image, adjacent codewords are assigned optical signal values ​​with small differences between the optical signal values ​​to be assigned to the codewords, and / or, within the range of low-frequency optical signal values ​​in the image, adjacent codewords are assigned optical signal values ​​with large differences between the optical signal values ​​to be assigned to the codewords. Specifically, when the codewords are connected to each other in ascending order via connecting lines, a steeper change process is obtained within the range of high-frequency optical signal values ​​in the image, and / or a flatter change process is obtained within the range of low-frequency optical signal values ​​in the image. Specifically, in the reading step, an additional image received by the optical sensor is read in; in the creation step, an additional frequency distribution of the light signal values ​​appearing at different additional image points in the additional image is created; and in the allocation step, additional codewords are assigned to the light signal values ​​using the additional frequency distribution to create an additional encoding criterion. In the allocation step, auxiliary values ​​obtained using the codeword and the additional codeword are allocated to image points in the edge regions of a portion of the image, and / or, in the allocation step, additional auxiliary values ​​obtained using the codeword and the additional codeword are allocated to additional image points in the edge regions of another image, wherein, when obtaining the auxiliary values ​​and / or the additional auxiliary values, interpolation is performed using at least one codeword and the additional codeword, wherein the auxiliary values ​​allocated to the image points in the edge regions of a portion of the image are obtained in such a way that the weight of the additional codeword increases as the image point approaches the boundary line where the image and the other image meet, wherein the additional auxiliary values ​​allocated to the additional image points in the edge regions of the other image are obtained in such a way that the weight of the codeword increases as the additional image point approaches the boundary line where the image and the other image meet.

2. The method according to claim 1, characterized in that, In the reading step, an image is read in, which represents a portion of the overall image received by the optical sensor.

3. The method according to claim 1 or 2, characterized in that, In the allocation step, additional codewords are used, which have a longer codeword length than the codewords in the encoding criteria.

4. The method according to claim 1 or 2, characterized in that, In the creation step, the frequency distribution is created for the optical signal values ​​of optical visible light within a predetermined spectral range, and / or within a spectral range perceptible by the sensor, and / or for optical parameters detectable by the sensor, and / or, in the allocation step, a binary sequence is used as a codeword.

5. The method according to claim 1 or 2, characterized in that, In the allocation step, the codeword with the lowest value is allocated an optical signal value within the tolerance range that is near the lowest optical signal value in the frequency distribution, and / or the codeword with the highest value is allocated an optical signal value within the tolerance range that is near the highest optical signal value in the frequency distribution.

6. The method according to claim 4, characterized in that, in, In the allocation step, binary sequences with the same predetermined bit length are used as codewords.

7. A method for encoding an image received by an optical sensor, the method comprising the steps of: Read in at least one image and encoding criteria, wherein, The encoding criterion represents the allocation of parameters of image points of the image to one of a plurality of distinguishable codewords, wherein, according to the encoding criterion, at least one first difference in the optical signal values ​​allocated between two adjacent codewords differs from a second difference in the optical signal values ​​allocated between two other adjacent codewords, wherein the codewords are allocated to the optical signal values ​​in such a manner that, within a range of high-frequency optical signal values ​​in the image, adjacent codewords are allocated optical signal values ​​with small differences between the optical signal values ​​allocated to the codewords, and / or within a range of low-frequency optical signal values ​​in the image, adjacent codewords are allocated optical signal values ​​with large differences between the optical signal values ​​allocated to the codewords; and A codeword is assigned to at least one image point among a plurality of image points in order to encode the image. Specifically, when the codewords are connected to each other in ascending order via connecting lines, a steeper change process is obtained within the range of high-frequency optical signal values ​​in the image, and / or a flatter change process is obtained within the range of low-frequency optical signal values ​​in the image. Specifically, in the reading step, an additional image received by the optical sensor is read in; in the creation step, an additional frequency distribution of the light signal values ​​appearing at different additional image points in the additional image is created; and in the allocation step, additional codewords are assigned to the light signal values ​​using the additional frequency distribution to create an additional encoding criterion. In the allocation step, auxiliary values ​​obtained using the codeword and the additional codeword are allocated to image points in the edge regions of a portion of the image, and / or, in the allocation step, additional auxiliary values ​​obtained using the codeword and the additional codeword are allocated to additional image points in the edge regions of another image, wherein, when obtaining the auxiliary values ​​and / or the additional auxiliary values, interpolation is performed using at least one codeword and the additional codeword, wherein the auxiliary values ​​allocated to the image points in the edge regions of a portion of the image are obtained in such a way that the weight of the additional codeword increases as the image point approaches the boundary line where the image and the other image meet, wherein the additional auxiliary values ​​allocated to the additional image points in the edge regions of the other image are obtained in such a way that the weight of the codeword increases as the additional image point approaches the boundary line where the image and the other image meet.

8. The method according to claim 7, characterized in that, In the reading step, one of a plurality of encoding criteria is selected, wherein the selection is made using the frequency distribution of the occurrence of light signal values ​​at different image points in the image.

9. The method according to claim 7 or 8, characterized in that, In the reading step, an image and another image are read in, the image and the other image at least partially overlap.

10. An information processing device, the information processing device comprising: An interface for reading images received by an optical sensor; The creation unit is used to create the frequency distribution of light signal values ​​at different image points in the image. An allocation unit is configured to assign codewords to the optical signal values ​​using the frequency distribution, to create at least one encoding criterion for encoding the image received by the optical sensor. The information processing device is configured to implement the steps of the method according to any one of claims 1 to 6 based on the interface, the creation unit, and the allocation unit.

11. An information processing device, the information processing device comprising: An interface for reading in at least one image and encoding criteria; An allocation unit is configured to assign one codeword to at least one of a plurality of image points in the image, in order to encode the image. The information processing device is configured to implement the steps of the method according to any one of claims 7 to 9 based on the interface and the allocation unit.

12. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 6 or 7 to 9.

13. A machine-readable storage medium having instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 6 or 7 to 9.

Images