Imaging device with improved image processing and image processing method

Abstract

An imaging device (300) and a method for operating an optical system (100), each with improved image processing, are provided. The optical system comprises an input optics (110), an image sensor chip (130), and an optical fiber bundle (120) for transporting light incident on the input optics (110) to the image sensor chip (130), wherein the image sensor chip (130) has pixels (131), each of which consists of one or more sub-pixels (132-i). The method comprises at least the following steps: Providing (S100) a fiber structure mask indicating pixels (131) or sub-pixels (132-i) whose output is to be processed for at least one subsequent image processing (S300) to a current state of the optical fiber bundle (120) with respect to the image sensor chip (130); Capturing (S200) an image of a medical scene by means of the optical system (100); and Carrying out (S300) at least one image processing of pixels to be processed of the captured image (71), wherein the pixels to be processed correspond to the pixels (131) or sub-pixels (132-i) displayed by the fiber structure mask.

Description

Technical field of the invention

[0001] The present invention relates to an imaging device with an optical system comprising an input optic, an image sensor chip, and an optical fiber bundle for transporting light incident on the input optic to the image sensor chip. Furthermore, a method for processing images from such an optical system is provided. Background of the invention

[0002] Fiber optic endoscopes are still widely used in endoscopy today. One reason for this is that their design allows them to be manufactured significantly smaller. This design also allows the endoscopes to be semi-flexible, meaning they can be slightly bent to a certain extent along the fiber optic cables.

[0003] However, fiber endoscopes also have a disadvantage: Due to the transmission of the image via the (glass) fiber bundle, their output image data exhibits a typical honeycomb pattern: Each fiber in the system typically represents one pixel, so the number of fibers / pixels determines the maximum resolution of the system. The fibers (or image guides) are usually round. This creates cavities between the fibers that contain no image information and are therefore displayed as black in the output image data. This structure can be unpleasant to the human eye and therefore undesirable. Fig. 1 a) shows exemplary output image data which have the mentioned honeycomb pattern. Fig. 1c) shows a detailed section.

[0004] At the proximal (output) end of the fibers, the light captured from a scene is transmitted to the image sensor chip. While the fibers are typically round and arranged, for example, in a roughly hexagonal grid, the pixels of the image sensor chip are typically square and arranged in a square grid. The collision of these different grid structures can lead to artifacts, for example, due to the Shannon-Nyquist sampling theorem. Fig. Figure 1 b), for example, illustrates the resulting aliasing artifacts, which falsely depict a significantly different structure despite the essentially identical scene. This strong variability in the images, despite minimal physical changes, makes it difficult to use artificial intelligence entities to resolve the honeycomb pattern. Fig. 1d) shows a detail from Fig. 1b).

[0005] Some fiberscope manufacturers attempt to compress the fibers so tightly together that they assume an octagonal shape through thermal deformation in order to reduce the area without image information. However, these processes do not eliminate the honeycomb pattern, which reduces the quality and usability of the output image data.

[0006] Furthermore, various image processing techniques are known, such as Gaussian blurring. While these produce a more pleasing image from the output image data, with the necessary parameters, they produce excessive blur, often resulting in unacceptable loss of detail. This further reduces the usable residual resolution due to the small number of fibers / pixels.

[0007] US 2023 / 0 105 073 A1 teaches methods and systems for removing disturbing honeycomb patterns from image data acquired through a fiber endoscope using trained artificial intelligence.

[0008] DE 10 2006 011 707 A1 discloses methods for generating interference-free images projected onto an image sensor using an optical fiber bundle. For this purpose, imaging parameters are provided for the fiber bundle and sensor system, which describe the geometric and optical properties of each individual optical fiber. Summary of the invention

[0009] It is therefore an object of the present invention to provide an imaging device with improved image processing and an improved method for processing images.

[0010] This object is solved by the subject matter of the independent patent claims of the present invention.

[0011] According to a first aspect, there is provided an imaging device with improved image processing, comprising: an optical system with an input optic, with an image sensor chip having pixels each consisting of one or more sub-pixels (preferably a plurality of sub-pixels), and with an optical fiber bundle for transporting light incident into the input optic to the image sensor chip; and a computing device which is designed to: - based on a current state of the optical fiber bundle with respect to the image sensor chip, provide a fiber structure mask indicating pixels or sub-pixels whose output is to be processed for at least one subsequent image processing; and - to carry out image processing of pixels to be processed of an image of a medical scene captured by the optical system, wherein the pixels to be processed correspond to the pixels or sub-pixels displayed by the fiber structure mask.

[0012] A fundamental idea of ​​the present invention is therefore that the basic, current structural design of the optical system of the imaging device is first recorded, and further image processing steps are then carried out based on this. It has proven particularly advantageous to record the basic structure of a specific individual optical system rather than starting from the generally known fiber bundle structure. Not only do different optical systems, even of the same type, differ, for example due to aging processes, defects, contamination, or the like, but even within one and the same optical system, the way in which its images are generated can change, for example due to the changing state of the optical fiber bundle with respect to the image sensor chip.

[0013] The optical fiber bundle can, in particular, be a bundle of glass fibers, preferably each round or octagonal in shape and packed as densely as possible, e.g., hexagonally. The optical fiber bundle is preferably flexible or semi-flexible (or semi-rigid), so that a user can use the flexible fiber bundle to guide the input optics to otherwise hard-to-reach locations, for example, during an endoscopic procedure or an examination of a confined space or component.

[0014] The image sensor chip can advantageously have a color filter array (CFA). The subpixels of each pixel of the image sensor chip can thus, for example, be subpixels that differ in their sensitivity to different wavelengths, in particular visible light. Thus, the image sensor chip can, for example, be a Bayer sensor with a checkerboard arrangement of pixels, each consisting of four subpixels, two of which are sensitive to green light, one to red light, and one to blue light, which is also referred to as RGGB (Red, Green, Green, Blue) for short. The color sensitivity of the subpixels can be realized, for example, by color filters selectively applied to previously similar subpixels. However, other implementations of subpixels are also conceivable, such as variants with two different shades of green (e.g., "RGEB").The sub-pixels can all be the same size, or have different sizes, as in the X-Trans scheme.

[0015] The term "pixel" or "sub-pixel" herein initially refers to appropriately provided, prepared, and readout sections of the image sensor chip. However, since the image sensor chip is configured to generate images (or image data), and each pixel or sub-pixel of the image sensor chip has its exact counterpart in the image data generated by the image sensor chip, in that each image is composed of image elements whose values ​​(e.g., brightness or color values) correspond to the output values ​​of the pixels or sub-pixels, and whose arrangement relative to one another resembles the arrangement of the pixels or sub-pixels, the terms "pixel" and "sub-pixel" also partially refer to the images / image data themselves. The meaning will be clear from the context in each case.

[0016] The state of the optical fiber bundle with respect to the image sensor chip can, for example, be an arrangement state which describes or is based on a spatial arrangement (i.e. position and / or orientation) of the optical fiber bundle with respect to the image sensor chip, for example a displacement, tilting, and / or twisting state of the optical fiber bundle with respect to the image sensor chip.

[0017] Among these, the twist state is typically particularly important: on the one hand, because even small differences in the twist of the optical fiber bundle relative to the image sensor chip can sometimes lead to significant changes; on the other hand, because fiber bundles and image sensor chips are often connected by screwing, i.e., rotating, and thus differences in twist often exist or develop over time.

[0018] The twist state can be represented or quantified, for example, by an angle relative to a desired rotational orientation of the fiber bundle relative to the image sensor chip, with the rotation axis of the angle preferably corresponding to an optical longitudinal axis at the transition between the fiber bundle and the image sensor chip. A tilt state, on the other hand, can be quantified as an angle relative to a rotation axis perpendicular to the optical longitudinal axis.

[0019] The state of the optical fiber bundle with respect to the image sensor chip can also be a state in which points or regions of the fiber bundle with known properties (e.g. fibers that image particularly well or poorly - or not at all) are associated with certain pixels and / or sub-pixels of the image sensor chip, i.e., are aligned with one another. Corresponding information (e.g. about the properties of the individual fibers) about the optical fiber bundle can be determined, for example, using the techniques described here and stored for the optical fiber bundle. By determining the arrangement state of the optical fiber bundle with respect to the image sensor chip, the stored information can also be used to determine said state, which relates the known properties of the fiber bundle to the image sensor chip.

[0020] A “mask”, for example the fiber structure mask, can be understood as any data structure which performs the function of a mask for image data, i.e. which contains information which distinguishes individual pixels or sub-pixels from other pixels or sub-pixels in the image data, in particular in order to mask them out, exclude them from image processing, discard them, or the like.

[0021] The optical system is preferably an optical system for medical applications, in particular an endoscope (a so-called fiber endoscope or fiberscope). The imaging device can thus be designed as an endoscope or comprise an endoscope. However, the optical system can also be used in other instruments where the flexibility of the fiber bundle of the optical system is advantageous, for example, when inspecting gaps or components.

[0022] The entrance optics can be a simple sealed closure, such as a transparent glass or plastic plate. In some variants, the entrance optics can also include additional optical elements, such as lenses or prisms.

[0023] The image captured by the optical system may, in particular, be an image of a medical scene. The term "medical scene" is defined broadly here: It can refer to an external or even internal view of a patient who is currently undergoing or is about to undergo a medical procedure. In particular, the medical scene may be a scene in which an organic, particularly human, tissue can be seen, for example, in a laboratory or an operating room, either in vitro and / or in vivo.

[0024] The imaging device, in particular its computing device, may comprise various modules to which various functions are assigned. Although some functions are described here, above, and below as being performed by "devices" or "modules," it should be understood that this does not necessarily mean that such devices or modules are provided as separate entities. In cases where one or more devices or modules are provided entirely or partially as software, the devices may be implemented by program code sections or snippets that are distinct from one another but may also be interwoven.

[0025] Similarly, in the case where one or more devices or modules are provided as hardware, the functions of one or more devices or modules may be provided by one and the same hardware component, or the functions of one device or the functions of several devices or modules may be distributed among several hardware components that do not necessarily correspond one-to-one to the devices or modules. Therefore, any device, system, method, etc. that has all the features and functions attributed to a particular device and / or module is to be understood as constituting, including, or implementing the device and / or module.

[0026] In particular, it is possible that all modules are implemented by program code that is executed by a computing device.

[0027] The computing device can be implemented as any device or means for computing, in particular for executing software, an app, or an algorithm. For example, the computing device can comprise at least one processor, such as at least one central processor (CPU), and / or at least one graphics processor (GPU), and / or at least one field-programmable gate array (FPGA), and / or at least one application-specific integrated circuit (ASIC), and / or a tensor processing unit (TPU), or other AI-optimized hardware, and / or any combination of the foregoing. The computing device can further comprise a main memory operatively connected to the at least one processor, and / or a non-volatile memory operatively connected to the at least one processor and / or the main memory.The computing device may be implemented partially and / or entirely in a local device and / or partially and / or entirely in a remote system, such as through a cloud computing platform.

[0028] In the present case, it is an advantageous variant if the imaging device is an endoscope system, that the computing device is integrated into a camera control of the endoscope system, and the optical system is the optical system of an endoscope of the endoscope system.

[0029] According to a second aspect, the invention provides a A computer-implemented method for processing images of an optical system comprising an input optic, an image sensor chip, and an optical fiber bundle for transporting light incident on the input optic to the image sensor chip, wherein the image sensor chip has pixels each of which consists of one or more sub-pixels (preferably a plurality of sub-pixels); wherein the method comprises at least the following steps: providing a fiber structure mask indicating pixels or sub-pixels whose output is to be processed for at least one subsequent image processing, to (ie, adapted to, or based on) a current state of the optical fiber bundle with respect to the image sensor chip; Capturing an image of a medical scene using the optical system; and Performing at least one image processing of pixels to be processed of the acquired image, wherein the pixels to be processed correspond to the pixels or sub-pixels displayed by the fiber structure mask.

[0030] Here too, the state of the optical fiber bundle with respect to the image sensor chip can be, in particular, an arrangement state of the optical fiber bundle with respect to the image sensor chip, preferably a displacement, tilting, and / or twisting state.

[0031] The method can be carried out with the imaging device according to embodiments of the first aspect, but also independently thereof. Accordingly, the imaging device can be adapted according to all embodiments, options, variants, and refinements of embodiments described with respect to the method, and vice versa.

[0032] According to some preferred embodiments, variants, or refinements of embodiments, the image processing of the pixels of the captured image comprises an interpolation method, in particular applying a super-resolution method to the pixels to be processed, in particular using a super-resolution artificial intelligence entity (SRKIE). In simple terms, this involves adding additional pixels between the existing ones to achieve a higher resolution. The super-resolution method can, for example, be a burst super-resolution method.

[0033] In the prior art, interpolation methods utilized all pixels, including those that were not illuminated at all or only partially by fibers, those illuminated by damaged fibers, or those illuminated with light distorted by optical defects. If such pixels are subjected to interpolation or super-resolution, significantly disruptive weaknesses such as the fiber structure are also amplified and thus become even more prominent, potentially resulting in a severely distorted image.

[0034] In contrast, in the present invention, the fiber structure mask selects those pixels or subpixels that are to be further processed. This preselection results in a significantly improved database for the super-resolution method and thus also better output image data after image processing. Pixels or subpixels that are not to be processed do not necessarily mean that they are completely ignored, but rather that they play no role in the result of the image processing, i.e., their content has no effect on the result of the image processing. Thus, instead of "pixels or subpixels to be processed," one can also speak of "pixels or subpixels to be utilized."

[0035] Furthermore, it has been shown that artificial intelligence entities (AIEs), especially artificial neural networks, have great difficulty ignoring the strong structuring due to the individual pixels of the individual fibers in order to capture the actual underlying image. The strong contrasts at the fiber edges are typically evaluated as very important by AIEs. Simple "templates" for preprocessing have also proven unsatisfactory, since the actual arrangement of the fibers and their fiber states almost always differ from a desired ideal state, and these differences also have a strong undesirable influence on AIEs.

[0036] According to some preferred embodiments, variants or refinements of embodiments, providing the fiber structure mask comprises generating a fiber position map which, for a plurality of sub-pixels, in particular for each sub-pixel, contains information about its illuminability by at least one fiber of the fiber bundle in the current state of the optical fiber bundle.

[0037] This illuminability depends on the current alignment of fiber bundles to pixels or sub-pixels (i.e., the arrangement state, in particular the translation, tilt, and / or twist state), as well as on the fiber states of the individual fibers. A fiber state of a fiber can, for example, be ideal (or nominal), broken, blind, or deformed, or the like. Providing the fiber structure mask advantageously includes generating the fiber structure mask based on the generated fiber position map, FPK.

[0038] A "map," such as a fiber position map (FPK), is a data structure that contains data for several, in particular all, pixels or subpixels of image data or an image sensor chip. In the case of a fiber position map (FPK), this data can include information about the illuminability of the subpixels.

[0039] The information about illuminability (or: illuminability information) can be binary, for example: - either a sub-pixel is fully illuminated (“1” / TRUE / HI) or not fully illuminated (“0” / FALSE / LO, where “not fully” includes both “partially” and “not at all”), or - either a sub-pixel is illuminated to a percentage above a threshold (“1” / TRUE / HI) or not (“0” / FALSE / LO, i.e. equal to or less than the threshold, where the case of illumination exactly equal to the threshold can be assigned to either of the two binary alternatives).

[0040] The information can also include percentage values ​​for each pixel or sub-pixel, which indicate the percentage to which the pixel or sub-pixel can be illuminated by the associated fiber (i.e., is illuminated when light is passed through the fiber). These percentage values ​​can, for example, be used later for a weighted evaluation of the sub-pixels. Thus, image processing can advantageously include a weighted evaluation of the pixels or sub-pixels based on the fiber position map, FPK. This can further improve the database on which later image processing and / or an image display to a user (e.g., surgeon) takes place. Preferably, only a single fiber is assigned to each pixel or sub-pixel, since these are usually significantly larger than the pixels and even more so than the sub-pixels, and thus the pixels or sub-pixels can practically only be illuminated by a single fiber, if at all.

[0041] According to some preferred embodiments, variants, or refinements of embodiments, the fiber structure mask is generated based on the fiber position map in such a way that the fiber structure mask assigns weights to individual pixels or sub-pixels (optionally including CFA filter color) to be used for further image processing. The weights can, in particular, assume a percentage value different from 0% and 100%. For example, depending on the specific application or implementation, the weighting can represent a selection, for example, from the values ​​{0%, 50%, 100%}, or from the values ​​{0%, 25%, 50%, 75%, 100%}, or even a selection from a continuous range of values ​​0%, 1%, ..., 99%, 100%.

[0042] According to some preferred embodiments, variants, or refinements of embodiments, the method further comprises determining a fiber state based on the fiber position map. A fiber state of a fiber can, for example, be ideal (or nominal), or broken, or blinded, or deformed, or the like. For example, a temporal progression of the information about the illuminability (i.e., the illuminability information) of all pixels or sub-pixels associated with a fiber can be recorded and analyzed, in particular with a constant arrangement state, e.g., displacement, tilt, and / or rotation state. If the arrangement state remains the same, but the illuminability information changes, typically negatively, i.e.If fewer subpixels are illuminated, or subpixels are illuminated less, this indicates that the fiber condition of the associated fiber has changed, particularly deteriorated. Pixels or subpixels associated with a fiber can be understood as those that, based on their geometric arrangement and current arrangement state, would theoretically be illuminated by the fiber if the fiber were in an ideal fiber condition.

[0043] Preferably, the fiber states of all fibers are determined based on the fiber position map. The information about the fiber states of all fibers as a whole provides a reliable picture of the condition of the optical system as a whole. Determining the fiber states, preferably of all fibers, can be performed, for example, as part of a quality control process for the optical system, for example, immediately after its manufacture.

[0044] The fiber position map can thus also provide a solid database for predictive maintenance. For example, the fiber position map can be used to monitor the number of unusable fibers (e.g., damaged, blind, or broken). If a predetermined threshold of unusable fibers is exceeded, a notification can be sent automatically, for example, suggesting replacement, automatically reordering, or similar.

[0045] According to some preferred embodiments, variants or refinements of embodiments, a temporal history of the determined fiber states is stored for the at least one fiber, preferably for all fibers.

[0046] According to some preferred embodiments, variants, or refinements of embodiments, an expected functional state of the optical system at a future point in time is determined based on the determined fiber state and / or the temporal progression of the determined fiber states of the at least one fiber, preferably all fibers. The functional state may, for example, be that the functional state of the optical system falls below a threshold, that the optical system is inoperable, and / or the like.

[0047] In this way, so-called predictive maintenance can be advantageously performed. Based on the detected functional status, a signal can be issued indicating, for example, a remaining service life, a need for replacement, a remaining number of uses, and / or the like. The signal can, for example, control a corresponding output device (e.g., a monitor or a loudspeaker) to issue a corresponding indication, for example, acoustically or visually.

[0048] According to some preferred embodiments, variants, or refinements of embodiments, the fiber structure mask is provided using a fiber structure mask generation artificial intelligence entity, FSMEKIE. This can be a trained artificial intelligence entity designed to receive images captured by optical systems with fiber bundles and image sensor chips as input data and, based thereon, to generate output data indicating a fiber structure mask associated with the respective input data. The present invention also encompasses a method for training such a fiber structure mask generation artificial intelligence entity, FSMEKIE, as will be explained in more detail below. The fiber structure mask generation artificial intelligence entity, FSMEKIE, can be, for example, an artificial neural network, ANN.

[0049] According to some preferred embodiments, variants or refinements of embodiments, providing the fiber structure mask comprises detecting the state (in particular arrangement state, ie displacement, tilt and / or twist state) by determining and analyzing aliasing artifacts in images acquired by means of the optical system, in particular based on artifacts due to the Shannon-Nyquist sampling theorem, as already described above with reference to Fig. 1 were described.

[0050] According to some preferred embodiments, variants, or refinements of embodiments, the method comprises outputting an indication signal indicating the condition. The indication signal can be output, for example, via an output device, e.g., via a display device or a loudspeaker of an imaging device that includes the optical system. Accordingly, the indication signal can be provided acoustically (e.g., via a voice output) and / or visually.

[0051] The indication signal can optionally also indicate a measure for correcting the state towards an optimal state. If the state is, for example, an arrangement state or has an arrangement state, a user, such as an operator or an employee of a production facility, can thus carry out a realignment (or adjustment, or calibration) in order to optimize the arrangement state. For example, a twist state can be optimized by further rotating the optical fiber bundle with respect to the image sensor. It can be provided that the optical system (or an imaging device comprising the optical system) has an adjustment mode (or calibration mode) in which the indication signal is output, while it is not output in another mode (e.g. an operating mode).

[0052] It can be provided that the indication signal (particularly in calibration mode) is issued periodically or continuously so that the user is informed in real time about the results of their actions to optimize the condition. In this way, the user receives information about the internal condition of the optical system and is guided, through intuitive human-machine interaction, to improve the performance of the optical system by optimizing the condition.

[0053] The optimal state (or target state) can be defined as one in which the number of illuminable pixels or subpixels, or the sum of the percentage illuminability values ​​of all pixels or subpixels, is maximized. An optimal state can also exist when only bright pixels are present in the shape of the fiber cross-section, and all other pixels are essentially black.

[0054] The optimal state for an optical system can change over time, for example due to aging. The optimal state can therefore also be determined during the process (or in a mode of the imaging device according to the invention). For example, the user (with or without appropriate instructions or human-machine interaction via a hint signal) can set various states, from which the one with the optimal properties (for example, with regard to the number of illuminable pixels or sub-pixels and / or the added percentage values ​​of the illuminabilities) is then automatically determined and defined as the optimal state.

[0055] Optionally, a control signal can also be output to automatically correct the detected state to an optimal state if the optical system has an actuator or the like for automatically changing the state in accordance with the control signal.

[0056] According to a third aspect, the invention provides a method for training a fiber structure mask generation artificial intelligence entity, FSMEKIE, comprising at least the following steps: Providing training data comprising images acquired by an optical system having an input optic, an image sensor chip, and an optical fiber bundle for transporting light incident on the input optic to the image sensor chip, wherein the training data further contains annotations indicating an associated fiber structure mask for each of the images; Providing an artificial intelligence entity initialized randomly or by pre-training, which is designed to receive images captured by such optical systems as input data and, based thereon, to generate output data indicating a fiber structure mask associated with the respective input data, which indicates pixels or sub-pixels whose output is to be processed for at least one subsequent image processing; and Training the provided initialized artificial intelligence entity with the provided training data to obtain the properties of a fiber structure mask generation artificial intelligence entity, FSMEKIE.

[0057] Training can be carried out, for example, using supervised learning (English: “supervised training”), where, put simply, differences between, on the one hand, output data based on training data and, on the other hand, the annotations belonging to the training data are penalized in a cost function that is iteratively minimized using a training algorithm.

[0058] According to some preferred embodiments, variants or refinements of embodiments, the images comprised by the training data (or: training images) comprise at least those images which: - were taken from a structureless, monochrome subject; and / or - have different states, in particular arrangement states (particularly preferably twist states), of the optical fiber bundle to the image sensor chip; and / or - were detected by different optical systems with different numbers of optical fibers in the optical fiber bundle; and / or - were acquired by different optical systems with different fiber thicknesses (or: fiber cross-sectional areas) and / or fiber shapes (round, square, octagonal...) of the optical fibers of the optical fiber bundle; and / or - were acquired by different optical systems with different resolutions; and / or - were captured by different optical systems with different sensor patterns (e.g. Bayer, RGEB, X-Trans etc.).

[0059] The structureless monochrome motif can, for example, be a uniform white (or blue, green, etc.) background.

[0060] Various twist states include, for example, twist states according to a C-symmetry, for example C3, C4, C6, or the like.

[0061] Furthermore, images may be included which, instead of the uniform background, have geometric patterns, for example one or more lines in different orientations, which can preferably also move across the background while the training images are captured.

[0062] For example, pre-training of the fiber structure mask generation artificial intelligence entity, FSMEKIE, can be performed with distorted geometric patterns.

[0063] According to some preferred embodiments, variants, or refinements of embodiments, providing the training data comprises automatically annotating at least some of the images acquired by the optical system. Preferably, the images are acquired in particular from black and white backgrounds (this step can also be part of embodiments of the third aspect of the present invention), for example, from black geometric patterns (lines, rectangles, triangles, circles, and / or ellipses) against a uniform, single-color, particularly white, background.

[0064] Automatic annotation can be performed using a threshold, with pixels or subpixels to be processed in the image processing system having values ​​of the threshold (on the side closer to the maximum illumination or illuminability, e.g., "fully white"), and pixels or subpixels not to be processed having values ​​of the threshold (on the side closer to the minimum illumination or illuminability, e.g., "fully black"). In this way, a very large amount of training data can be generated automatically using simple means and minimal human effort.

[0065] The present invention thus also provides, according to a fourth aspect, a method for generating training data, comprising: - capturing images of geometric patterns against a uniform, single-coloured, in particular white, background; and - Automatic annotation of the pixels or sub-pixels of the captured images, whereby pixels or sub-pixels to be processed in the image processing have values ​​of the threshold on the one hand, and pixels or sub-pixels not to be processed have values ​​of the threshold on the other hand.

[0066] The totality of the information about the pixels or sub-pixels to be processed thus represents a fiber structure mask annotating the respective captured image. The image data annotated in this way can therefore be used to train a fiber structure mask generation artificial intelligence entity, FSMEKIE, as also described herein.

[0067] Furthermore, according to a fifth aspect, the invention provides a computing device which is configured to carry out a method according to an embodiment of the present invention, in particular a method according to an embodiment of the second, third or fourth aspect of the present invention.

[0068] According to a sixth aspect, the invention provides a computer program product comprising executable program code which, when executed by a computing device, is configured to perform the method according to an embodiment of the second, third, and / or fourth aspect of the present invention.

[0069] According to a seventh aspect, the invention provides a non-transitory, computer-readable data storage medium comprising executable program code which, when executed by a computing device, is configured to perform the method according to an embodiment of the second, third, and / or fourth aspect of the present invention.

[0070] The non-volatile, computer-readable data storage medium may comprise or consist of any type of computer memory, in particular semiconductor memory, such as solid-state memory. The data carrier may also comprise or consist of a CD, a DVD, a Blu-ray disc, a USB memory stick, or the like.

[0071] According to an eighth aspect, the invention provides a data stream comprising executable program code or configured to generate executable program code which, when executed by a computing device, is configured to perform the method according to an embodiment of the second, third, and / or fourth aspect of the present invention.

[0072] Further advantageous variants, options, embodiments, and modifications will become apparent from the following figures, the detailed description, and the claims. It should be understood, however, that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given for illustrative purposes only, since various changes and modifications within the scope of the invention will become apparent to those skilled in the art. Short description of the characters

[0073] Individual embodiments of the present disclosure will be explained in detail with reference to the following figures. The components in the drawings are not to scale, but serve to illustrate the principles of the present invention. Parts in the various figures that correspond to the same elements or method steps have been provided with the same reference numerals in the figures. The numbering of method steps initially serves only to distinguish them and does not necessarily imply a corresponding order; however, it is a variant to perform the steps in the order of their numbering. Multiple steps can also be performed overlappingly or simultaneously. The figures show: Fig. 1 is a schematic diagram illustrating artifacts due to various twisting conditions between a light-conducting fiber bundle and an image sensor chip; Fig. 2 is a schematic diagram of an imaging device according to an embodiment of the present invention; Fig. 3a schematically shows an output side of an optical fiber bundle; Fig. 3b schematically shows an input side of an image sensor chip with pixels and sub-pixels; Fig. 4a is a schematic representation of how fibers and pixels as well as sub-pixels can be arranged and assigned to each other; Fig. 4b a schematic representation of a result of a super-resolution procedure; Fig. 5 is a schematic flowchart for explaining a method according to an embodiment of the present invention; Fig. 6 is a schematic flow diagram for explaining a method according to another embodiment of the present invention; Fig. 7 is a schematic block diagram for explaining a computer program product according to yet another embodiment of the present invention; and Fig. 8 is a schematic block diagram for explaining a data storage medium according to yet another embodiment of the present invention. Detailed description of the characters

[0074] Fig. Figure 1 shows a schematic diagram to explain artifacts due to different twisting states between a light-conducting fiber bundle and an image sensor chip, which has already been described above.

[0075] Here and below, the functioning of embodiments of the present invention is described, in particular, using the example of detecting (and, if necessary, optimizing, compensating, etc.) a twisting state as one possible example of a state. Twisting states are easily detectable and allow for a clear graphical representation for the present description. However, it is understood that instead of the twisting state (or in addition to the twisting state) in each example, another arrangement state can also occur, for example, a displacement or tilting state. Finally, other states can also be substituted as arrangement states in the explained embodiments.

[0076] Fig. 2 shows a schematic representation of an imaging device 300 with enhanced image processing according to an embodiment of the present invention.

[0077] The imaging device 300 comprises an optical system 100 with an input optics 110, an image sensor chip 130 having pixels, each of which consists of (up to) a plurality of sub-pixels, and an optical fiber bundle 120 for transporting light incident on the input optics 110 to the image sensor chip 130 via a plurality of fibers 121, for example, glass fibers, combined to form the optical fiber bundle 120. The image sensor chip 130 can also be a grayscale sensor (e.g., for infrared imaging, fluorescence imaging, or the like).

[0078] Fig. 3a schematically shows an output side (or interface) of the fiber bundle 120 facing the image sensor chip 130 (or: proximal, with respect to a user of the optical system, wherein the input optics 110 is arranged distally and the image sensor chip 130 is arranged proximally). The number of fibers 121 is significantly reduced here for the sake of better clarity, and their diameters and distances with respect to the outer circumference of the fiber bundle 120 are shown significantly enlarged.

[0079] Fig. 3b shows one of the Fig. 3a, the light-sensitive side of the image sensor chip 130 facing the output side of the fiber bundle 120, with an exemplary (but typical) checkerboard pattern of pixels 131, each of which consists of four sub-pixels 132-i arranged in a square and which are themselves also square. The image sensor chip 130 shown here as an example is constructed according to the Bayer system (or RGGB system), i.e., each pixel 131 consists of a red sub-pixel 132-1, two diagonally arranged green sub-pixels 132-2, 132-3, and a blue sub-pixel 132-4. The color sensitivity of the sub-pixels 132-i can be achieved using color filters, as described above.

[0080] By comparing Fig. 3a and Fig. 3b shows how the possible twist states (in Fig. 3a by a curved rotation arrow R) can produce completely different relationships between the fibers 121 and the pixels 131 and in particular sub-pixels 132-i.

[0081] Fig. Figure 4a shows a schematic diagram, again not to scale, illustrating approximately how the outputs (or, equivalently, cross-sections) of the various fibers 121 can overlap the individual pixels 131 or sub-pixels 132-i. Thus, each pixel 131 or sub-pixel 132-i can be assigned (or: belong to) (at most) one fiber 121. Fig. 4a also illustrates that some sub-pixels 132-i are only partially illuminable in the twisted state shown, while others are fully illuminable, and still others are not illuminable at all.

[0082] With reference to Fig. 2, the computing device 200 is now configured to provide a fiber structure mask based on a current twist state of the optical fiber bundle 120 to the image sensor chip, which fiber structure mask indicates pixels 131 or sub-pixels 132-i, the output of which is to be processed for at least one subsequent image processing.

[0083] The computing device 200 is also configured to perform image processing of pixels to be processed in an image 71 captured by the optical system 100, wherein the pixels to be processed correspond to the pixels 131 or subpixels 132-i displayed by the fiber structure mask. The image processing receives image data 71 from the optical system 100 and generates output image data 79 therefrom.

[0084] In the following, also based on Fig. 5, a method according to a further embodiment of the present invention is described, specifically a method according to an embodiment of the second aspect of the present invention, i.e., a computer-implemented method for processing images 71 of an optical system 100. The computing device 200 of the imaging device 300 can, in particular, be configured to carry out all steps of any embodiment of the second aspect of the present invention and can also be adapted according to any options, variants, and refinements described for the embodiments of the second aspect. However, it is understood that the method according to the invention can also be carried out independently of the imaging device 300.

[0085] In a step S100, a fiber structure mask is provided indicating pixels 131 or sub-pixels 132-i whose output is to be processed for at least one subsequent image processing operation. The fiber structure mask is provided in relation to a current state of the optical fiber bundle 120 with respect to the image sensor chip 130 (and thus directly or indirectly based on the current state). Here, too, the state is preferably an arrangement state, in particular (at least) a twist state, although other states or combinations of states are also usable.

[0086] The provision S100 of the fiber structure mask preferably takes place as part of a calibration or initialization of the optical system 100. If the method is carried out by the computing device 200 of the imaging device 300, the imaging device 300 can be set into a calibration mode for this purpose, for example.

[0087] Preferably, the provision S100 of the fiber structure mask occurs while the optical system 100 detects one or more predetermined test patterns, such as a pure white background, a white background with one or more black geometric patterns, or the like.

[0088] Providing S100 the fiber structure mask can be performed using a fiber structure mask generation artificial intelligence entity, FSMEKIE, which is configured to receive image data 71 of the optical system 100 as input data and, based thereon, to generate output data that displays (or encodes, or represents) the fiber structure mask. The fiber structure mask generation artificial intelligence entity, FSMEKIE, can have been trained, among other things, with the predetermined test patterns in annotated form.

[0089] Alternatively, the current state, for example, the twist state, can first be determined in a step S110, for example, by a first image processing based on image artifacts, for example, aliasing artifacts such as those resulting from the Shannon-Nyquist sampling theorem. These image artifacts are determined in image data 71, for example, raw image data, acquired by the optical system 100. In simple variants, the associated fiber structure mask can subsequently be read from a database (for example, a database of the imaging device 300).

[0090] Alternatively, information about the state determined in step S110, in particular an arrangement state, can be used to optimize the state, in particular the arrangement state, in a step S120. As already described above, an acoustic or visual indication signal can be output for this purpose in a step S121, for example, via a loudspeaker or a display device of the imaging device 300.

[0091] The optimization S120 of the state, in particular the arrangement state, can be carried out manually, for example by instructing the user through a human-machine interaction based on the indication signal to improve the state, in particular the arrangement state.

[0092] Alternatively, the optimization S120 of the state, in particular the arrangement state, can also be performed automatically if the optical system 100 has corresponding actuators. In this case, the optimization S120 of the state, in particular the arrangement state, can include generating and outputting corresponding control signals S122. It is understood that the optimal state always refers to an ideal state under the given circumstances.

[0093] The optimization S120 of the state preferably takes place iteratively with the determination S110 of the state until an optimal state is reached. Therefore, the "current state" will be referred to below, whereby it is understood that this current state is preferably a state that has been established after a corresponding optimization S120 of the state. Following the optimization S120 of the state, the focus of the optical system 100 can advantageously be adjusted, particularly if the optimization S120 of the state included optimizing an arrangement state, in particular a rotation state.

[0094] In a step S130, a fiber position map is generated in the current state. The fiber position map contains (or describes) for each sub-pixel 132-i information about its illuminability by at least one fiber 121 of the fiber bundle 120 (preferably: by exactly one fiber 121) in the current state. As shown in Fig. As can be seen in Figure 4a, due to the different geometries (round fiber 121, square sub-pixels 132-i), some sub-pixels 132-i are naturally only partially illuminated by the corresponding fiber 121.

[0095] As already explained in detail above, the fiber position map can include information of various types and qualities regarding the illuminability of the individual sub-pixels, such as binary information (illuminated: YES / NO) or percentage values. A fiber position map including the latter can also be referred to as a light transmission value map and, as also already explained, can be used in particular for evaluating the fiber states (or: the functionality) of the optical system 100 over time. The fiber position map can also be used for sub-pixel-specific intensity correction. In the case of inhomogeneous brightness imaging of the light guides, i.e., the fibers 121, a color shift can occur if individual sub-pixels (according to a scanning color filter grating, or: CFA) of a full pixel are irradiated with different intensities.Based on the fiber position map, automatic color correction (i.e., correction of color shift) can be performed via sub-pixel-specific intensity correction.

[0096] The information from the fiber position map (or light transmission value map) can be referenced back to individual fibers 121 via the respective current state. In this way, the information about the fiber states of each fiber 121 is retained even across different states (in particular, arrangement states). It is therefore advantageous to store the respective fiber position map together with the respective associated current state (in particular, arrangement state), for example, in a database of the imaging device 300.

[0097] Alternatively, the fiber position map with the fiber states can be converted to a target state using the current state and stored in this form, for example for predictive maintenance.

[0098] Based on the fiber position map, the fiber structure mask can now be generated in a step S140. This can, for example, only include those pixels 131 or sub-pixels 132-i that were fully illuminated according to the fiber position map. Alternatively, the fiber structure mask can be generated based on the fiber position map in such a way that it assigns weightings to individual sub-pixels 132-i to be used for further image processing. The weightings can, in particular, assume a percentage value different from 0% and 100%. The weightings can be rounded or correspond exactly to the detected illuminability values. For this purpose, for example, a white background with predefined properties (e.g., a specific shade of white, specific illumination, etc.) can be detected. It can then be determined which sub-pixel 132-i produces the strongest output (i.e.,, is best lit), this output is set to 100%, and the outputs of the other sub-pixels 132-i are then assigned appropriately scaled percentage values ​​as their weighting. Alternatively, an absolute scale can be used, according to which a target output is defined, and the outputs of all pixels indicate their weighting as a percentage of this target output.

[0099] Referring to the imaging device 300 and Fig. 2, the computing device 200 may include a fiber structure mask provision module 210, which is configured to provide the fiber structure mask, for example, as described above with reference to method steps S100 and in particular S110-S140. Thus, the computing device 200 may also be configured, in particular, to optimize the state S120, either manually through human-machine interaction S121 or automatically S122.

[0100] In a step S200, an image 71, in particular of a medical scene, is captured by means of the optical system 100. This can be done, in particular, after the calibration mode, for example, in an operating mode of the optical system 100.

[0101] In a step S300, image processing of pixels of the acquired image to be processed is performed, wherein the pixels to be processed correspond to the pixels 131 or sub-pixels 132-i displayed by the fiber structure mask.

[0102] The image processing S300 can in particular comprise interpolation, i.e., a method that, as seen by the human eye, fills the gaps between the pixels to be processed. The image processing preferably comprises a super-resolution method, particularly preferably a burst super-resolution method. As already explained, the restriction to the pixels to be processed results in a significant improvement in the initial database for the super-resolution method, which in turn significantly improves its result. In burst super-resolution, temporally preceding frames are used (e.g., 5-10, approximately 8), which preferably comprise minimal position changes (e.g., of 1-5 pixels) in order to increase the resolution of the output image data of the image processing method.

[0103] Fig. Figure 4b shows an example of the result of a super-resolution method using a fiber structure mask with weighting. Firstly, it is clear how, compared to the Fig. 4a, a significantly higher resolution is present due to the number and arrangement of the sub-pixels 132-i. For example, each sub-pixel 132-i has been divided into four sub-sub-pixels 133, resulting in a total resolution four times higher. These sub-sub-pixels 133 no longer correspond to actual structures of the image sensor chip 130, but are purely virtual constructs.

[0104] The outputs of the individual subpixels 132i were adjusted according to their weightings by the fiber structure mask: in the current state (especially the arrangement state), subpixels 132i that cannot be illuminated at all (and are therefore also not illuminated) are masked out with a weight of 0%, fully illuminated subpixels 132i are present with a weight of 100%, and only partially illuminated subpixels 132i are considered, for example, according to their weighting and / or their share in the Color Filter Array (CFA), with a corresponding pixel color shift. Instead, the positions of the masked subpixels 132i are filled using the super-resolution method. The result is a (here) 4x increased resolution, based on an excellent data basis.

[0105] Of course, further image processing steps known in the prior art can also be carried out after the image processing S300 or within the scope of the image processing S300.

[0106] Referring to the imaging device 300 and Fig. 2, the computing device 200 may comprise an image processing module 220 which is designed to perform the image processing S300 in order to generate the output image data 79, in particular using a super-resolution method, particularly preferably a burst super-resolution method.

[0107] Fig. Figure 6 shows a schematic flowchart for explaining a method according to an embodiment of the third aspect of the present invention, ie, a method for training a fiber structure mask generation artificial intelligence entity, FSMEKIE. In this method, at least the following steps are performed: In a step S10, training data is provided, which comprises images 71 acquired by an optical system 100 having an input optics 110, an image sensor chip 130, and an optical fiber bundle 120 for transporting light incident on the input optics 110 to the image sensor chip 130. The training data also contains annotations indicating an associated fiber structure mask for each image. The fiber structure mask can be configured as explained in detail above.

[0108] Providing S10 the training data may include capturing S11 the images 71 by means of the optical system 100 and / or automatically annotating S12 at least some of the images 71 captured by the optical system.

[0109] In step S11, the images 71 can be captured, in particular, from black and white backgrounds (e.g., white backgrounds with black geometric shapes). The automatic annotation S12 can be performed using a threshold value, specifically such that pixels 131 or subpixels 132-i to be processed in the image processing S300 have values ​​of the threshold value on the one hand, and pixels 131 or subpixels 132-i not to be processed have values ​​of the threshold value on the other.

[0110] In a step S20, an artificial intelligence entity initialized randomly or by pre-training is provided, which is designed to receive images 71 captured by such optical systems 100 as input data and, based thereon, to generate output data which indicate a fiber structure mask associated with the respective input data, which indicates pixels 131 or sub-pixels 132-i, the output of which is to be processed (or: utilized) for at least one subsequent image processing S300.

[0111] The step S20 may comprise a step S21 of pre-training (or: pre-training) the fiber structure mask generation artificial intelligence entity, FSMEKIE, wherein, for example, white backgrounds with black geometric figures may be used as pre-training data, the latter in particular being able to exhibit distortions.

[0112] In step S30, the provided initialized artificial intelligence entity is trained with the provided training data to obtain the fiber structure mask generation artificial intelligence entity, FSMEKIE. This can be done, for example, through supervised learning, as explained above.

[0113] Fig. Figure 7 shows a schematic block diagram of a computer program product 400 according to an embodiment of the sixth aspect of the present invention. The computer program product 400 comprises executable program code 450, which, when executed (e.g., by a computing device), is configured to perform the method according to an embodiment of the present invention, for example, according to Fig. 5 or Fig. 6, or according to any other embodiment of the second, third, or fourth aspect of the present invention.

[0114] Fig. Figure 8 shows a schematic block diagram of a non-transitory computer-readable data storage medium 500 according to an embodiment of the present invention. The data storage medium 500 comprises executable program code 550 which, when executed (e.g., by a computing device), is configured to perform the method according to an embodiment of the present invention, for example, according to Fig. 5 or Fig. 6, or according to any other embodiment of the second, third, or fourth aspect of the present invention.

[0115] The non-volatile computer-readable data storage medium 500 may, for example, be embodied as or comprise a semiconductor memory, e.g., an SSD memory chip. The data storage medium 500 may also comprise or comprise a CD, DVD, Blu-ray, or a magnetic storage device.

[0116] The above description of the disclosed embodiments merely contains examples of possible implementations described to enable a person skilled in the art to make or use the present invention. Various variations and modifications of these embodiments will be readily apparent to those skilled in the art, given knowledge of the present invention, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure.

[0117] Thus, the present invention is not intended to be limited to the specific embodiments shown herein, but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein. Therefore, the present invention is to be limited only in accordance with the following claims.

[0118] The invention can be roughly summarized as follows: depending on a current state (for example, an arrangement state such as a twist state) of an optical fiber bundle 120 with respect to an image sensor chip 130, a fiber structure mask is created. This indicates which pixels 131 or sub-pixels 132-i of the image sensor chip 130 are to be used for subsequent image processing S300. Advantageously, these are particularly well, or even completely, illuminated pixels 131 or sub-pixels 132-i. The image processing S300 can, for example, comprise a super-resolution method, particularly preferably a burst super-resolution method. Due to the output data prepared by the fiber structure mask for the image processing S300, the result is output image data 79 with high resolution and, at the same time, high quality. List of reference symbols 71 raw image data 79 Output image data 100 Optical System 110 Entrance optics 120 optical fiber bundles 121 fiber 130 image sensor chip 131 pixels 132-i subpixel 133 sub-sub-pixels 200 computing device 210 Fiber Structure Mask Provisioning Module 220 image processing module 300 imaging device 400 computer program product 450 program code 500 data storage media 550 program code R Rotation arrow S10..S300 Process steps

Claims

[1] Imaging device (300) with enhanced image processing (S300), comprising: an optical system (100) with an input optics (110), an image sensor chip (130) having pixels (131) each consisting of one or more sub-pixels (132-i), and an optical fiber bundle (120) for transporting light incident on the input optics (110) to the image sensor chip (130); and a computing device (200) which is configured to: - based on a current state of the optical fiber bundle (120) with respect to the image sensor chip (130), to provide a fiber structure mask indicating pixels (131) or sub-pixels (132-i) whose output is to be processed for at least one subsequent image processing (S300); and - to carry out image processing (S300) of pixels to be processed of an image (71) of a medical scene captured by the optical system (100), wherein the pixels to be processed correspond to the pixels (131) or sub-pixels (132-i) displayed by the fiber structure mask. [2] A computer-implemented method for processing images (71) of an optical system (100) comprising an input optics (110), an image sensor chip (130), and an optical fiber bundle (120) for transporting light incident on the input optics (110) to the image sensor chip (130), wherein the image sensor chip (130) comprises pixels (131) each consisting of one or more sub-pixels (132-i); wherein the method comprises at least the following steps: Providing (S100) a fiber structure mask indicating pixels (131) or sub-pixels (132-i) whose output is to be processed for at least one subsequent image processing (S300) to a current state of the optical fiber bundle (120) with respect to the image sensor chip (130); Capturing (S200) an image of a medical scene by means of the optical system (100); and Carrying out (S300) at least one image processing of pixels to be processed of the captured image (71), wherein the pixels to be processed correspond to the pixels (131) or sub-pixels (132-i) displayed by the fiber structure mask. [3] Method according to claim 2, wherein the image processing (S300) of the pixels of the captured image (71) comprises applying a super-resolution method to the pixels to be processed, in particular using a super-resolution artificial intelligence entity, SRKIE. [4] Method according to one of claims 2 or 3, wherein the provision (S100) of the fiber structure mask generating (S130) a fiber position map which contains, for each pixel (131) or sub-pixel (132-i), information about its illuminability by at least one fiber (121) of the fiber bundle (120) in the current state, and also generating (S140) the fiber structure mask based on the generated fiber position map. [5] Method according to claim 4, wherein the fiber structure mask is generated (S140) based on the fiber position map in such a way that it assigns weights to be used for the further image processing (S300) to individual pixels (131) or sub-pixels (132-i), wherein the weights can in particular assume a percentage value different from 0% and 100%. [6] A method according to claim 4 or 5, further comprising: Determining a fiber state of at least one fiber (121) based on the fiber position map. [7] Method according to claim 6, wherein a time course of the determined fiber states is stored for the at least one fiber (121). [8] A method according to claim 6 or 7, further comprising: Determining an expected functional state of the optical system (100) at a future point in time based on the determined fiber state and / or the temporal progression of the determined fiber states of the at least one fiber (121). [9] Method according to one of claims 2 to 8, wherein the providing (S100) of the fiber structure mask is carried out using a fiber structure mask generation artificial intelligence entity, FSMEKIE. [10] Method according to one of claims 2 to 9, wherein providing (S100) the fiber structure mask comprises detecting the state (S110) by determining and analyzing aliasing artifacts in images (71) acquired by means of the optical system (100). [11] Method according to one of claims 2 to 10, comprising outputting (S121) an indication signal indicating the state, and optionally a measure for correcting the state towards an optimal state, and / or outputting (S122) a control signal for automatically correcting the detected state towards an optimal state. [12] Computer-implemented method for training a fiber structure mask generation artificial intelligence entity, FSMEKIE, comprising: Providing (S10) training data comprising images (71) acquired by means of an optical system (100) having an input optic (110), an image sensor chip (130), and an optical fiber bundle (120) for transporting light incident on the input optic (110) to the image sensor chip (130), wherein the training data further contains annotations indicating an associated fiber structure mask for each of the images (71); Providing (S20) an artificial intelligence entity initialized randomly or by pre-training, which is designed to receive images (71) captured by such optical systems (100) as input data and, based thereon, to generate output data indicating a fiber structure mask associated with the respective input data, which indicates pixels (131) or sub-pixels (132-i) whose output is to be processed for at least one subsequent image processing (S300); and training (S30) the provided initialized artificial intelligence entity with the provided training data in order to obtain the fiber structure mask generation artificial intelligence entity, FSMEKIE. [13] Method according to claim 12, wherein the images (71) comprised by the training data comprise at least those images (71) which: - were taken from a structureless, monochrome subject; and / or - have different states of the optical fiber bundle (120) with respect to the image sensor chip (130); and / or - were detected by different optical systems (100) with different numbers of optical fibers (121) in the optical fiber bundle (120); and / or - were detected by different optical systems (100) with different fiber thicknesses and / or fiber shapes of the optical fibers (121) of the optical fiber bundle (120). [14] Method according to claim 12 or 13, wherein the provision (S10) of the training data comprises an automatic annotation (S12) of at least some of the images (71) acquired by means of the optical system (100), wherein the images (71) were acquired in particular from black and white backgrounds and the automatic annotation (S12) is carried out by means of a threshold value, wherein pixels (131) or sub-pixels (132-i) to be processed in the image processing have values ​​of the threshold value on the one hand, and pixels (131) or sub-pixels (132-i) not to be processed have values ​​of the threshold value on the other hand. [15] Computer program product (400) comprising executable program code (450) which, when executed, is configured to carry out the method according to any one of claims 2 to 14. [16] Non-transitory, computer-readable data storage medium (500) comprising executable program code (550) which, when executed, is adapted to carry out the method according to any one of claims 2 to 14.

Images