Method for processing image data using nonlinear scaling model and medical visual assistance system

Abstract

A method for obtaining and processing image data by using a medical visual assistance system including an endoscope and a monitor is proposed. The endoscope is configured to be inserted into a body cavity and includes an image capture device and a light emitting device. The method includes illuminating a field of view of the image capture device using the light emitting device, capturing image data using the image capture device, providing a nonlinear scaling model suitable for the body cavity, adjusting the image data by applying the nonlinear scaling model so as to form adjusted image data, and the method includes presenting the adjusted image data on the monitor.

Description

Technical Field

[0001] The present invention relates to an endoscope and, more particularly, to a method for obtaining and processing image data, a medical visual aid system, an endoscope forming part of a medical visual aid system, and a monitor forming part of a visual aid system. Background Art

[0002] Endoscopes are well-known devices for visually inspecting inaccessible places such as body cavities of the human body. Typically, an endoscope comprises an elongated insertion tube, with a handle at the proximal end of the elongated insertion tube from the operator's perspective, and a visual inspection device such as a built-in camera with an image sensor and a light source at the distal end of the elongated insertion tube. An endoscope is usually connected to a monitor so that the image captured by the camera is displayed when it is inserted into an object to be observed. Wires for the camera and light source (such as an LED) extend from the handle to the end at the distal end along the interior of the elongated insertion tube. Instead of an LED, the endoscope can also be optical fiber, in which case the optical fiber extends along the interior of the elongated insertion tube.

[0003] In order to be able to manipulate the endoscope inside the body cavity, the distal end of the endoscope may include a section with increased flexibility, for example, an articulated end portion allows the operator to bend this section. Typically, this is accomplished by tightening or loosening a pull wire, which also extends from the articulated end portion along the interior of the elongated insertion tube to a control mechanism of the handle. In addition, a working channel may extend from the handle to the end portion along the interior of the insertion tube, for example to allow removal of liquid from the body cavity or to allow insertion of surgical instruments, etc., into the body cavity.

[0004] In order to reduce the risk of cross contamination and avoid the cumbersome procedure of cleaning endoscopes after use, it is desirable to provide endoscopes designed for single use. In order to keep costs at a low level, single-use endoscopes are usually designed to have as few components as possible. However, it is still desirable to obtain the best possible image displayed on the screen. In complex reusable endoscopes, one way to ensure image quality is to provide a light source that fully illuminates the object to be observed, and the light intensity from the light source can even be automatically adjusted by analyzing the image captured by the camera. In single-use endoscopes, it is desirable to have a simple light source (such as LED), which can be arranged at the distal end without any optical components such as lenses, light guides or reflective elements to focus, shape or distribute the light emitted from the LED. Examples of this configuration are known from WO 14106511.

[0005] Although this configuration is desirable due to its simplicity of design, the light source may cause overexposure of portions of the object to be observed, thereby causing pixel saturation of the image sensor, resulting in loss of information about the observed object. As a result, the image displayed on the monitor will appear too bright in some areas and too dark in other areas in order to obtain the desired information about the object to be observed. This is particularly true when the endoscope is inserted into a tubular structure, such as a human lung.

[0006] In view of this, it is an object to provide a method and an endoscope system which improve the image quality in a simple and cost-effective manner. Summary of the invention

[0007] Therefore, the present invention preferably seeks to mitigate, alleviate or eliminate one or more of the above-mentioned defects and shortcomings in the art, either individually or in any combination, and at least solve the above-mentioned problems, for example by providing a method for processing image data obtained using a medical visual assistance system including an endoscope and a monitor according to a first aspect, wherein the endoscope is configured to be inserted into a body cavity and includes a light-emitting device and an image capture device for capturing image data, the method comprising providing a nonlinear scaling model suitable for the body cavity, adjusting the image data by applying the nonlinear scaling model so as to form adjusted image data, and thereby presenting the adjusted image data on the monitor.

[0008] The advantage is that by using a non-linear scaling model and adapting this model to the body cavity, an operator can easily and quickly analyze images presented on a monitor based on the adjusted image data, which in turn means improved health monitoring.

[0009] The method may further comprise applying an exposure setting of the image capture device such that the light does not saturate an area of ​​the image covering more than two adjacent pixels, preferably more than a single pixel.

[0010] The advantage is that information is not lost in overexposed areas of the image. The darkest areas of the image may become darker, but the information is still in the pixels. By applying a nonlinear scaling model, the information in the darkest areas will be visible to the user.

[0011] The method may further include applying an exposure setting that emphasizes a central portion of the field of view of the image capture device.

[0012] The advantage in the case of inspecting tube-formed cavities is that, by emphasizing the central portion, the image data reflecting this portion will include additional information, which in turn means that refinement of the image data (e.g. by using a non-linear scaling model) can be done at a later stage so that additional detail is visible to the operator.

[0013] Non-linear scaling models can also be adapted for monitors.

[0014] Different monitors may process image data differently, and thus by knowing which type of monitor is being used, the non-linear scaling model can be adapted accordingly, which in turn enables the operator to easily analyze the image presented on the monitor.

[0015] The non-linear scaling model can also be adapted for light emitting devices.

[0016] Different lighting devices may illuminate the body cavity differently.Thus, by knowing which type of lighting device the endoscope is using, the non-linear scaling model can be adapted, which in turn allows the operator to easily analyze the image presented on the monitor.

[0017] The non-linear scaling model may be a non-linear intensity scaling model, such as a non-linear gamma correction model.

[0018] A nonlinear intensity scaling model can be constructed to increase the contrast in dark portions of an image displayed on a monitor and to reduce the contrast in portions of the image with medium light intensities in a manner such that pixels with low pixel intensity values ​​are significantly enhanced and pixels with mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

[0019] Therefore, the nonlinear scaling model can provide a lower average gain to pixels with mid-range pixel intensity values ​​than the average gain provided by the standard gamma function, and provide the same average gain to pixels with low pixel intensity values ​​as the nonlinear scaling model. The standard gamma function is defined as:

[0020] V 输出 =V 输入 γ

[0021] This has the advantage that it appears as if the light source can illuminate both areas close to the endoscope tip and areas further away from the endoscope tip with the same light intensity.

[0022] Dark parts of the image may be defined as parts of the image having an intensity between 0% and 7% of the maximum intensity. Parts of the image with medium light intensity may be defined as parts of the image having an intensity between 8% and 30% of the maximum intensity.

[0023] A non-linear intensity scaling model may be a scaling function that maps input intensity to output intensity.

[0024] The scaling function may be provided with a bend, i.e. the slope of the scaling function may neither increase nor decrease continuously. In some embodiments, the scaling function has a first portion, followed by a second portion, followed by a third portion, and wherein the average slope of the second portion is lower than the average slope of the first portion and the average slope of the third portion.

[0025] This allows high gain to be provided to dark portions of the image and low gain to portions of the image with moderate light intensities, while utilizing the full dynamic range of the monitor.

[0026] The step of adjusting the image data by applying a nonlinear intensity scaling model so as to form the adjusted image data may further include increasing the intensity of a low-intensity image data subset and decreasing the intensity of a high-intensity image data subset, wherein the low-intensity image data subset includes image data having intensity levels up to 25% of the maximum intensity and wherein the high-intensity image data subset includes image data having intensity levels starting from 95% of the maximum intensity.

[0027] This has the advantage that an operator can easily analyse the adjusted image data representing a remote (relative to the image capture device) region of the body cavity.

[0028] The non-linear scaling model may be arranged to increase the intensity of a low intensity subset of image data by a first increase factor, wherein the first increase factor is greater than intensity factors for other subsets of the image data.

[0029] An advantage of increasing the intensity of the low image intensity image data subset to a higher degree than the rest of the image data is that an operator may easily analyze remotely located regions of the body cavity.

[0030] The step of providing a non-linear scaling model suitable for the body cavity may further include determining a body cavity type associated with the body cavity, and selecting the non-linear scaling model based on the body cavity type.

[0031] This has the advantage that differences in shape and light reflection properties of different body cavities can be taken into account, which in turn provides a non-linear scaling model that can be tailored to different body cavities, which in turn makes it possible to provide images that can be easily analyzed via a monitor for a wide range of different body cavities.

[0032] In some embodiments, the non-linear scaling model used to adjust the image data is selected from a group of non-linear scaling models including a first non-linear scaling model and a second non-linear scaling model.

[0033] In some embodiments, the set includes at least 3, at least 4, or at least 5 non-linear scaling models.

[0034] In some embodiments, both the first non-linear scaling model and the second non-linear scaling model are adapted for the same monitor.

[0035] In some embodiments, the image data is obtained using a single use endoscope, and wherein the monitor adjusts the image data by applying a non-linear scaling model.

[0036] In some embodiments, the first non-linear scaling model and the second non-linear scaling model are stored in the monitor.

[0037] According to a second aspect, a medical visual assistance system is provided, comprising an endoscope and a monitor, wherein the endoscope is configured to be inserted into the body cavity and comprises an image capture device and a light emitting device, and the monitor comprises: an image data processing device for adjusting image data by applying a nonlinear scaling model suitable for the body cavity so as to form adjusted image data; and a display device for presenting the adjusted image data.

[0038] The advantage is that by using a non-linear scaling model and adapting this model to the body cavity, an operator can easily and quickly analyze images presented on a monitor based on the adjusted image data, which in turn means improved health monitoring.

[0039] Furthermore, the exposure setting of the image data processing device may be configured to emphasize a central portion of the field of view of the image capture device.

[0040] The advantage in the case of inspecting tubular shaped cavities is that by emphasizing the central part, the image data reflecting this part will include additional information, which in turn means that refinement of the image data (e.g. by using a non-linear scaling model) can be done at a later stage so that additional details are visible to the operator.

[0041] The non-linear scaling model may also be adapted to a display device provided as a recipient of the adjusted image data.

[0042] Different monitors may process image data differently, and thus by knowing which type of monitor is being used, the non-scaling model can be adapted accordingly, which in turn enables the operator to easily analyze the image presented on the monitor.

[0043] The non-linear scaling model can also be adapted for light emitting devices.

[0044] Different lighting devices may illuminate the body cavity differently.Thus, by knowing which type of lighting device the endoscope is using, the non-scaling model can be adapted, which in turn enables the operator to easily analyze the image presented on the monitor.

[0045] The non-linear scaling model may be a non-linear intensity scaling model, such as a non-linear gamma correction model.

[0046] A nonlinear intensity scaling model can be constructed to increase the contrast in dark portions of an image displayed on a monitor and to reduce the contrast in portions of the image with medium light intensities in a manner such that pixels with low pixel intensity values ​​are significantly enhanced and pixels with mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

[0047] Therefore, the nonlinear scaling model can provide a lower average gain to pixels with mid-range pixel intensity values ​​than the average gain provided by the standard gamma function, and provide the same average gain to pixels with low pixel intensity values ​​as the nonlinear scaling model. The standard gamma function is defined as:

[0048] V 输出 =V 输入 γ

[0049] This has the advantage that it appears as if the light source can illuminate both areas close to the endoscope tip and areas further away from the endoscope tip with the same light intensity.

[0050] Dark parts of the image may be defined as parts of the image having an intensity between 0% and 7% of the maximum intensity. Parts of the image with medium light intensity may be defined as parts of the image having an intensity between 8% and 30% of the maximum intensity.

[0051] A non-linear intensity scaling model may be a scaling function that maps input intensity to output intensity.

[0052] The scaling function may be provided with a bend, i.e. the slope of the scaling function may neither increase nor decrease continuously. In some embodiments, the scaling function has a first portion, followed by a second portion, followed by a third portion, and wherein the average slope of the second portion is lower than the average slope of the first portion and the average slope of the third portion.

[0053] This allows high gain to be provided to dark portions of the image and low gain to portions of the image with moderate light intensities, while utilizing the full dynamic range of the monitor.

[0054] The image data processing device for adjusting image data by applying a nonlinear scaling model can be constructed to increase the intensity of a low-intensity image data subset and decrease the intensity of a high-intensity image data subset, wherein the low-intensity image data subset includes intensity levels up to 25% of the maximum intensity and wherein the high-intensity image data subset includes intensity levels starting from 95% of the maximum intensity.

[0055] This has the advantage that an operator can easily analyse the adjusted image data representing a remote (relative to the image capture device) region of the body cavity.

[0056] The non-linear scaling model may be arranged to increase the intensity of a low intensity subset of image data by a first increase factor, wherein the first increase factor is greater than intensity factors for other subsets of the image data.

[0057] An advantage of increasing the intensity of the low image intensity image data subset to a higher degree than the rest of the image data is that an operator may easily analyze remotely located regions of the body cavity.

[0058] The image data processing device may be further configured to determine a body cavity type associated with the body cavity, and select the non-linear scaling model based on the body cavity type.

[0059] This has the advantage that differences in shape and light reflection properties of different body cavities can be taken into account, which in turn provides a non-linear scaling model that can be tailored to different body cavities, which in turn makes it possible to provide images that can be easily analyzed via a monitor for a wide range of different body cavities.

[0060] In some embodiments, the non-linear scaling model used to adjust the image data is selected from a group of non-linear scaling models including a first non-linear scaling model and a second non-linear scaling model.

[0061] In some embodiments, both the first non-linear scaling model and the second non-linear scaling model are suitable for a display device.

[0062] In some embodiments, the image data is obtained using a single use endoscope.

[0063] In some embodiments, the first non-linear scaling model and the second non-linear scaling model are stored in the monitor.

[0064] According to a third aspect, there is provided an endoscope configured to be inserted into a body cavity. This endoscope comprises an image capture device and a light emitting device and forms part of the medical visual aid system according to the second aspect.

[0065] According to a fourth aspect, a monitor is provided, which includes: an image data processing device, which is used to adjust image data by applying a nonlinear scaling model suitable for a body cavity to form adjusted image data; and a display device, which is used to present the adjusted image data, and the monitor forms a part of the medical visual assistance system according to the second aspect.

[0066] According to a fifth aspect, there is provided a computer program comprising computer program code adapted to perform the method according to the first aspect when the computer program is run on a computer.

[0067] According to a sixth aspect, a method for obtaining and presenting image data by using a medical visual assistance system is provided, the medical visual assistance system comprising an endoscope and a monitor, wherein the endoscope is constructed to be inserted into a body cavity and comprises an image capture device and a light emitting device, the method comprising: illuminating the field of view of the image capture device using the light emitting device, capturing the image data using the image capture device, providing a nonlinear scaling model suitable for the body cavity, adjusting the image data by applying the nonlinear scaling model to form adjusted image data, and the method presenting the adjusted image data on the monitor. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The above and additional objects, features and advantages of the present invention will be better understood through the following illustrative and non-limiting detailed description of preferred embodiments of the present invention with reference to the accompanying drawings, in which:

[0069] Figure 1 An example of an endoscope is shown.

[0070] Figure 2 Shows that you can connect to Figure 1 An example of an endoscope monitor is shown in FIG.

[0071] Figure 3 is used to display Figure 2 Flowchart of the multiple steps of processing image data before the monitor presents the image data to the operator.

[0072] Figure 4 The tip of an endoscope is generally shown positioned inside a bronchus.

[0073] Figure 5 An example of an image captured by an endoscope depicting the interior of a bronchus is shown.

[0074] Figure 6 An example of a gamma correction model is shown, which in turn is an example of a non-linear scaling model according to the present invention.

[0075] Figure 7a Demonstrated in application Figure 6 An instance of image data before the gamma correction model.

[0076] Figure 7b Demonstrated in application Figure 6 An example of image data after the gamma correction model.

[0077] Figure 8 is a flow chart illustrating the steps of a method for processing image data prior to presentation on a monitor. DETAILED DESCRIPTION

[0078] Figure 1 An example of an endoscope 100 is shown. This endoscope may be suitable for single use. The endoscope 100 is provided with a handle 102, which is attached to an insertion tube 104 provided with a curved section 106. The insertion tube 104 and the curved section 106 may be provided with one or more working channels, so that instruments such as clamping devices can be inserted into the human body via the endoscope. One or more exit holes of the one or more channels may be provided in a terminal section 108 of the endoscope 100. In addition to the exit holes, a camera sensor (such as a CMOS sensor or any other image capture device) and one or more light sources (such as a light emitting diode (LED) or any other light emitting device) may be placed in the terminal section 108. By using Figure 2 The camera sensor and light source and monitor 200 shown in the figure are configured to display images based on the image data captured by the camera sensor, and the operator can see and analyze the inside of the human body to, for example, locate a position for collecting a sample. In addition, due to the visual feedback available through the camera sensor and monitor, the operator will be able to control the instrument in a precise manner. In addition, since some diseases or health problems may cause shifts in natural colors or other visual symptoms, the operator is equipped with valuable input for diagnosis based on the image data provided by the camera sensor and monitor.

[0079] In order to allow the operator to guide the camera sensor so that different fields of view can be obtained, the endoscope includes a bending section 106 that can be bent in different directions relative to the insertion tube 104. The operator can control the bending section 106 by using a knob 110 disposed on the handle 102. Figure 1 The handle 102 shown in FIG. 1 is designed so that the knob 106 is controlled by the operator's thumb, but other designs are possible. In order to control the clamping device or other device provided through the working channel, a button 112 can be used. Figure 1 The handle 102 shown in FIG. 1 is designed so that the operator's index finger is used to control the clamping device, but other designs are possible.

[0080] Image data captured by the camera sensor and optionally other data captured by other sensors placed in the tip can be transmitted to the Figure 2Even though line-based data transmission is shown, it is equally possible to transmit image data by using wireless data transmission.

[0081] The monitor device 200 is preferably a reusable piece of equipment. By having a disposable piece of equipment and a reusable piece of equipment, most of the data processing power can be placed in the reusable piece of equipment to achieve a cost-effective level while being safe to use from a health perspective.

[0082] Data processing operations closely related to the operation of, for example, a camera sensor (such as reading image data) may be performed in the endoscope itself, while more complex data processing operations requiring more computing power may be performed in the monitor 200. Since most of the more complex data processing operations are related to image data processing, an image signal processor (ISP) may be provided in the monitor and used for image data processing operations.

[0083] In order to be able to display an image on a monitor (e.g. an image depicting the interior of a body being examined) so that an operator can easily interpret it and draw conclusions from it, the image data captured by the image sensor can be processed in a number of steps. Figure 3 A series of steps 300 are shown in FIG. 1 , which may be performed in order to present an image in a manner that is easily usable by an operator.

[0084] In a first step 302, an endoscope (e.g. Figure 1 The endoscope shown in FIG. 1 captures image data comprising pixels, each pixel having at least one pixel intensity value.

[0085] In a second step 304, adjustments may be made using ISP settings related to, for example, exposure, in order to prepare for capturing image data that may be perfected at a later stage. If it is detected that the exposure settings are not adjusted correctly, for example because a large portion of the image data is overexposed and / or underexposed, the exposure settings may be changed. This step may be performed in the endoscope.

[0086] In a third step 306, the image data may be demosaiced, for example converted from the Bayer level color space to the RGB color space. For example, in this step, a so-called Freeman interpolation (described in US 4,774,565) may be performed in order to remove color artifacts in the image data. However, other algorithms for demosaicing may also be applied.

[0087] In a fourth step 308, a color temperature adjustment may be made. This adjustment may be based on user input. For example, the operator may have selected a color temperature scheme called "warm" that includes a warm yellowish color, and in this step the image data is adapted to conform to this color temperature scheme.

[0088] In a fifth step 310, in addition to or instead of color temperature adjustment, independent gains may be set for different color channels, which in most cases are red, green and blue (RGB). This gain setting may be based on user input.

[0089] In the sixth step 312, gamma correction may be performed. Gamma correction in this context should be understood as how much to increase or decrease pixel intensity values ​​in different spans to provide details in dark areas (i.e., low pixel intensity values) and bright areas (i.e., high pixel intensity values) that are visible to the operator. As will be described in further detail below and Figure 6 As further shown in , this step can be performed by using a non-linear scaling model (e.g., a non-linear gamma correction model). Using a non-linear gamma correction model has the effect that, for example, a first pixel having a low pixel intensity value (although above zero) is significantly amplified, i.e., the low pixel intensity value is significantly increased, while a second pixel having a mid-range pixel intensity value is only slightly adjusted or not adjusted at all. By using a non-linear scaling model in this manner, the details of the underexposed area become easier to analyze for an operator analyzing the image provided by the monitor 200.

[0090] The non-linear scaling model may preferably be applied in conjunction with control of the exposure setting of the image capture device so that overexposure and thereby pixel saturation is avoided. If there is an area where a large portion of the pixels are saturated, information on the actual structure in this area is lost and cannot be recovered. Preferably, the exposure setting is chosen so that the light does not saturate an area of ​​the image covering more than two adjacent pixels, preferably more than a single pixel.

[0091] Even though shown as being performed after the third step 306, in alternative embodiments, the gamma correction may be performed before the third step 306 associated with demosaicing.

[0092] In a seventh step 314, color enhancement may be performed. The result of the gamma correction performed in the sixth step 312 may be that the image data loses its color intensity. Therefore, by having the color enhancement performed after the gamma correction, the color intensity may be adjusted after the gamma correction and thus compensate for the effects caused by the gamma correction. The adjustment may be based on performing a saturation gain, and the choice of how much gain to use in this step may be based on the gamma level used in the previous step, i.e., the nonlinear gamma correction model used in the previous step if several alternatives are available. For example, if the gamma level used in the previous step provides only a slight deviation from a linear gamma correction model, there may be less need for color enhancement than if the gamma level used in the previous step deviates significantly from the linear gamma correction model.

[0093] In an eighth step 316, denoising / sharpening may be performed. This step may include both identifying noise caused by temporal effects, spatial effects, and signal level effects (eg, fixed pattern noise), and then removing this noise.

[0094] In a ninth step 318, the image data may be scaled to reach a preset size, if different from the current size.

[0095] In a tenth step 320, the image data may be converted from one form to another, in this particular example from a 10-bit format to an 8-bit format, before being saved in an eleventh step 322.

[0096] In a twelfth step 324, the image data may be converted from one form to another, in this particular example from a 10-bit format to a 6-bit format, before being displayed in a thirteenth step 326.

[0097] The third step 306 to the tenth step 320 , and the twelfth step 324 may be performed in a so-called FPGA (Field Programmable Gate Array) device provided in the monitor 200 .

[0098] Even if illustrated in a certain order, a different order is possible. In addition, one or several steps may be omitted if it is deemed unnecessary, for example based on the quality of the image data and / or the requirements of the image displayed on the monitor.

[0099] By way of example, Figure 4 Demonstrating that, when using endoscope 402 (similar to Figure 1When the endoscope 100 shown in FIG. 1 is used to examine a bronchus 400, the light sources 404a, 404b placed next to the camera sensor 406 may cause a first region 408 of the bronchus located nearby to be significantly illuminated by the light sources 404a, 404b, while a second region 410 located far away due to the tubular shape of the bronchus may not be illuminated by the light sources 404a, 404b at all, or at least less than the first region. Therefore, due to the tubular shape of the bronchus, the first region 408 may be overexposed, while the second region 410 may be underexposed.

[0100] Figure 5 shows an example of an image captured from a human bronchi, which clearly shows the Figure 4 The outer area 508 of the image (corresponding to Figure 4 The area 408 shown in the figure is overexposed, especially in the upper right corner, and the central area 510 (corresponding to Figure 4 The effect of having overexposed and underexposed areas is that the operator will not be able to analyze the portions of the bronchus corresponding to these areas.

[0101] To compensate for the body cavity (e.g. Figure 4 and Figure 5 The shape and reflective properties of the bronchi shown by example in Figure 1 can be modeled using a nonlinear gamma correction model, such as Figure 6 Demonstrated through examples.

[0102] like Figure 4 As shown, pixel saturation is typically a problem on portions of the image showing the near side wall in the channel (in which the endoscope is operated) near the tip of the endoscope. These areas tend to be overexposed. When looking deeper into the far end of the channel (i.e., the area farther from the tip of the endoscope and therefore farther from the light source), the inability to see darker details is a problem.

[0103] This problem can be solved by implementing the following method. When capturing image data, for example Figure 4 Under the conditions shown, the sensitivity of the pixels is adjusted to a level at which pixel saturation is unlikely to occur even in image portions with high light intensity. This adjustment can be performed by setting the exposure time and the gain (sensitivity) of the pixels of the image data.

[0104] This will have the consequence that pixels in parts of the image with lower light intensity will be even darker, and details in these parts will be indistinguishable when a user views the image like this. However, information about the details in these parts is still there, and by performing non-linear scaling (such as Figure 6), the image can be processed so that details in both bright and dark parts are distinguishable to the user.

[0105] You can refer to Figure 6 Non-linear scaling is described in detail in FIG. 1 , which shows how scaling from an original (input) image to a scaled (output) image is performed.

[0106] In a simple example, pixels can be scaled one by one, i.e. a pixel with value x in the input image will get value y in the output image. Figure 6 As shown in the black end towards the input axis, due to the steep slope of the curve in the darker areas, a small change in x will result in a fairly large change in y. This will both increase the contrast in the darker areas, and it will also increase the level of the dark areas, i.e. these areas will receive a higher light intensity.

[0107] Figure 6 The straight dashed line 600 in FIG. 6 is a one-to-one scaling, which will not change the image at all. As mentioned, the curve 602 will significantly increase the contrast in the dark parts of the image and make the information in the darkest areas visible to the user. At the same time, due to the relatively flat slope of the curve, the contrast in areas with medium light intensity is reduced. This has the purpose of not making areas that are already relatively bright too bright for the user to distinguish details in the image.

[0108] The effect of this non-linear contrast scaling is that the output image should preferably appear as if the light source illuminates areas close to the endoscope tip and areas further away from the endoscope tip with the same light intensity.

[0109] The exact shape of the curve will determine how the image will be scaled. In practice, slightly differently shaped curves will be provided and the user will select between different options in the user interface to use one of these curves. Alternatively, the selection of the curve may be made automatically, for example based on an initial analysis of the image from the body cavity being examined.

[0110] Figure 7a and Figure 7b By way of example, images are presented that depict the Figure 6 The bronchus before and after the gamma correction model is shown in . Figure 7a , which shows the central part 710a, 710b of the image, i.e. the part of the bronchus located relatively far from the camera sensor (which corresponds to Figure 4 The second area 410 in FIG. 4 is dark and therefore difficult for the operator to analyze. Figure 7bIn FIG. 7 , after the gamma correction model has been applied, these central parts 710 c , 710 d are not too dark, which has the positive effect that they can be easily analyzed.

[0111] Different body cavities are shaped in different ways. Therefore, by having different gamma correction models for different body cavities, image data can be easily converted so that an operator can more easily analyze this image data and make relevant conclusions based on the image data. For example, Figure 6 The gamma correction model presented in can be used for image data captured in a human bronchi. The body cavity being examined can be provided as a user input, for example, by a monitor, but it can also be automatically detected by the endoscope and / or the monitor itself (e.g., by analyzing image data obtained by an image capture device).

[0112] Transforming the image data by applying a non-linear gamma correction model requires data processing capabilities. Since the endoscope can be a single-use piece of equipment to ensure that the risk of cross-contamination between patients is eliminated, image data processing operations (such as applying the gamma correction model) can advantageously be performed in the monitor, which is a reusable piece of equipment to keep the overall operating costs low.

[0113] Figure 8 A flowchart 800 is generally shown, which illustrates a method for processing image data by using a medical visual assistance system including an endoscope and a monitor. Figure 1 and Figure 4 The endoscope shown in FIG. 1 is configured to be inserted into a body cavity and includes an image capture device and a light emitting device, and a monitor is configured to present image data, such as Figure 2 The method consists of four main steps.

[0114] In a first step 802, a field of view of an image capture device is illuminated using a light emitting device.

[0115] In a second step 804, image data is captured using an image capture device.

[0116] In a third step 806, a non-linear scaling model suitable for the body cavity is provided.

[0117] In a fourth step 808 , the image data is adjusted by applying a non-linear scaling model such that adjusted image data is formed.

[0118] Reference again Figure 1In order to reduce the influence of noise in the image data transmitted from the image sensor, it is known to shield the insertion tube cable placed inside the insertion tube 102 and the connecting cable 114 provided between the endoscope 100 and the monitor 200 in use. By doing so, the risk of having the image data be affected by signals from elsewhere can be reduced, so that there is less noise in the image data received by the monitor 200.

[0119] However, a disadvantage of shielding the insertion tube cable and the connecting cable is the increased production costs and also the increased environmental costs due to the need for additional materials. This is more relevant for single-use endoscopes than for other devices that are to be used multiple times.

[0120] Another disadvantage is that by shielding the insertion tube cable, the flexibility of the insertion tube 102 may be reduced, which may mean that additional energy is required to operate the bending section 106 and / or the range of the bending section 106 is limited due to the shielding.

[0121] Because at least a portion of the noise is periodic noise, or in other words fixed pattern noise, instead of shielding the insertion tube cable and / or connecting cable 114, the insertion tube cable and / or connecting cable 114 can be made unshielded or provided with reduced shielding, and the image data in the monitor 200 can be denoised to remove or at least reduce the noise in the image data.

[0122] If the image sensor is read out row by row at regular intervals, periodic noise (e.g., noise caused by the clock frequency) will be present in the image data with regularity, which may show more noise in a first direction than in a second direction. For example, image data that may be represented as a matrix with rows and columns may have stronger noise presence in the horizontal direction (i.e., column noise) than in the vertical direction (i.e., row noise).

[0123] Therefore, in order to provide a cost-effective single-use endoscope with improved flexibility, a method for denoising image data presented as follows may be utilized.

[0124] A method for denoising image data including first directional noise in a first direction and second directional noise in a second direction, wherein the first directional noise is greater than the second directional noise, the method comprising:

[0125] receiving image data including first pixels in a first direction and second pixels in a second direction from an image sensor disposed in the tip section 108 of the endoscope 100, and

[0126] A convolution is performed between the image data and a directionally biased periodic noise compensation kernel, wherein the directionally biased periodic noise compensation kernel is a convolution between an averaging kernel and a sharpening kernel, wherein the directionally biased periodic noise compensation kernel provides equal averaging of the image data in a first direction and a second direction, and provides sharpening in the second direction.

[0127] A directionally biased periodic noise compensation kernel may provide sharpening only in the second direction.

[0128] The first direction and the second direction may be perpendicular.

[0129] The endoscope 100 may include an insertion tube cable connected to an image sensor, wherein the insertion tube cable may be unshielded.

[0130] The endoscope 100 may include a connecting cable 114 connected to the insertion tube cable, wherein the connecting cable may be unshielded.

[0131] The step of performing a convolution between the image data and the directionally biased periodic noise compensation kernel may be performed in a monitor 200 connected to the endoscope 100 .

[0132] The direction-biased periodic noise compensation kernel can be:

[0133]

[0134]

[0135] The averaging kernel can be:

[0136] 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9

[0137] The sharpening kernel can be:

[0138] -1 -1 -1 -1 9 -1 -1 -1 -1

[0139] The invention has mainly been described above with reference to a few embodiments. However, as readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.

Claims

1. A method for processing image data obtained using a medical visual aid system, the medical visual aid system comprising an endoscope and a monitor, wherein the endoscope is configured to be inserted into a body cavity and comprises a light emitting device and an image capturing device for capturing image data, the method comprising: applying an exposure setting of the image capture device such that light does not saturate areas of the image covering more than two adjacent pixels, Providing a nonlinear scaling model suitable for the body cavity, adjusting the image data by applying the non-linear scaling model so as to form adjusted image data, presenting the adjusted image data on the monitor, The nonlinear scaling model is configured to establish a nonlinear relationship between the light intensity of an image captured using the image capture device and the light intensity generated by the image displayed on the monitor, wherein the nonlinear scaling model provides an average gain to pixels having mid-range pixel intensity values ​​that is lower than the average gain provided by a standard gamma function, and provides the same average gain to pixels having low pixel intensity values, so that pixels having low pixel intensity values ​​are significantly enhanced, and pixels having mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

2. The method according to claim 1, wherein: The exposure settings of the image capture device are applied so that the light does not saturate areas of the image covering more than a single pixel.

3. The method according to any one of the preceding claims, further comprising: An exposure setting is applied that emphasizes a central portion of the image capture device's field of view.

4. A method according to any one of the preceding claims, wherein: The non-linear scaling model is also adapted to the monitor.

5. A method according to any one of the preceding claims, wherein: The non-linear scaling model is also applicable to the light emitting device.

6. A method according to any one of the preceding claims, wherein: The nonlinear scaling model is a nonlinear intensity scaling model.

7. The method according to claim 6, wherein: The nonlinear intensity scaling model is constructed to increase the contrast in dark portions of the image and reduce the contrast in portions of the image with medium light intensities in the image displayed on the monitor in a manner such that pixels with low pixel intensity values ​​are significantly enhanced and pixels with mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

8. The method according to claim 7, wherein: The nonlinear intensity scaling model is a scaling function that maps input intensity to output intensity.

9. The method according to claim 8, wherein: The scaling function can be provided with a curvature.

10. The method according to claim 9, wherein: The scaling function has a first portion, the first portion is followed by a second portion, the second portion is followed by a third portion, and wherein an average slope of the second portion is lower than an average slope of the first portion and an average slope of the third portion.

11. The method according to any one of claims 6 to 10, wherein: The step of adjusting the image data by applying the non-linear intensity scaling model so as to form adjusted image data further comprises: increasing the intensity of a low intensity subset of image data, wherein the low intensity subset of image data includes image data having intensity levels up to 25% of the maximum intensity, and The intensity of a subset of high intensity image data is reduced, wherein the subset of high intensity image data includes image data having intensity levels starting at 95% of the maximum intensity.

12. The method according to claim 11, wherein: The non-linear scaling model is arranged to increase the intensity of the low intensity subset of image data by a first increase factor, wherein the first increase factor is greater than increase factors for other subsets of the image data.

13. A method according to any one of the preceding claims, wherein: The step of providing a non-linear scaling model suitable for the body cavity further comprises: determine the type of body cavity to which the cavity is associated, and The nonlinear scaling model is selected based on the body cavity type.

14. A method according to any one of the preceding claims, wherein: The non-linear scaling model used to adjust the image data is selected from a group of non-linear scaling models including a first non-linear scaling model and a second non-linear scaling model.

15. The method according to claim 14, wherein: Both the first non-linear scaling model and the second non-linear scaling model are suitable for the same monitor.

16. The method according to claim 15, wherein: The image data is obtained using a single use endoscope, and wherein the monitor adjusts the image data by applying the non-linear scaling model.

17. The method according to claim 16, wherein: The first non-linear scaling model and the second non-linear scaling model are stored in the monitor.

18. A medical visual aid system, the medical visual aid system comprising an endoscope and a monitor, wherein the endoscope is configured to be inserted into a body cavity and comprises an image capturing device and a light emitting device, and the monitor comprises: an image data processing device for adjusting the image data by applying a non-linear scaling model suitable for the body cavity so as to form adjusted image data, and a display device for presenting the adjusted image data, wherein the exposure settings of the image capture device are configured such that light does not saturate areas of the image covering more than two adjacent pixels, The nonlinear scaling model is configured to establish a nonlinear relationship between the light intensity of an image captured using the image capture device and the light intensity generated by the image displayed on the monitor, wherein the nonlinear scaling model provides an average gain to pixels having mid-range pixel intensity values ​​that is lower than the average gain provided by a standard gamma function, and provides the same average gain to pixels having low pixel intensity values, so that pixels having low pixel intensity values ​​are significantly enhanced, and pixels having mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

19. The medical visual aid system according to claim 18, wherein: The exposure setting of the image data processing device is configured to emphasize a central portion of the field of view of the image capture device.

20. The medical visual aid system according to any one of claims 18 to 19, wherein: The non-linear scaling model is also adapted to a display device provided as a recipient of the adjusted image data.

21. The medical visual aid system according to any one of claims 18 to 20, wherein: The non-linear scaling model is also applicable to the light emitting device.

22. The medical visual aid system according to any one of claims 18 to 21, wherein: The nonlinear scaling model is a nonlinear intensity scaling model.

23. The medical visual aid system according to claim 22, wherein: The nonlinear intensity scaling model is constructed to increase the contrast in dark portions of the image and reduce the contrast in portions of the image with medium light intensities in the image presented on the display device in a manner such that pixels with low pixel intensity values ​​are significantly enhanced and pixels with mid-range pixel intensity values ​​are only slightly adjusted or not adjusted at all.

24. The medical visual aid system according to claim 23, wherein: The nonlinear intensity scaling model is a scaling function that maps input intensity to output intensity.

25. The medical visual aid system according to claim 24, wherein: The scaling function is provided with a curvature.

26. The medical visual aid system according to claim 25, wherein: The scaling function has a first portion, the first portion is followed by a second portion, the second portion is followed by a third portion, and wherein an average slope of the second portion is lower than an average slope of the first portion and an average slope of the third portion.

27. The medical visual aid system according to any one of claims 22 to 26, wherein: The image data processing device for adjusting the image data by applying a non-linear scaling model is constructed to increase the intensity of a low-intensity image data subset and to decrease the intensity of a high-intensity image data subset, wherein the low-intensity image data subset includes intensity levels up to 25% of the maximum intensity and wherein the high-intensity image data subset includes intensity levels starting from 95% of the maximum intensity.

28. The medical visual aid system according to claim 27, wherein: The non-linear scaling model is arranged to increase the intensity of the low intensity subset of image data by a first increase factor, wherein the first increase factor is greater than increase factors for other subsets of the image data.

29. The medical visual aid system according to any one of claims 18 to 28, wherein: The image data processing device is further configured to determine a body lumen type associated with the body lumen and select the non-linear scaling model based on the body lumen type.

30. The medical visual aid system according to claim 29, wherein: The non-linear scaling model used to adjust the image data is selected from a group of non-linear scaling models including a first non-linear scaling model and a second non-linear scaling model.

31. The medical visual aid system according to claim 30, wherein: Both the first non-linear scaling model and the second non-linear scaling model are suitable for the display device.

32. The medical visual aid system according to claim 31, wherein: The image data was obtained using a single-use endoscope.

33. The medical visual aid system according to claim 32, wherein: The first non-linear scaling model and the second non-linear scaling model are stored in the monitor.

34. An endoscope configured to be inserted into a body cavity, the endoscope comprising an image capture device and a light emitting device and forming part of a medical visual aid system according to any one of claims 18 to 33.

35. A monitor, the monitor comprising: an image data processing device for adjusting the image data by applying a nonlinear scaling model adapted to the body cavity so as to form adjusted image data; and a display device for presenting the adjusted image data, and the monitor forms part of a medical visual aid system according to any one of claims 18 to 33.

36. A computer program product comprising computer program code adapted to perform the method of any one of claims 1 to 17 when said computer program product is run on a computer.

Images