Method and system for mitigating ghost spots and / or flares within a compact imaging camera system

Abstract

The invention relates to a computer-implemented method and a system for mitigating ghost spots and / or flares within a compact imaging camera system (1), wherein said system comprising optical components (10), an image sensor (12), and an image processor (20) configured to execute the method. The method being initiated upon activation of a photo capturing function (14) and comprises arranging the optical components (10) into a first, second and optional additional configurations (71, 72, 73) relative to the image sensor, capturing images (30, 32, 34) per each configuration, analysing the captured images (30, 32, 34) within the image processor (20) to identify the presence of ghost spots (77) and / or flares (78), and generating an output image (40) based on the analysis of the captured images reducing the presence of ghost spots (77) and / or flares (78) within the output image.

Description

[0001] 83712PC01

[0002] 1

[0003] METHOD AND SYSTEM FOR MITIGATING GHOST SPOTS AND / OR FLARES WITHIN A COMPACT IMAGING CAMERA SYSTEM

[0004] FIELD OF THE INVENTION

[0005] The present invention relates to a method and a system for mitigating ghost spots and / or flares within a compact imaging system. This is achieved by comparing two or more images captured with different configuration of optical components within the camera system thereby reducing or elimination ghost spots and / or flares.

[0006] BACKGROUND OF THE INVENTION

[0007] In the realm of optical systems, particularly within camera technology, two prevalent issues often compromise image quality: ghost spots and flares. Ghost spots, also known as ghost images, are unwanted artifacts that appear within the captured image due to internal reflections within the lens system. Flares, on the other hand, are scattered light within the lens system that washes out the image, reducing contrast and colour saturation.

[0008] These phenomena pose significant challenges within image capture, leading to degraded image quality and reduced user satisfaction. The primary method of mitigating these issues has been the implementation of anti- reflective (ARC) coatings. These coatings, applied to the lens surface, aim to minimize reflections and thus reduce the occurrence of ghost spots and flares.

[0009] However, the implementation of high-performance ARC coatings, particularly those with low reflectivity, is both costly and technically challenging. The highest performance ARCs, which can reduce reflectivity to below 0.5%, are not always feasible due to their high cost and complex application process. This is especially true for multilayer coatings on plastic, which are difficult to implement and significantly more expensive.

[0010] An alternative approach to reducing reflectivity comprises the use of nanostructures. However, achieving a reflectivity of less than 0.5% with nanostructures is a challenging task. 83712PC01

[0011] 2

[0012] Moreover, the infrared (IR) cut filter, typically applied on top of most sensors, needs to be optimized for IR cut efficiency. These filters are made of bulk material and multilayer stack coating, which limits the potential for ARC optimization. Consequently, IR cut filters often exhibit suboptimal ARC performance, with reflectivity ranging from 1% for the best filters to 2-3% for regular filters.

[0013] Hence, an improved method and system to reduce ghost spots and flares within camera systems, would be advantageous, and in particular a more efficient and / or reliable method and system enhancing image quality would be advantageous.

[0014] OBJECT OF THE INVENTION

[0015] It is an object of the present invention to provide a method and a system for reducing or mitigating ghost spots and flares within photographic images.

[0016] It is a further object of the present invention to provide an alternative to the prior art.

[0017] In particular, it may be seen as an object of the present invention to provide a method and a system that solves the above mentioned problems of the prior art with ghost spots and flares.

[0018] SUMMARY OF THE INVENTION

[0019] Thus, the above described object and several other objects are intended to be obtained in a first aspect of the invention by providing a computer implemented method for mitigating ghost spots and / or flares within a compact imaging camera system, wherein said system comprising optical components, an image sensor, and an image processor configured to execute the method; the method being initiated upon activation of a photo capturing function and comprising the following steps:

[0020] - arranging the optical components into a first configuration relative to the image sensor,

[0021] - capturing a first image, 83712PC01

[0022] 3

[0023] - arranging the optical components into a second configuration, different from the first configuration relative to the image sensor,

[0024] - capturing a second image,

[0025] - optional arranging the optical components into one or more additional configuration(s), different from the first and second configuration relative to the image sensor, and capturing an additional image per each additional configuration,

[0026] - analysing the captured images within the image processor to identify the presence of ghost spots and / or flares,

[0027] - generating an output image based on the analysis of the captured images reducing the presence of ghost spots and / or flares within the output image compared to the captured images, and

[0028] - storing the output image.

[0029] The method and system of the present invention provide an effective solution for mitigating ghost spots and / or flares within images captured by compact imaging camera systems, thereby improving the quality of the captured images. The invention is particularly beneficial for applications where high-quality image capture is essential, such as in professional photography, surveillance systems, and various scientific and industrial applications.

[0030] In a camera system, "ghost spots" or "flares" are typically caused by the reflection, refraction, or scattering of light within the camera system. This can occur when light enters the camera at certain angles, causing it to bounce around between the optical components and eventually hit the image sensor, generating unwanted artifacts within the final image.

[0031] The method comprises capturing two or more images, with the configuration of the optical components within the camera system being changed between each capture. This change of configuration changes the path of incoming light through the camera system. By doing so, the direction of the incoming light that generates the image, the "useful" light, and the incoming light that generates the ghost spots and / or flares, the "unwanted" light, are shifted relative to each other. Therefore, ghost spots or flares appear in different positions in each captured image. By combining these captured images, an output image is generated that 83712PC01

[0032] 4 either eliminates or significantly reduces ghost spots and flares and thereby mitigates ghost spots and flares.

[0033] This method is particularly beneficial in situations where the camera is facing a bright light source, such as the sun or a lamp, which can often cause significant ghost spots or flares. By allowing for the direction of incoming light to be shifted, the camera system can better handle these challenging lighting conditions and produce higher quality images.

[0034] The camera system comprises optical components, an image sensor, and an image processor configured to execute the method.

[0035] The method comprises various configurations of the optical components relative to the image sensor, capturing images within each configuration, and analysing these images within the image processor to identify and reduce the presence of ghost spots and / or flares. The method also includes the generation of an output image based on the analysis and the storage of this output image. The image processor controls the optical components to shift between different configurations.

[0036] "Reducing the presence of ghost spots and / or flares within the output image" is to be understood as that the ghost spots and / or flares present within the captured images are either removed or reduced within the output image. This results in the ghost spots and / or flares are mitigated.

[0037] That the ghost spot and / or flare is reduced may be understood as that within a grayscale version of the output image the brightness pixel value at the location of the ghost spots and / or flares will be lower than the corresponding pixel value within a grayscale version of the captured image wherein the ghost spot and / or flares were initially present.

[0038] That a ghost spot or flare is reduced may be detectable by generating a grayscale version of the output image and generating a grayscale version of the captured image, wherein the ghost spots and / or flares originally was present, and compare the brightness pixel value of corresponding pixels and detect that at least one 83712PC01

[0039] 5 pixel within the identified ghost spots and / or flares has a lower brightness pixel value within the output image than within the captured image wherein the ghost spots and / or flares originally was present.

[0040] The brightness pixel value ranges from 0 to 255, where 0 represents black, 255 represents white, and values in between represent varying shades of gray.

[0041] A "photo capturing function" is to be understood as the process or feature within a camera system that allows it to take or capture a photograph. This function is typically initiated when a user presses a button or triggers a command within the camera system. The function controls the camera's shutter, which opens to allow light to hit the image sensor, generating a photographic image.

[0042] The image processor may be configured to execute the method of mitigating ghost spots and / or flares within a compact imaging camera system. The image processor is analysing the captured images to identify the presence of ghost spots and / or flares. The image processor works by receiving two, or more, images captured under different configurations of the optical components. The image processor then analyses these images to identify and locate ghost spots and / or flares. Based on this analysis, the image processor generates an output image where the presence of these artifacts is reduced. The ghost spots and flares may be identified by the image processor analyses the captured images and identifies differences between the images, if an area is brighter within one captured image than within the other captured images, it may be a ghost spot or a flare.

[0043] A proposed algorithm to identify and correct ghost spots and / or flares involves the following steps:

[0044] - convert the images to grayscale images.

[0045] - align the grayscale images.

[0046] - identify ghost spots and flares. replace ghost spots and flares within the first image with corresponding pixels within the second image or additional images to create the output image.

[0047] - save the output image. 83712PC01

[0048] 6

[0049] Converting images to grayscale helps in accurately identifying and processing ghost spots and / or flares based on their intensity, without being affected by colour variations.

[0050] When two images are taken with slightly different configurations, there may be shifts or rotations between them. Aligning ensures that corresponding areas or pixels in both images match up correctly. Proper alignment helps in minimizing artifacts or distortions that can occur if the images are not perfectly overlaid, ensuring a seamless and natural-looking result. This step is crucial for accurately detecting and replacing ghost spots or flares, enhancing the overall quality of the output image.

[0051] Corresponding pixels or areas are pixels or areas in two images that align with each other, representing the same area or feature in both images.

[0052] Ghost spots and flares may be identified in the grayscale image by detecting pixels that are significantly brighter than their surroundings. For instance, in a grayscale image where image brightness pixel values range from 0 (black) to 255 (white), a threshold value may be 200. Pixel with a brightness pixel value higher than this threshold may be considered part of a ghost spot or flare.

[0053] Once identified, ghost spots and flares in the original first image are replaced with corresponding pixels from the second image or additional images. Hereby, the first image, now corrected, becomes the output image. The final step is to save the output image, which is free from ghost spots and flares.

[0054] An alternative method may be to use machine learning. A neural network may be trained using images with and without ghost sports and flares. The neural network may be trained with sets of first images, second images and possible additional images, and output images, and is trained so the machine learning algorithm is able to when receiving a set comprising a first image, a second image and possible additional images to generate an output image with ghost spots and flares being mitigated. 83712PC01

[0055] 7

[0056] The image processor may be a dedicated hardware component within the camera system, or it could be a software module running on a general-purpose processor. The specific configuration and implementation may depend on factors like the overall design of the camera system, power and performance requirements, cost considerations, etc. The image processor interacts with other components of the camera system, such as the optical components and the image sensor.

[0057] According to an embodiment, the direction of the image incoming light, generating the image, and the unwanted incoming light, generating the ghost spots and the flares, are shifted relatively to each other within the camera system, when shifting between two configurations.

[0058] The method also allows for the direction of image incoming light that generates the image and the unwanted incoming light that generates the ghost spots and / or flares to be shifted relative to each other within the camera system when shifting between two configurations.

[0059] The direction of the image incoming light, which generates the image, and the unwanted incoming light, which generates the ghost spots and flares, are shifted relative to each other within the camera system when shifting between two configurations. This relative shift of the direction of the incoming light and unwanted light effectively changes the path of the unwanted light, thereby changing the position of ghost spots and flares within the different captured images.

[0060] Shifting between two configurations is to be understood as shifting between the first configuration and the second configuration or shifting between the second configuration and an additional configuration or shifting between two different additional configurations.

[0061] It is to be understood that incoming light may be image incoming light and / or unwanted incoming light.

[0062] According to an embodiment, the optical components comprise at least one of a cover glass, an optical image stabilization lens, a folding prism, a mirror, a 83712PC01

[0063] 8 tuneable lens assembly comprising a transparent, deformable, non-fluid lens body and / or an IR filter.

[0064] The optical components may include a cover glass, an optical image stabilization lens, a folding prism, a mirror, a tuneable lens assembly comprising a deformable non-fluid lens body, an IR filter, or a combination thereof.

[0065] In the tuneable lens assembly, the transparent, deformable, non-fluid lens body is sandwiched between the first transparent substrate and either a second transparent substrate or a transparent back window, so that the tuneable lens assembly constitutes a lens having an optical axis intersecting the first transparent substrate and either the second transparent substrate or the transparent back window. The non-fluid lens body abuts the inwardly facing surfaces of the first transparent substrate and either the second transparent substrate or the transparent back window.

[0066] The optical axis may be defined as the axis which passes through centres of the first transparent substrate and either the second transparent substrate or the transparent back window and which is normal to a plane of one of the transparent substrates. The optical axis is further defined according to conventional optical definitions.

[0067] According to an embodiment, the optical components comprises a optical tuneable lens assembly, the optical tuneable lens assembly comprises a transparent, deformable, non-fluid lens body sandwiched between a first transparent substrate and a second transparent substrate, and wherein arranging the optical components into a first configuration comprises arranging the lens body into a first shape, and wherein arranging the optical components into a second configuration comprises arranging the lens body into a second shape different from the first shape changing the optical path of the incoming light.

[0068] The optical components may include a transparent, deformable, non-fluid lens body sandwiched between two transparent substrates. The arrangement of these components may be changed by an actuator system, which may include one or more piezoelectric actuators. The transparent substrates may be circular and 83712PC01

[0069] 9 there may be placed several actuators around the transparent substrate, for instance there may be 8 actuators.

[0070] The actuators may be linear displacement actuators, such as linear piezoelectric actuators or electromagnetic motors, piezoelectrically actuated cantilever actuators, shape memory alloys, linear screw drives, or linear voice-coil actuators, arranged to apply a displacement at several points. The actuators may be controlled by the image processor.

[0071] A unique aspect of this invention is the inclusion of an optical tuneable lens assembly within the optical components. This assembly comprises a transparent, deformable, non-fluid lens body sandwiched between a first and a second transparent substrate. The arranging of the optical components into different configurations comprises moving or forming the lens body into different shapes. This movement or forming changes the optical path of the incoming light passing through the lens body, thereby affecting the formation of the image and the presence of ghost spots and flares.

[0072] According to an embodiment, arranging the optical components into an additional configuration comprising arranging the lens body into an additional shape, different from the first shape and the second shape changing the optical path of the incoming light.

[0073] It is possible, if more than two images are required, to arrange the optical components into additional configurations, which may comprise arranging the lens body into additional shapes. These additional shapes are different from the first and second shapes. This change of the arrangement of the optical components changes the optical path of the incoming light, thereby affecting the formation of the image and the presence of ghost spots and flares. Within each additional configuration an additional image is captured.

[0074] This provides an enhanced level of flexibility and control over the image capturing process. By allowing for more than two shapes of the lens body, the method can adapt to a wider range of lighting conditions and scenarios, resulting in higher 83712PC01

[0075] 10 quality images with reduced visual artifacts. This makes the compact imaging camera system more versatile and effective in a broader range of applications.

[0076] According to an embodiment, the compact imaging camera system comprises an actuator system able to change the shape of the lens body.

[0077] An "actuator system" refers to a mechanical system that moves, shapes or controls the optical components within the camera system, specifically the lens body, in order to change the lens body's shape.

[0078] One or more actuators are arranged to generate a controllable bending and / or tilt of at least the first transparent substrate dependent on a control signal, where the control signal is obtainable from measurement signals.

[0079] Sensors may be arranged to provide the measurement signals so that the measurement signals are indicative of the bending and / or tilt of at least the first transparent substrate, The sensors may be an optical sensor and / or a deformation sensor.

[0080] With the actuators arranged to generate a controllable bending, tilt and / or shaping of at least the first transparent substrate it is understood that the actuators may alternatively or additionally be arranged to generate a controllable bending, tilt and / or shaping of the first and / or the second transparent substrate. One or more sensors may be arranged to provide the measurement signal so that the measurement signal is indicative of the bending, tilt or shape of the first and / or the second transparent substrate.

[0081] The actuators may be arranged to act on either the first or the second transparent substrate. It is also possible that the actuators are arranged to act on both the first and the second transparent substrate so that both transparent substrates are forced to bend by the action of the actuators, possibly so that actuators on either side are independently controllable, i.e. so that the displacement or force applied on one of the transparent substrates is controllable independent of the displacement or force applied on the other. 83712PC01

[0082] 11

[0083] The lens body is a part of the camera's optical system. It helps focus the incoming light onto the image sensor to form an image. By changing the shape of the lens body, the camera can change the path of the incoming light, which may help mitigate issues like ghost spots or flares.

[0084] The actuator system can include various types of actuators, such as piezoelectric actuators, which can precisely control the movement of the lens body. Piezoelectric actuators work by expanding or contracting in response to an applied electric voltage, allowing for very precise movements.

[0085] In this case, the actuator system changes the shaping of the lens body by applying force to the first transparent substrate and / or to the second transparent substrate. This changes the shape of these substrates, and thereby changes the shape of the lens body.

[0086] This ability to change the shaping of the lens body allows the camera system to adapt to different lighting conditions and image requirements, thereby improving the quality of the captured images and reducing the presence of ghost spots and / or flares.

[0087] According to an embodiment, the actuator system is able to change the shape of the lens body by applying force to the first transparent substrate and / or to the second transparent substrate thereby changing the shape of the first transparent substrate and / or the second transparent substrate and thereby changing the shape of the lens body.

[0088] The actuator system plays an important role in this method. It changes the shape of the lens body by applying force to the first transparent substrate and / or the second transparent substrate. This force application results in a change of the shape of these substrates, which in turn changes the shape of the lens body. This change of the lens body's shape affects the optical path of the incoming light, thereby influencing the formation of the image and the presence and location of ghost spots and flares.

[0089] The actuator system allows for high speed of actuation, such as in the range less than 5 milliseconds, for example between 0.5 and 2 milliseconds. 83712PC01

[0090] 12

[0091] According to an embodiment, the actuator system comprises one or more piezoelectric actuators.

[0092] In a specific embodiment, the actuator system comprises one or more piezoelectric actuators. These actuators convert electrical energy into precise linear displacement, thereby providing the force needed to change the shapes of the transparent substrates and the lens body.

[0093] According to an embodiment, the optical components comprise a wedge prism, a folding prism and / or a mirror, and wherein arranging the optical components into a first configuration comprises arranging the wedge prism, the folding prism, and / or mirror into a first position and arranging the optical components into a second configuration comprises arranging wedge prism, the folding prism, and / or mirror into a second position different from the first position changing the optical path of incoming light.

[0094] The optical components may also comprise a folding prism or mirror, and the arranging of these components within different configurations changes the optical path of incoming light. A folding prism is a prism that somehow changes its orientation or position within the different configurations to change the path of light.

[0095] The arranging of the optical components into different configurations, may comprise moving the folding prism and / or mirror to different positions, changes the optical path of the incoming light. This change affects the formation of the image and the presence and location of ghost spots and flares.

[0096] Specifically, arranging the optical components into a first configuration comprises arranging the folding prism or mirror into a first position. Arranging the optical components into a second configuration comprises arranging the folding prism or mirror into a second position, different from the first position. This change of position changes the optical path of the incoming light and thereby the location of ghost spots and flares. 83712PC01

[0097] 13

[0098] According to an embodiment, arranging the optical components into one or more additional configuration(s) comprises arranging the wedge prism, the folding prism, and / or the mirror into one or more additional positions different from the first position and second position, changing the optical path of incoming light.

[0099] The method allows for arranging the optical components into one or more additional configurations. This comprises arranging the prism or mirror into an additional position, different from the first and second positions. This further change of position changes the optical path of the incoming light.

[0100] According to an embodiment, the optical components comprise an optical image stabilization lens and wherein arranging the optical components into a first configuration comprises arranging the optical image stabilization lens into a first position, and arranging the optical components into a second configuration comprises arranging the optical image stabilization lens into a second position different from the first position changing the optical path of the incoming light.

[0101] The optical components may also comprise an optical image stabilization lens, and the arranging of these components into different configurations changes the optical path of incoming light. The method may comprise arranging the optical components into a first configuration, which comprises placing the optical image stabilization lens into a first position. In a second configuration, the optical image stabilization lens is arranged into a second position, different from the first position. This change of lens position changes the optical path of the incoming light, and thereby the location of ghost spots and flares.

[0102] An Optical Image Stabilization (OIS) lens is a type of lens usually used within cameras to reduce blurring associated with the motion of a camera during exposure. Specifically, it compensates for pan and tilt (angular movement, equivalent to yaw and pitch) of a camera or other imaging device.

[0103] The OIS lens works by moving lens elements to counteract camera movement. This movement is accomplished by small motors, magnets, or piezo elements physically moving a lens element or group of elements, thus affecting the light 83712PC01

[0104] 14 path before it reaches the sensor. However, the Optical Image Stabilization (OIS) lens may in this invention be used to change the direction of the image incoming light generating the image, and the unwanted incoming light generating the ghost spots and flares, so the optical path of the image incoming light and the unwanted incoming light are shifted relative to each other within the camera system when shifting between two configurations.

[0105] This method provides a flexible solution for enhancing image quality under various conditions by dynamically adjusting the optical path of incoming light through the strategic arranging of the optical image stabilization lens. This method offers a robust solution for reducing the impact of ghost spots and flares on the final image, thereby improving the overall image quality.

[0106] According to an embodiment, arranging the optical components into one or more additional configuration(s) comprises arranging the optical image stabilization lens into one or more additional positions different from the first position and the second position changing the optical path of the incoming light.

[0107] The method may further allow for the arranging of the optical components into one or more additional configurations. This may comprise arranging the optical image stabilization lens into additional positions, different from the first and second positions, further changing the optical path of the incoming light. An image may be captured per each additional configuration.

[0108] According to an embodiment, analysing the captured images to identify the presence of ghost spots and / or flares comprises identifying differences between the captured images.

[0109] The method may further comprise analysing the images to identify the presence of ghost spots and / or flares. This analysis is performed by identifying differences between images captured within various configurations of the optical components.

[0110] In colour images pixels are typically represented by three values corresponding to the Red, Green, and Blue (RGB) channels. Each channel can have values from 0 to 255, allowing for a wide range of colours. If one or more of these values differs 83712PC01

[0111] 15 between two captured images a difference is detected. For a difference to be detected it may be required that the difference in the value for a colour channel is at least a minimum value, this minimum value preferably may be at least 10, more preferably at least 15 and even more preferably at least 20.

[0112] According to an embodiment, analysing the captured images to identify the presence of ghost spots and / or flares comprises converting the captured images to grayscale image and identify pixels with a brightness pixel value higher than a threshold value, preferably the threshold value is 200, more preferably at least 210 and even more preferably at least 220.

[0113] When a brightness pixel value is higher than a threshold value, it is an indication that there may be a ghost spot or a flare. This may be verified by comparing the pixel to corresponding pixels in other captured images.

[0114] According to an embodiment, generating an output image comprises selecting parts of the images where there are no ghost spots and / or flares or where there are reduced ghost spots and / or flares and combine the parts to generate the output image.

[0115] The method may also comprise selecting parts of the images where there are no or reduced ghost spots and / or flares and combining these parts to generate the output image.

[0116] This innovative approach allows for the generation of high-quality images with reduced artifacts, representing a significant advancement in the field of optical image stabilization technology.

[0117] In a second aspect, the invention relates to a compact imaging camera system comprising optical components, an image sensor, and an image processor; the system is adapted to

[0118] - arrange the optical components into a first configuration relative to the image sensor,

[0119] - capture a first image, 83712PC01

[0120] 16

[0121] - arrange the optical components into a second configuration, different from the first configuration relative to the image sensor,

[0122] - capture a second image,

[0123] - optional arrange the optical components into one or more additional configuration(s), different from the first and second configuration relative to the image sensor, and capture an additional image per each additional configuration,

[0124] - analyse the captured images within the image processor to identify the presence of ghost spots and / or flares,

[0125] - generate an output image based on the analysis of the captured images wherein the presence of ghost spots and / or flares within the output image is reduced compared to the captured images, and

[0126] - store the output image.

[0127] In a third aspect, the invention relates to a computer program product being adapted to enable a computer system comprising at least one computer having data storage means in connection therewith to control an compact imaging camera system according to the second aspect of the invention, such as a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of the first aspect of the invention.

[0128] This aspect of the invention is particularly, but not exclusively, advantageous in that the present invention may be accomplished by a computer program product enabling a computer system to carry out the operations of the system of the second aspect of the invention when down- or uploaded into the computer system. Such a computer program product may be provided on any kind of computer readable medium, or through a network.

[0129] The individual aspects of the present invention may each be combined with any of the other aspects. These and other aspects of the invention will be apparent from the following description with reference to the described embodiments. 83712PC01

[0130] 17

[0131] BRIEF DESCRIPTION OF THE FIGURES

[0132] The method and system according to the invention will now be described in more detail with regard to the accompanying figures. The figures show one way of implementing the present invention and is not to be construed as being limiting to other possible embodiments falling within the scope of the attached claim set.

[0133] Fig. 1 illustrates a compact imaging camara system.

[0134] Fig. 2 illustrates a possible embodiment of the compact imaging camara system.

[0135] Fig. 3 illustrates an embodiment of the optical tuneable lens assembly.

[0136] Fig. 4 illustrates another embodiment of the optical tuneable lens assembly.

[0137] Fig. 5 illustrates an embodiment of the optical tuneable lens assembly comprising a transparent, deformable, non-fluid lens body.

[0138] Fig. 6a-c are sketches illustrating possible optical paths of the image incoming light and the unwanted incoming light within different configurations where the shape of the lens body is changed.

[0139] Fig. 7a-c are sketches illustrating possible optical paths of the image incoming light and the unwanted incoming light within different configurations where the arrangement of an optical component is changed.

[0140] Fig. 8a and 8b are examples of a first image captured by a first configuration and a second image captured by a second configuration.

[0141] Fig. 9 is an example of an output image generated by the image processor.

[0142] Fig. lOa-lOd show sketches of the first image, the second image, an additional image, and an output image.

[0143] Fig. 11 is illustrating an embodiment wherein the optical components comprise a wedge prism.

[0144] Fig. 12 is illustrating an embodiment wherein the optical components comprise a folding prism.

[0145] Fig. 13 is illustrating an embodiment wherein the optical components comprise a mirror.

[0146] Fig. 14 is a schematic system-chart representing the method according to the invention. 83712PC01

[0147] 18

[0148] DETAILED DESCRIPTION OF AN EMBODIMENT

[0149] Fig. 1 illustrates the compact imaging camara system 1, comprising optical components 10, an image sensor 12, a photo capturing function 14, an image processor 20 and the optical axis 150. There may be different optical components 10 within such a compact imaging camara system, like a lens stack 15, an optical tuneable lens assembly 18, an optical image stabilization lens 17, a wedge prism 60, a folding prism 61, a mirror 62, a cover glass 16, and / or an IR-filter 19, and therefore the optical components 10 in fig. 1 is illustrated by dashed squares, which may comprise different optical components. When the photo capturing function 14 is activated, which may be done by a user pushing a button, or it may be done automatically for instance by a timer preset to activate the photo capturing function, then the image processor may capture a first image, a second image, and optional additional images changing the configuration of the optical components between each image capture, hereby changing the optical paths of the incoming light within the camera system. The image processor 20 then analysis the captured images 30, 32, 34 to detect ghost spots or flares, and the image processor then generates an output image 40 based on the captured images to mitigate or reduce flares and ghost spots, so the flares and / or ghost spots on the output image are reduced or removed.

[0150] Fig. 2 illustrates a possible embodiment of the compact imaging camara system 1, wherein the optical components 10 comprise a cover glass 16, an optical image stabilization lens 17, an optical tuneable lens assembly 18, a lens stack 15, and an IR-filter 19. In this case the optical tuneable lens assembly comprises a transparent, deformable, non-fluid lens body 21, one, or more, piezoelectric actuators 45, a first transparent substrate 23 and a second transparent substrate 24. The lens stack 15 may comprise several different lenses.

[0151] Fig. 3 illustrates an embodiment of the optical tuneable lens assembly 18 comprising a transparent, deformable, non-fluid lens body 21, one, or more, piezoelectric actuators 45, a first transparent substrate 23, a transparent back window 25, and a support structure 26. In fig. 3 there is only one transparent substrate 23. 83712PC01

[0152] 19

[0153] Fig. 4 illustrates another possible embodiment of the optical tuneable lens assembly 18, also shown in fig. 2. The optical tuneable lens assembly 18 is comprising a transparent, deformable, non-fluid lens body 21, one, or more, piezoelectric actuators 45, a first transparent substrate 23, a second transparent substrate 24, placed opposite to the first transparent substrate, and a support structure 26. The actuator system 44 comprises the piezoelectric actuators 45, and possible other elements like wiring and power source (not shown) to activate the actuators 45.

[0154] Fig. 5 illustrates an embodiment of the optical tuneable lens assembly 18 comprising a transparent, deformable, non-fluid lens body 21, a first transparent substrate 23, and a transparent back window 25. The actuator system 44 comprises the actuators 51. The actuators 51 are displacement actuators 51 capable of generating a displacement in response to a control signal. Each of the actuators 51 may have a displacement element 52 arranged to displace in a direction parallel or substantially parallel with the optical axis 150. The one or more actuators 51 are arranged to generate forces on the first transparent substrate 23. The actuators may be mounted on a support structure 53.

[0155] Fig. 6a-c are sketches illustrating possible optical paths of the image incoming light 22 and the unwanted incoming light 24 within different configurations. The unwanted incoming light may be light from the sun or from a powerful lamp. Only some of the optical components are shown in fig. 6a-c; there may be several other optical components, but the figures are simplified for clarity.

[0156] Fig. 6a illustrates a first configuration 71, wherein the unwanted incoming light 24 is generating a ghost spot 77 on the image sensor 12, and the image incoming light 22 is generating an image on the image sensor 12. Here only one image incoming light beam 22 is shown, but of course there are image incoming light beams all over the image sensor. The shown image incoming light beam 22 follows the optical path 74 passing through the cover glass 16 and all the other optical components and hitting the image sensor 12. The unwanted incoming light beam 24 follows the optical path 75 passing through the cover glass 16, an optical image stabilization lens 17, the tuneable lens assembly 18, it is then reflected from the IR filter 19 and returns through the tuneable lens assembly 18, the lens 83712PC01

[0157] 20

[0158] 17 and the beam is again reflected by the cover glass 16, back through the different optical components and is eventually hitting the image sensor 12 generating a ghost spot 77 on the image sensor and thereby within the first image. The tuneable lens assembly 18 is arranged to arrange the lens body 21 into the first shape 84.

[0159] In fig. 6b the optical components are arranged into a second configuration 72, wherein the tuneable lens assembly 18 is arranged into a second shape 85 different from the first shape 84. Again, the unwanted incoming light 24 is generating a ghost spot 77 on the image sensor 12 after having been reflected by different optical components. The unwanted incoming light follows the optical path 75 passing through the different optical components and is reflected first from the IR filter 19 and then from the cover glass 16 to generate a ghost spot 77 on the image sensor, and thereby on the second image. The ghost spot is, as can be seen from fig. 6a and 6b, placed at a different position in fig. 6b than in fig. 6a. Also, as in fig. 6a, the image incoming light beam 22 follows the optical path 74 passing through the cover glass 16 and all the other optical components and hitting the image sensor 12. Due to the different configurations of the optical components, the image incoming light beam 22 hits the image sensor at a different position than in fig. 6a.

[0160] In fig. 6c the optical components are arranged into a third or additional configuration 73. The tuneable lens assembly 18 is arranged into an additional shape 86, different from the first and second shape. Thereby again, the unwanted incoming light 24 follows the optical path 75 passing through the different optical components and is reflected first from the IR filter 19 and then from the cover glass 16 to generate a ghost spot 77 on the image sensor 12, and thereby within the additional image. The ghost spot is, as can be seen, placed at a different position in fig. 6c than in fig. 6a and 6b. Also, as in fig. 6a and 6b, the image incoming light beam 22 follows the optical path 74 passing through the cover glass 16 and all the other optical components and hitting the image sensor 12. Due to the different configurations of the optical components, the image incoming light beam 22 hits the image sensor at a different position in fig. 6c than in fig. 6a and 6b. 83712PC01

[0161] 21

[0162] Fig. 7a-c are similar to fig. 6a-c with the difference that it is the position of the other optical components 10 that is changed, not the shape of the tunable lens assembly 21. Only some of the optical components are shown in fig. 7a-c, there may be several other optical components, but the figures are simplified for clarity.

[0163] In fig. 7a, the optical components are arranged into a first position 81, in fig. 7b the optical components are arranged into a second position 82, and in fig. 7c the optical components 10 are arranged into an additional position 83. The optical components may comprise an optical image stabilization lens 17, a folding wedge 60, folding prism 61, a mirror 62. That the optical components are in various positions in fig. 7a-c are illustrated by the optical components are represented by a dashed box of different sizes in the figures 7a-7c.

[0164] Fig. 8a and 8b are examples of a first image 30 captured when the optical components are arranged in the first configuration 71 and a second image 32 captured when the optical components are arranged in the second configuration 72. Ghost spots 77 are located at different positions within the two images 30, 32 showing that the optical path of the unwanted incoming light is different when capturing the image shown in fig. 8b relative to capturing the image shown in fig. 8a. Also, the items on the images are placed differently, for instance the boat in the river is placed at a higher position in fig. 8b than in fig. 8a illustrated by the white arrow showing that the optical path of the image incoming light is different when capturing the image shown in fig. 8b relative to capturing the image shown in fig. 8a. This also shows that the optical path of the image incoming light and the unwanted incoming light is shifted differently so the ghost spot is located differently within the two images relative to the items, for instance the boat on the river.

[0165] Fig. 9 is an example of an output image 40 generated by the image processor analyzing the two images 30, 32, identifying the ghost spots 77 and mitigating or reducing the ghost spots for instance by combining the two images to generate an output image with mitigated or reduced ghost spots. 83712PC01

[0166] 22

[0167] Fig. lOa-lOd show sketches to illustrate the invention. The sketches are of course not actually captured images but illustrate the method of the invention. Fig. 10a shows the first image 30 with a ghost spot 77 and a flare 78. Fig 10a is a sketch illustration an image which may be captured with the optical components 10 arranged in a first configuration. Fig. 10b shows a second image 32 with a ghost spot 77 and a flare 78. Fig. 10b is a sketch illustrating an image which may be captured with the optical components 10 arranged in a second configuration. Fig. 10c shows an additional image 34 with a ghost spot 77 and a flare 78. Fig. 10c is a sketch illustrating an image which may be captured with the optical components 10 arranged in an additional configuration. As can be seen from fig. lOa-c the position of the ghost spots and flares within the images are changed between each configuration of optical components as the unwanted incoming light generates the ghost spots and flares at different positions depending on the configuration of the optical components. Also, the items within the image, the house and the tree, are positioned at different positions within the images depending on the configuration of the optical components when the image incoming light takes different paths within the different configurations. But the shifts of the images are different from the shifts of the ghost spots and flares, so the ghost spots and flares are covering different parts of the items within the image. Fig. lOd shows the output image 40, where the ghost spots 77 and flares 78 has been mitigated by analysing the first image 30, the second image 32 and the additional image 34 to generate the output image.

[0168] Fig. 11 is illustrating an embodiment wherein the optical components 10 comprises a wedge prism 60. The wedge prism may be moved between the first position, the second position and additional positions respectively within the first configuration, the second configuration and additional configurations. In fig. 11 only one of the positions of the wedge prism is shown. The wedge prism may for instance be moved between different positions by being tilted to change the angle of incidence of the incoming light.

[0169] Fig. 12 is illustrating an embodiment wherein the optical components 10 comprises a folding prism 61. The folding prism may be moved between the first position, the second position and additional positions respectively within the first configuration, the second configuration and additional configurations. In fig. 12 83712PC01

[0170] 23 only one of the positions of the folding prism is shown. The folding prism may for instance be moved between different positions by being tilted to change the angle of incidence of the incoming light. The folding prism is illustrated as a triangle but may have any form suitable for deflecting the paths of the image incoming light and unwanted incoming light within different positions.

[0171] Fig. 13 is illustrating an embodiment wherein the optical components 10 comprises a mirror 62. The mirror may be moved between the first position, a second position and additional positions respectively within the first configuration, the second configuration and additional configurations. In fig. 12 only one of the positions of the mirror is shown. The mirror may be moved between different positions by being tilted to change the angle of incidence of the incoming light. The mirror is placed to reflect incoming light passing through the cover glass 16 to direct the image incoming light through the other optical components 10 to the image sensor 12. Unwanted incoming light may also be reflected by the mirror resulting in ghost spots or flares on the image sensor 12.

[0172] Fig. 14 is a flow chart illustrating the method of the invention. First in step SI, the photo capturing function is activated. Then in step S2, the image processor 20 is arranging the optical components 10 into a first configuration relative to the image sensor. Then in step S3 the compact imaging camara system captures the first image 31. In step S4 the image processor 20 is then arranging the optical components 10 into a second configuration relative to the image sensor. Then in step S5 the compact imaging camara system captures the second image 32. Then in step S6 the image processor determines whether more images are required. If more images are required, the method continues with step S7 with the image processor arranging the optical components 10 into an additional configuration and then capturing an additional image in step S8. Then the method returns to step S6 to decide whether another additional image is to be taken and continues in step S6-S8 as long as additional images are required. When no more additional images are required, the method continues to step S9, wherein the image processor is analyzing the captured images to identify ghost spots and flares within the images. Based on the analysis in step S9, the image processor in step S10 generates an output image based on the captured images reducing the presence of ghost spots and / or flares within the output images and then finally 83712PC01

[0173] 24 storing the output image in step Sil. The output image may be stored within an internal storage area within the compact imaging camera system, or it may be transmitted to external storage, for instance within an external database. The internal storage area may be a built-in memory chip, a hard drive, or a solid-state drive (SSD) depending on the specific design of the camera system. The output image, once captured by the camera, is processed, and then stored. This allows the image to be retrieved and viewed later.

[0174] The invention can be implemented by means of hardware, software, firmware or any combination of these. The invention or some of the features thereof can also be implemented as software running on one or more data processors and / or digital signal processors.

[0175] The individual elements of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way such as in a single unit, in a plurality of units or as part of separate functional units. The invention may be implemented in a single unit or be both physically and functionally distributed between different units and processors.

[0176] Although the present invention has been described in connection with the specified embodiments, it should not be construed as being in any way limited to the presented examples. The scope of the present invention is to be interpreted in the light of the accompanying claim set. In the context of the claims, the terms "comprising" or "comprises" do not exclude other possible elements or steps. Also, the mentioning of references such as "a" or "an" etc. should not be construed as excluding a plurality. The use of reference signs in the claims with respect to elements indicated in the figures shall also not be construed as limiting the scope of the invention. Furthermore, individual features mentioned in different claims, may possibly be advantageously combined, and the mentioning of these features in different claims does not exclude that a combination of features is not possible and advantageous.

Claims

83712PC0125CLAIMS1. A computer implemented method for mitigating ghost spots and / or flares within a compact imaging camera system (1), wherein said system comprising optical components (10), wherein the optical components (10) comprises a optical tuneable lens assembly (18), the optical tuneable lens assembly comprises a transparent, deformable, non-fluid lens body (21) sandwiched between a first transparent substrate (23) and a second transparent substrate (24), and wherein arranging the optical components (10) into a first configuration (71) comprises arranging the lens body (21) into a first shape (84), and wherein arranging the optical components (10) into a second configuration (72) comprises arranging the lens body (21) into a second shape (85) different from the first shape changing the optical path of the incoming light (22, 24); said system further comprising an image sensor (12), and an image processor (20) configured to execute the method; the method being initiated upon activation of a photo capturing function (14) and comprising the following steps:- arranging the optical components (10) into a first configuration (71) relative to the image sensor,- capturing a first image (30),- arranging the optical components (10) into a second configuration (72), different from the first configuration relative to the image sensor (12),- capturing a second image (32),- optional arranging the optical components (10) into one or more additional configuration(s) (73), different from the first and second configuration relative to the image sensor (12), and capturing an additional image (34) per each additional configuration,- analysing the captured images (30, 32, 34) within the image processor (20) to identify the presence of ghost spots (77) and / or flares (78),- generating an output image (40) based on the analysis of the captured images reducing the presence of ghost spots (77) and / or flares (78) within the output image compared to the captured images, and- storing the output image (40).83712PC01262. The method according to claim 1, wherein the direction of the image incoming light (22), generating the image (30, 32, 34), and the unwanted incoming light (24), generating the ghost spots (77) and the flares (78), are shifted relatively to each other within the camera system (1), when shifting between two configurations (71, 72, 73).

3. The method according to claim 1 or 2, wherein the optical components (10) comprise at least one of a cover glass (16), an optical image stabilization lens (17), a folding prism (60), a mirror (61), a tuneable lens assembly (18) comprising a transparent, deformable, non-fluid lens body (21) and / or an IR filter (19).

4. The method according to claim 1, wherein arranging the optical components (10) into an additional configuration (73) comprising arranging the lens body (21) into an additional shape (86), different from the first shape (84) and the second shape (85) changing the optical path of the incoming light (22, 24).

5. The method according to claim 1 or 4, wherein the compact imaging camera system (1) comprises an actuator system (44) able to change the shape of the lens body (21).

6. The method according to claim 5, wherein the actuator system (44) is able to change the shape of the lens body (21) by applying force to the first transparent substrate (23) and / or to the second transparent substrate (24) thereby changing the shape of the first transparent substrate and / or the second transparent substrate and thereby changing the shape of the lens body (21).

7. The method according to claims 5 or 6, wherein the actuator system (44) comprises one or more piezoelectric actuators (45).83712PC01278. The method according to any of the preceding claims, wherein analysing the captured images (30, 32, 34) to identify the presence of ghost spots (77) and / or flares (78) comprises identifying differences between the captured images.

9. The method according to any of the preceding claims, wherein generating an output image (40) comprises selecting parts of the images (30, 32, 34) where there are no ghost spots (77) and / or flares (78) or where there are reduced ghost spots (77) and / or flares (78) and combine the parts to generate the output image.

10. A compact imaging camera system (1) comprising optical components (10), an image sensor (12), and an image processor (20); the system is adapted to- arrange the optical components (10) into a first configuration (71) relative to the image sensor (12),- capture a first image (30),- arrange the optical components (10) into a second configuration (72), different from the first configuration relative to the image sensor (12),- capture a second image (32),- optional arrange the optical components (10) into one or more additional configuration(s) (73), different from the first and second configuration relative to the image sensor (12), and capture an additional image (34) per each additional configuration,- analyse the captured images (30, 32, 34) within the image processor (20) to identify the presence of ghost spots (77) and / or flares (78),- generate an output image (40) based on the analysis of the captured images wherein the presence of ghost spots and / or flares within the output image is reduced compared to the captured images, and- store the output image.

Images