Ambient light suppression

Abstract

According to an aspect, there is provided a system (200) comprising: an image projection unit (210) configured to project an illumination pattern onto at least a portion of a scene; an imaging unit (220) configured to capture a plurality of images of the scene while the illumination pattern is projected onto the scene; and a processing unit (230) configured to: demodulate the plurality of images based on the illumination pattern and with respect to a target section in the plurality of captured images, wherein the target section corresponds to one of: a portion of the scene onto which the illumination pattern is selectively projected while the plurality of images is captured, a portion of the scene that the projected illumination pattern is resolvable, a portion of the scene having pixel depth values that satisfy a predetermined range; and generate an ambient light rejection image of the scene based on the demodulation result.

Description

Technical Field

[0001] This disclosure relates to a system for performing ambient light suppression and a control method thereof. Background Technology

[0002] In the field of non-invasive measurement and monitoring, particularly in skin sensing for personal care and health applications, considerable development and investment have been made in exploratory activities within digital innovation. Currently known skin measurement systems guarantee the quantification and monitoring of skin characteristics, providing consumers with information related to changes that may be too small to detect, too faint to notice, and / or too slow to follow. For these systems to be consumer-acceptable, the sensing methods and systems must be sensitive and specific. Additionally, robustness of the measurement is essential for building consumer trust. A key issue associated with such imaging-based systems is the unpredictable and potentially variable ambient lighting when placed in uncontrolled environments (e.g., at home).

[0003] Modulation imaging techniques (such as spatial frequency domain imaging (SFDI)) primarily use projections of specific light patterns, phase-shifted sinusoidal patterns, to generate images that can be used, for example, to analyze skin properties. Three spatially modulated images with the same sinusoidal pattern but different phase shifts are sufficient to reconstruct the demodulated AC image, where all DC components of the light are excluded, thus removing ambient light. Demodulation requires three images of the object of interest, I1, I2, and I3, which are projections of sinusoidal patterns with the same spatial frequency, but each with a different phase shift. Phase difference The demodulation of the image can be represented by the following formulas (1) and (2):

[0004]

[0005] M DC = (I1+I2+I3) / 3 (2)

[0006] Where M AC It is the AC component of the image (which can be considered to correspond to modulated illumination), and M DC It is the DC component of the image (which can be considered as corresponding to ambient lighting).

[0007] As an example, Figure 1 The ambient light correction operation is illustrated through multiple images. Specifically, Figure 1Includes an initial image 110, multiple modulated images 120A, 120B, and 120C, and demodulated images 130 and 140, illustrating how ambient light can be corrected based on the multiple modulated images. In the context of this disclosure, the term "modulated image" can refer to an image depicting a modulation pattern projected onto a portion of a scene, and the term "demodulated image" can refer to an image that has undergone demodulation relative to the depicted modulation pattern.

[0008] In the initial image 110, the scene is illuminated only by ambient light, and the face is unevenly illuminated. Each of the modulation images 120A, 120B, and 120C is associated with a different phase shift. In this example, the first modulation image 120A is associated with a 0° phase shift, the second modulation image 120B with a 120° phase shift, and the third modulation image 120C with a 240° phase shift. The uneven illumination appearing in the initial image 110 can be corrected by demodulating the modulation images 120A, 120B, and 120C according to the above formulas (1) and (2) to generate AC component 130 and DC component 140. Specifically, the three sinusoidal patterns captured in the three modulation images 120A, 120B, and 120C are demodulated to obtain demodulated images 130 and 140, which correspond to AC component 130 (representing the alternating portion of the demodulated signal) and DC component 140 (representing the constant portion of the demodulated signal), respectively. In this case, when the scene is illuminated by both modulated lighting and ambient lighting from the projection, the AC component 130 corresponds to the modulated lighting, while when the scene is illuminated by both modulated lighting and ambient lighting, the DC component 140 of the image corresponds to the ambient lighting. Therefore, the AC component 130 can be considered as an "ambient light corrected / suppressed" version representing the scene depicted in the initial image 110.

[0009] Note that US Patent Application US 2019 / 0101383 A1 discloses a technique for determining an object using structured light to overcome ambient light effects. The technique according to US 2019 / 0101383 A1 utilizes structured light at various spatial frequencies.

[0010] It is also noted that Bodenschatz et al. discussed the use of structured light at different spatial frequencies for observation at different tissue depths in their paper “Diffuse optical microscopy for quantification of depth-dependent epithelial backscattering in the cervix” (Journal of biomedical optics (vol. 21, no. 6, June 1, 2016)). Summary of the Invention

[0011] In some smart mirror systems, imaging units (e.g., cameras) can be provided to record images and / or videos of the face or other parts of the user's body for skin analysis. One of the concerns associated with these systems is potential or perceived privacy intrusion. In currently known systems, the imaging units do not distinguish between the user and other elements in the background, and even if only the user's face is recorded, user identification remains relatively easy. For example, in currently available systems, blurring is used to hide image areas of no interest.

[0012] According to a first specific aspect, a system for performing ambient light suppression is provided, the system comprising: an image projection unit configured to project an illumination pattern onto at least a portion of a scene, wherein the illumination pattern is a time-varying spatial modulation pattern having a predetermined spatial frequency; an imaging unit configured to capture a plurality of images of a scene while the time-varying spatial modulation illumination pattern having a predetermined spatial frequency is projected onto the scene; and a processing unit configured to: demodulate the plurality of images based on the illumination pattern and relative to a target segment in the plurality of captured images, wherein the target segment corresponds to one of the following: a portion of the scene onto which the illumination pattern is selectively projected when the plurality of images are captured, a portion of the scene resolvable by the projected illumination pattern, a portion of the scene having pixel depth values ​​satisfying a predetermined range, wherein the target segment is part of the field of view of the imaging unit; and generating an ambient light suppressed image of the scene based on the demodulation result.

[0013] In some embodiments, the image projection unit may be configured to selectively project an illumination pattern only onto a selected portion of the field of view of the imaging unit, and the target segment corresponds to a portion of the scene on which the illumination pattern is selectively projected. In these embodiments, the processing unit may be configured to demodulate multiple images relative to the target segment such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the selected portion of the field of view of the imaging unit.

[0014] In some embodiments, the imaging unit may be configured to capture multiple images of a scene at a predetermined depth of focus, wherein a predetermined spatial frequency of the illumination pattern and a predetermined depth of focus of the imaging unit are selected such that the illumination pattern is resolvable only within a certain distance from the focal point of the imaging unit, and wherein a target segment corresponds to a portion of the scene that is resolvable by the projected illumination pattern. In these embodiments, the processing unit may be configured to demodulate multiple images relative to the target segment such that a generated ambient light suppressed image of the scene depicts only one or more elements included in the field of view within a certain distance from the focal point of the imaging unit.

[0015] In some embodiments, the processing unit may be configured to analyze multiple images to determine 3D depth information of a scene. The 3D depth information may include depth values ​​for each pixel of the multiple images, and the target segment may be based on the 3D depth information of the scene. In these embodiments, the processing unit may be configured to generate an ambient light suppressed image of the scene by outputting only the demodulation result relative to the target segment.

[0016] In some embodiments, the processing unit may be configured to determine the target segment by applying a phase mask to multiple images.

[0017] In some embodiments, the illumination pattern may include a phase-shifted sinusoidal pattern, and the imaging unit may be configured to capture multiple images with a predetermined phase difference relative to the phase of the sinusoidal pattern.

[0018] In some embodiments, the lighting pattern may also include at least one phase ramp with a predetermined step size.

[0019] In some embodiments, the imaging unit may be configured to capture three sets of images as the illumination pattern is projected onto the scene. In these embodiments, the first set of images may correspond to a 0° phase shift of the sinusoidal pattern, the second set of images may correspond to a 120° phase shift of the sinusoidal pattern, and the third set of images may correspond to a 240° phase shift of the sinusoidal pattern.

[0020] In some embodiments, the imaging unit may include a color camera, and each of the first set of images, the second set of images, and the third set of images may include a single image.

[0021] In some embodiments, each of the first, second, and third image groups may include three images. The first image in each of the three image groups may correspond to the red channel, the second image in each of the three image groups may correspond to the green channel, and the third image in each of the three image groups may correspond to the blue channel.

[0022] In some embodiments, the processing unit may be configured to demodulate multiple images to generate a first image corresponding to the AC component and a second image corresponding to the DC component. In these embodiments, the first image may be selected as an ambient light suppression image of the scene.

[0023] According to a second specific aspect, a method for controlling a system to perform ambient light suppression is provided. The system includes an image projection unit, an imaging unit, and a processing unit, and the method includes: projecting an illumination pattern onto at least a portion of a scene by the image projection unit, wherein the illumination pattern is a time-varying spatial modulation pattern; capturing multiple images of the scene by the imaging unit while the time-varying spatial modulation illumination pattern is projected onto the scene; and demodulating the multiple images by the processing unit based on the illumination pattern and relative to a target segment in the multiple captured images, wherein the target segment corresponds to one of the following: a portion of the scene on which the illumination pattern is selectively projected when the multiple images are captured, a portion of the scene on which the projected illumination pattern can be resolved, a portion of the scene having pixel depth values ​​satisfying a predetermined range; and generating an ambient light suppressed image of the scene by the processing unit based on the demodulation result.

[0024] In some embodiments, projecting an illumination pattern onto at least a portion of a scene may include selectively projecting the illumination pattern only onto a selected portion of the imaging unit's field of view. In these embodiments, the target segment may correspond to a portion of the scene on which the illumination pattern is selectively projected. Furthermore, in these embodiments, demodulation may be performed on multiple images relative to the target segment such that the resulting ambient light suppressed image of the scene depicts only one or more elements included in the selected portion of the imaging unit's field of view.

[0025] In some embodiments, the illumination pattern may have a predetermined spatial frequency, and multiple images of the scene may be captured at a predetermined depth of focus. In these embodiments, the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit can be selected such that the illumination pattern is resolveable only within a certain distance from the focal point of the imaging unit, and the target segment may correspond to the portion of the scene that the projected illumination pattern can resolve. Furthermore, demodulation may be performed on the multiple images relative to the target segment such that the resulting ambient light suppressed image of the scene depicts only one or more elements included in the field of view within a certain distance from the focal point of the imaging unit.

[0026] According to a third specific aspect, a computer program product including a computer-readable medium is provided, the computer-readable medium having computer-readable code contained therein, the computer-readable code being configured to cause the computer or processor, when executed by a suitable computer or processor, to perform the methods described herein.

[0027] These and other aspects will become apparent from the embodiments described below and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0028] Exemplary embodiments will now be described by way of example only with reference to the following figures, in which:

[0029] Figure 1 An example of ambient light correction operation is illustrated using multiple images;

[0030] Figure 2 A block diagram of a system according to one embodiment is shown;

[0031] Figure 3 A method for controlling a system to perform ambient light suppression according to one embodiment is shown;

[0032] Figure 4 An ambient light suppression operation using multiple images is illustrated according to one embodiment;

[0033] Figure 5 An ambient light suppression operation via multiple images is illustrated according to another embodiment;

[0034] Figure 6 An ambient light suppression operation via multiple images is illustrated according to another embodiment;

[0035] Figure 7 An ambient light suppression operation according to another embodiment is shown; and

[0036] Figure 8 The results of ambient light suppression operation according to one embodiment are shown compared with other imaging or processing techniques. Detailed Implementation

[0037] As described above, an improved system and control method for suppressing ambient light in each instance are provided, which solves the existing problems.

[0038] Figure 2 A block diagram of a system 200 according to one embodiment is shown, which can be used to perform ambient light suppression, particularly for ambient light suppression in images. Figure 2 As shown, the system includes an image projection unit 210, an imaging unit 220, and a processing unit 230.

[0039] Image projection unit 210 is configured to project a lighting pattern onto at least a portion of the scene. The lighting pattern is a time-varying spatial modulation pattern and may include a phase-shifted sinusoidal pattern. Additionally, in some embodiments, the lighting pattern may also include at least one phase ramp of a predetermined step size. In some embodiments, the lighting pattern may have a predetermined spatial frequency and / or a predetermined wavelength. Furthermore, in some embodiments, image projection unit 210 may be configured to project the lighting pattern in a manner focused on the object plane of the scene.

[0040] In some embodiments, the image projection unit 210 may be configured to selectively project an illumination pattern onto a selected portion of the field of view of the imaging unit.

[0041] Imaging unit 220 is configured to capture multiple images of the scene when a time-varying spatially modulated illumination pattern is projected onto the scene. Imaging unit 220 can be configured to capture three sets of images when the illumination pattern is projected onto the scene. In these three sets of images, the first set may correspond to a 0° phase shift of the sinusoidal pattern, the second set may correspond to a 120° phase shift of the sinusoidal pattern, and the third set may correspond to a 240° phase shift of the sinusoidal pattern.

[0042] In some embodiments, the imaging unit includes a color camera, and each of the first, second, and third image sets comprises a single image. In some alternative embodiments, each of the first, second, and third image sets may include three images. In these alternative embodiments, the first image in each of the three image sets may correspond to the red channel, the second image in each of the three image sets may correspond to the green channel, and the third image in each of the three image sets may correspond to the blue channel.

[0043] In some embodiments, the imaging unit 220 may be configured to capture multiple images of a scene at a predetermined depth of focus. In some embodiments where the imaging unit 220 is configured to capture multiple images of a scene at a predetermined depth of focus and the illumination pattern has a predetermined spatial frequency, the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit may be selected such that the illumination pattern is only resolvable within a certain distance from the focal point of the imaging unit 220.

[0044] As described above, in some embodiments, the illumination pattern may include a phase-shifted sinusoidal pattern. In these embodiments, the imaging unit 220 may be configured to capture multiple images with a predetermined phase difference relative to the phase of the sinusoidal pattern.

[0045] Processing unit 230 is configured to demodulate multiple images based on an illumination pattern and relative to a target segment in multiple captured images. The target segment corresponds to one of the following: a portion of the scene on which the illumination pattern is selectively projected during the capture of multiple images; a portion of the scene on which the projected illumination pattern is resolvable; or a portion of the scene having pixel depth values ​​within a predetermined range. Furthermore, processing unit 230 is also configured to generate an ambient light suppressed image of the scene based on the demodulation result.

[0046] Processing unit 230 can be configured to demodulate multiple images to generate a first image corresponding to the AC component and a second image corresponding to the DC component. The AC component may correspond to alternating portions of the demodulated signal associated with the multiple images, and the DC component may correspond to a constant portion of the demodulated signal. In these embodiments, the first image may be selected as an ambient light-suppressed image of the scene. It should be understood that, for example, the above references can be used... Figure 1 The formulas (1) and (2) shown are used to perform demodulation. It should also be understood that other formulas not explicitly discussed here can be used to obtain the AC and DC components of the signal, such as those associated with the Discrete Cosine Transform, Fourier Transform, etc.

[0047] As described above, in some embodiments, the image projection unit 230 may be configured to selectively project an illumination pattern only onto a selected portion of the field of view of the imaging unit. In these embodiments, the target segment may correspond to a portion of the scene on which the illumination pattern is selectively projected. Moreover, in these embodiments, the processing unit 230 may be configured to demodulate multiple images relative to the target segment, such that the resulting ambient light suppressed image of the scene depicts only one or more elements included in the selected portion of the field of view of the imaging unit 220.

[0048] As described above, in some embodiments where the imaging unit 220 is configured to capture multiple images of a scene at a predetermined depth of focus and the illumination pattern has a predetermined spatial frequency, the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit can be selected such that the illumination pattern is only resolvable within a certain distance from the focal point of the imaging unit 220. Additionally, in these embodiments, the target segment may correspond to a portion of the scene that is resolvable by the projected illumination pattern, and the processing unit 230 may be configured to demodulate multiple images relative to the target segment such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the field of view within a certain distance from the focal point of the imaging unit 220.

[0049] In some embodiments, processing unit 230 may be configured to analyze multiple images to determine three-dimensional (3D) depth information of a scene. The 3D depth information may include depth values ​​for each pixel of the multiple images. Additionally, in these embodiments, the target segment may be based on the scene's 3D depth information. Processing unit 230 may be configured to generate an ambient light-suppressed image of the scene by outputting only the demodulation result relative to the target segment.

[0050] Furthermore, in these embodiments, the processing unit 230 can be configured to determine the target segment by applying a phase mask to multiple images.

[0051] Typically, processing unit 230 can control the operation of system 200 and implement the methods described herein. Processing unit 230 may include one or more processors, processing units, multi-core processors, or modules configured or programmed to control system 200 in the manner described herein. In certain embodiments, processing unit 230 may include multiple software and / or hardware modules, each configured to perform or be used to perform one or more steps of the methods described herein.

[0052] Although not in Figure 2 As shown herein, but in some embodiments, system 200 may also include at least one user interface. Alternatively or additionally, at least one user interface may be external to system 200 (i.e., detached from or remote from system 200). For example, at least one user interface may be part of another device. The user interface may be used to provide information generated by the methods described herein to a user of system 200. Alternatively or additionally, the user interface may be configured to receive user input. For example, the user interface may allow a user of system 200 to manually input instructions, data, or information. In these embodiments, processing unit 230 may be configured to obtain user input from one or more user interfaces.

[0053] The user interface can be any user interface capable of presenting (or outputting or displaying) information to the user of system 200. Alternatively or additionally, the user interface can be any user interface that enables the user of system 200 to provide user input, interact with system 200, and / or control system 200. For example, the user interface may include one or more switches, one or more buttons, keypads, keyboards, touchscreens or applications (e.g., on tablets or smartphones), displays, graphical user interfaces (GUIs) or other visual presentation components, one or more speakers, one or more microphones or any other audio components, one or more lights, components for providing haptic feedback (e.g., vibration functionality), or any other user interface, or a combination of user interfaces.

[0054] In some embodiments, system 200 may include memory. Alternatively or additionally, one or more memories may be external to system 200 (i.e., separate from or remote from system 200). For example, one or more memories may be part of another device. Memory may be configured to store program code that can be executed by processing unit 230 to perform the methods described herein. Memory may be used to store information, data, signals, and measurements acquired or generated by processing unit 230 of system 200. For example, memory may be used to store multiple captured images, multiple candidate images, and / or ambient light suppressed images. Processing unit 230 may be configured to control memory to store multiple captured images, multiple candidate images, and / or ambient light suppressed images.

[0055] In some embodiments, system 200 may include a communication interface (or circuitry) for enabling system 200 to communicate with any interface, memory, and / or device, either internal or external to system 200. The communication interface may communicate wirelessly or via a wired connection with any interface, memory, and / or device. For example, the communication interface may communicate wirelessly or via a wired connection with one or more user interfaces. Similarly, the communication interface may communicate wirelessly or via a wired connection with one or more memories.

[0056] It should be understood that Figure 2 Only the components required to illustrate one aspect of system 200 are shown, and in a practical implementation, system 200 may include alternative or additional components to the components shown.

[0057] Figure 3 A method for controlling a system to perform ambient light suppression is illustrated. The illustrated method can generally be performed by system 200, and specifically, in some embodiments by or under the control of processing unit 230 of system 200. For illustrative purposes, reference will be made below. Figure 2 The various components of the system 200 are described. Figure 3 At least some of the boxes in the text.

[0058] refer to Figure 3 At frame 302, an illumination pattern is projected onto at least a portion of the scene. The projection can be performed by the image projection unit 210 of system 200, and can be performed in a manner focused on the object plane of the scene. The illumination pattern is a time-varying spatial modulation pattern, a phase-shifted sinusoidal pattern, and the imaging unit is configured to capture multiple images with a predetermined phase difference relative to the phase of the sinusoidal pattern. In some embodiments, the illumination pattern may have a predetermined spatial frequency and / or a predetermined wavelength. Alternatively or additionally, the illumination pattern may include at least one phase ramp of a predetermined step size.

[0059] In some embodiments, projecting a lighting pattern onto at least a portion of the scene at block 302 may include selectively projecting the lighting pattern onto a selected portion of the field of view of the imaging unit only.

[0060] Back Figure 3 At box 304, multiple images of the scene are captured, while at box 302, a time-varying spatially modulated lighting pattern is projected onto the scene. Therefore, at least in some embodiments, the steps shown in boxes 302 and 304 can be considered to be performed simultaneously. The capture of multiple images can be performed by the imaging unit 220 of system 200.

[0061] As described above, in some embodiments, the illumination pattern may include a phase-shifted sinusoidal pattern. In these embodiments, multiple images may be captured at frame 304 with a predetermined phase difference relative to the phase of the sinusoidal pattern.

[0062] In some embodiments, at box 304, three sets of images are captured while the lighting pattern is projected onto the scene. In these embodiments, the first set of images may correspond to a 0° phase shift of the sinusoidal pattern, the second set of images may correspond to a 120° phase shift of the sinusoidal pattern, and the third set of images may correspond to a 240° phase shift of the sinusoidal pattern. In some of these embodiments, each of the first, second, and third sets of images may comprise a single image. Alternatively, each of the first, second, and third sets of images may comprise three images—the first image in each set may correspond to the red channel, the second image in each set may correspond to the green channel, and the third image in each set may correspond to the blue channel.

[0063] return Figure 3 At box 306, the multiple images captured at box 304 are demodulated based on the illumination pattern and relative to a target segment among the multiple captured images. Demodulation at box 306 can be performed by the processing unit 230 of system 200. The target segment corresponds to one of the following: a portion of the scene onto which the illumination pattern is selectively projected when capturing multiple images; a portion of the scene onto which the projected illumination pattern is resolvable; or a portion of the scene having pixel depth values ​​that satisfy a predetermined range.

[0064] Back Figure 3 At box 308, an ambient light suppression image of the scene is generated based on the demodulation result. The generation of the ambient light suppression image can be performed by the processing unit 230 of system 200.

[0065] In some embodiments, demodulating multiple images at block 306 can produce a first image corresponding to the AC component and a second image corresponding to the DC component. The AC component may correspond to alternating portions of the demodulated signal associated with the multiple images, and the DC component may correspond to a constant portion of the demodulated signal. In these embodiments, at block 308, the first image can be selected as an ambient light-suppressed image of the scene. It should be understood that, for example, the above references can be used... Figure 1 The formulas (1) and (2) shown are used to perform demodulation. It should also be understood that other formulas not explicitly discussed here can be used to obtain the AC and DC components of the signal, such as those associated with the Discrete Cosine Transform, Fourier Transform, etc.

[0066] As described in reference box 302 above, in some embodiments, projecting an illumination pattern onto at least a portion of the scene at box 302 may include selectively projecting the illumination pattern onto a selected portion of the imaging unit's field of view only. In these embodiments, the target segment may correspond to a portion of the scene on which the illumination pattern is selectively projected, and demodulation of multiple images relative to the target segment may be performed at box 306 such that the ambient light suppressed image of the scene generated at box 308 depicts only one or more elements included in the selected portion of the imaging unit's field of view.

[0067] As described above, in some embodiments, the illumination pattern may have a predetermined spatial frequency. In these embodiments, capturing multiple images of the scene at frame 304 may be performed with a predetermined depth of focus. In these embodiments, the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit may be selected such that the illumination pattern is only resolvable within a certain distance from the focal point of the imaging unit 220 of the system 200. Additionally, the target segment may correspond to a portion of the scene that the projected illumination pattern can resolve. Moreover, in these embodiments, demodulation may be performed on multiple images relative to the target segment at frame 306 such that the ambient light suppressed image of the scene generated at frame 308 depicts only one or more elements within the field of view included within a certain distance from the focal point of the imaging unit 220 of the system 200.

[0068] although Figure 3 Not shown, but in some embodiments, the method may further include analyzing multiple images to determine 3D depth information of the scene. The 3D depth information may include depth values ​​for each pixel of the multiple images, and in these embodiments, the target segment may be based on the 3D depth information of the scene. In this regard, the method may also include determining the target segment by applying a phase mask to the multiple images captured at box 304.

[0069] Furthermore, in these embodiments, generating an ambient light suppression image of the scene at box 308 can be performed by simply outputting the demodulation result relative to the target segment.

[0070] Figure 4 An ambient light suppression operation using multiple images is illustrated according to an embodiment. The illustrated operation can generally be performed by system 200, and specifically, in some embodiments by or under the control of processing unit 230 of system 200. For illustrative purposes, reference will be made below. Figure 2 The various components of the system 200 are described. Figure 4 At least a portion of the operations shown.

[0071] like Figure 4 As shown, multiple modulation images 410A, 410B, and 410C, an AC component image 420, and a DC component image 430 are provided. The multiple modulation images include a first modulation image 410A, a second modulation image 410B, and a third modulation image 410C. The AC component image 420 may correspond to the alternating portion of the demodulated signal, while the DC component image 430 may correspond to the constant portion of the demodulated signal.

[0072] As can be seen from this embodiment, in each of the first modulation image 410A, the second modulation image 410B, and the third modulation image 410C, the illumination pattern is selectively projected onto a selected (rectangular) portion by the image projection unit 210 of the system 200. The illumination pattern is a time-varying spatial modulation pattern. Specifically, in this embodiment, the illumination pattern consists of multiple alternating horizontal dark and bright bands.

[0073] In this embodiment, the selected portion of the illumination pattern projected onto it corresponds to a portion of the field of view of the imaging unit 220 of the system 200, rather than the entire field of view of the imaging unit 220. Furthermore, each modulated image in this embodiment has been captured at different time points corresponding to different phase shifts of the projected illumination pattern. In the first modulated image 410A, the illumination pattern has a 0° phase shift; in the second modulated image 410A, the illumination pattern has a 120° phase shift; and in the third modulated image 410C, the illumination pattern has a 240° phase shift.

[0074] Once modulated images 410A, 410B, and 410C have been captured, processing unit 230 of system 200 can demodulate modulated images 410A, 410B, and 410C based on the lighting patterns depicted in these modulated images and relative to a target segment, in this embodiment, the target segment corresponding to a portion of the scene on which the lighting pattern is selectively projected. Subsequently, processing unit 230 can generate an ambient light suppression image of the scene based on the demodulation result.

[0075] More specifically, processing unit 230 can demodulate modulated images 410A, 410B, and 410C relative to the target segment, such that the generated ambient light suppression image of the scene depicts only one or more elements included in a selected portion of the field of view of imaging unit 220. In other words, in this embodiment, the generated ambient light suppression image of the scene depicts only the elements(s) included in a rectangular portion onto which the illumination pattern is projected. Processing unit 230 can demodulate modulated images 410A, 410B, and 410C to generate a first image corresponding to the AC component (i.e., AC component image 420) and a second image corresponding to the DC component (i.e., DC component image 430), wherein the AC component corresponds to alternating portions of the demodulated signal associated with the modulated image, and the DC component corresponds to a constant portion of the demodulated signal. It should be understood that, for example, the above references can be used... Figure 1 The formulas (1) and (2) shown are used to perform demodulation. It should also be understood that other formulas not explicitly discussed here can be used to obtain the AC and DC components of the signal, such as those associated with the Discrete Cosine Transform, Fourier Transform, etc.

[0076] In this case, AC component image 420 is selected as the ambient light suppression image of the scene. Therefore, elements that are in the scene of the modulated image but not in the selected portion can be excluded from the ambient light suppression image of the scene.

[0077] Figure 5 An ambient light suppression operation using multiple images is illustrated according to another embodiment. The illustrated operation can generally be performed by system 200, and specifically, in some embodiments by or under the control of processing unit 230 of system 200. For illustrative purposes, reference will be made below. Figure 2 The various components of the system 200 are described. Figure 5 At least a portion of the operations shown.

[0078] exist Figure 5 In the illustrated embodiment, the image projection unit 210 of system 200 is configured to project an illumination pattern having a predetermined spatial frequency. Furthermore, the imaging unit 220 of system 200 includes a monochrome camera. The imaging unit 220 is configured to capture multiple images of the scene at a predetermined depth of focus for three different color channels (RGB). The predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit are selected such that the illumination pattern is only resolvable within a certain distance from the focal point of the imaging unit 220.

[0079] like Figure 5As shown, three sets of modulated images are provided, including a first set of modulated images 520-1, a second set of modulated images 520-2, and a third set of modulated images 520-3. The first set of modulated images 520-1 corresponds to the blue channel captured by the monochrome camera, the second set of modulated images 520-2 corresponds to the green channel captured by the monochrome camera, and the third set of modulated images 520-3 corresponds to the red channel captured by the monochrome camera. Furthermore, in Figure 5 Also provided is a normal image 510, which represents an image of the scene depicted in modulated images 520-1, 520-2, and 520-3 captured under normal conditions with a lighting pattern without projection. Furthermore, in Figure 5 The document provides a set of AC component images 530, a set of DC component images 540, and a result image 550.

[0080] As referenced above Figure 2 The illumination pattern is a time-varying spatial modulation pattern. In this embodiment, each of the three modulation images in each of the first, second, and third groups of modulation images 520-1, 520-2, and 520-3 has been captured at different time points corresponding to different phase shifts of the projected illumination pattern. For example, each of the three images in a group can correspond to a time point when the illumination pattern has a 0° phase shift, a time point when the illumination pattern has a 120° phase shift, and a time point when the illumination pattern has a 240° phase shift.

[0081] Furthermore, in this embodiment, the target segment corresponds to a portion of the scene that the projected lighting pattern can distinguish. For example... Figure 5 As shown, by using the correct combination of a predetermined spatial frequency of the illumination pattern and a predetermined depth of focus of the imaging unit, a distance from the focused object exists in a scene where the illumination pattern cannot be distinguished. More specifically, for this embodiment, it can be seen that in each of the first set of modulation images 520-1, the second set of modulation images 520-2, and the third set of modulation images 520-3, the projected illumination pattern (which consists of multiple alternating horizontal dark and bright bands) is distinguishable only on the circular elements and the pen, but is indistinguishable in the background of the scene (objects located at a depth of focus greater than 1m).

[0082] Once the three sets of modulated images 520-1, 520-2, and 520-3 have been captured, the processing unit 230 of the system 200 can demodulate the three sets of modulated images 520-1, 520-2, and 520-3 based on the lighting patterns depicted in these modulated images and relative to a target segment. In this embodiment, the target segment corresponds to a portion of the scene that can be resolved by the projected lighting pattern. Subsequently, the processing unit 230 can generate an ambient light suppression image of the scene based on the demodulation result.

[0083] More specifically, processing unit 230 can demodulate each of the three sets of modulated images 510-1, 520-2, and 520-3 relative to the target segment to generate three AC component images corresponding to the blue, green, and red channels, respectively. These three AC component images form as follows: Figure 5 The AC component image group 530 is shown. Furthermore, the demodulation operation can also generate three DC component images corresponding to the blue, green, and red channels, respectively. These three DC component images are formed as follows: Figure 5 The DC component image group 540 is shown. It should be understood that, for example, the above reference can be used. Figure 1 Demodulation is performed using the formulas (1) and (2) shown. It should also be understood that other formulas not explicitly discussed here can be used to obtain the AC and DC components of the signal, such as those associated with the Discrete Cosine Transform, Fourier Transform, etc.

[0084] In this case, the AC component image group 530 is selected and subsequently processed (i.e., RGB reconstruction) to generate an ambient light suppression image 550 of the scene. Therefore, elements in the background (with a depth of focus greater than 1m) of the scene in the modulated image can be excluded from the ambient light suppression image 550. In other words, in this embodiment, the generated ambient light suppression image 550 of the scene depicts only the elements(s) included in the foreground where the lighting pattern is discernible.

[0085] Figure 6 An ambient light suppression operation using multiple images is illustrated according to another embodiment. The illustrated operation can generally be performed by system 200, and specifically, in some embodiments by or under the control of processing unit 230 of system 200. For illustrative purposes, reference will be made below. Figure 2 The various components of the system 200 are described. Figure 6 At least a portion of the operations shown.

[0086] Similar to Figure 5 The arrangement shown is in Figure 6 In one embodiment, the image projection unit 210 of system 200 is configured to project an illumination pattern having a predetermined spatial frequency. Furthermore, similar to... Figure 5 In the arrangement shown, in this embodiment, the imaging unit 220 of system 200 includes a monochrome camera. The imaging unit 220 is configured to capture multiple images of a scene at a predetermined depth of focus for three different color channels (RGB). A predetermined spatial frequency of the illumination pattern and a predetermined depth of focus of the imaging unit are selected such that the illumination pattern can only be resolved within a certain distance from the focal point of the imaging unit 220.

[0087] like Figure 6As shown, three sets of modulated images are provided, including a first set of modulated images 610-1, a second set of modulated images 610-2, and a third set of modulated images 610-3. The first set of modulated images 610-1 corresponds to the blue channel captured by the monochrome camera, the second set of modulated images 610-2 corresponds to the green channel captured by the monochrome camera, and the third set of modulated images 610-3 corresponds to the red channel captured by the monochrome camera.

[0088] As referenced above Figure 2 The illumination pattern is a time-varying spatial modulation pattern. In this embodiment, each of the three modulation images in each of the first, second, and third groups of modulation images 610-1, 610-2, and 610-3 has been captured at different time points corresponding to different phase shifts of the projected illumination pattern. For example, each of the three images in a group may correspond to a time point when the illumination pattern has a 0° phase shift, a time point when the illumination pattern has a 120° phase shift, and a time point when the illumination pattern has a 240° phase shift.

[0089] Furthermore, in this embodiment, the target segment corresponds to the portion of the scene that is distinguishable by the projected lighting pattern. For example... Figure 6 As shown, by using the correct combination of a predetermined spatial frequency of the illumination pattern and a predetermined depth of focus of the imaging unit, a distance to the focused object exists in a scene where the illumination pattern cannot be distinguished. More specifically, for this embodiment, it can be seen that in each of the first set of modulation images 610-1, the second set of modulation images 610-2, and the third set of modulation images 610-3, the projected illumination pattern (which consists of multiple alternating horizontal dark and bright bands) is distinguishable only on rectangular objects in the background of the scene.

[0090] Once the three sets of modulated images 610-1, 610-2, and 610-3 have been captured, the processing unit 230 of the system 200 can demodulate the three sets of modulated images 610-1, 610-2, and 610-3 based on the lighting patterns depicted in these modulated images and relative to a target segment. In this embodiment, the target segment corresponds to the portion of the scene where the projected lighting pattern is resolvable. Subsequently, the processing unit 230 can generate an ambient light suppression image of the scene based on the demodulation result.

[0091] More specifically, processing unit 230 can demodulate each of the three sets of modulated images 610-1, 620-2, and 620-3 relative to the target segment to generate three AC component images corresponding to the blue, green, and red channels, respectively. Furthermore, the demodulation operation can also generate three DC component images corresponding to the blue, green, and red channels, respectively. In this case, the AC component image set can be selected and subsequently processed (i.e., RGB reconstruction) to generate an ambient light suppression image 620 of the scene. It should be understood that, for example, the above reference can be used... Figure 1 The formulas (1) and (2) shown are used to perform demodulation. It should also be understood that other formulas not explicitly discussed here can be used to obtain the AC and DC components of the signal, such as those associated with the Discrete Cosine Transform, Fourier Transform, etc.

[0092] Therefore, elements in the foreground of the scene in the modulated image can be excluded from the ambient light suppression image 620 of the scene. In other words, in this embodiment, the generated ambient light suppression image 620 of the scene only depicts certain elements(s) included in the background where the lighting pattern can be distinguished.

[0093] Although the above description, with reference to some embodiments, describes each set of modulated images corresponding to a color channel as comprising three images, it should be understood that in alternative embodiments, each set of modulated images may comprise fewer or more images, depending on factors such as the amount of time and resources available, the motion of the target or system itself, and the desired level of ambient light suppression.

[0094] Figure 7 An ambient light suppression operation according to another embodiment is illustrated. The illustrated operation can generally be performed by system 200, and specifically, in some embodiments by or under the control of processing unit 230 of system 200. For illustrative purposes, reference will be made below. Figure 2 The various components of the system 200 are described. Figure 7 At least a portion of the operations shown.

[0095] like Figure 7 As shown, multiple modulation images 710 are provided. (Refer to the above reference.) Figure 2 As described, the illumination pattern is a time-varying spatial modulation pattern, so each of the plurality of modulation images 710 in this embodiment can be captured at different time points corresponding to different phase shifts of the projected illumination pattern. Furthermore, the illumination pattern in this embodiment also includes at least one phase ramp with a predetermined step size.

[0096] Once multiple modulated images 7103 are captured, the processing unit 230 of the system 200 can demodulate the modulated images based on the lighting patterns depicted in these modulated images and relative to a target segment. As will be explained in more detail in the following paragraphs, the target segment in this embodiment is based on 3D depth information of the scene depicted in the multiple modulated images 710.

[0097] In this embodiment, the processing unit 230 is configured to analyze multiple modulated images 710 to determine 3D depth information of the depicted scene. More specifically, a phase mask may be applied to the multiple modulated images 710—this operation is performed by... Figure 7 The phase mask 720 is represented and is executed before determining the 3D depth information of the scene. The 3D depth information includes the depth value for each pixel of the multiple modulated images 710. Once the 3D depth information of the scene is determined, the processing unit 230 is configured to determine the target segment based on the 3D depth information of the depicted scene. The 3D depth information of the scene is determined by... Figure 7 The 3D depth model 730 shown is illustrated. As an example of the operation of determining the target segment in this embodiment, the processing unit 230 can be configured to select one or more portions (e.g., portions corresponding to the user's face) of a predetermined range of 3D depth values ​​from a plurality of modulated images 710.

[0098] Subsequently, the processing unit 230 can generate an ambient light suppression image of the scene based on the demodulation result. Specifically, in this embodiment, the processing unit 230 is configured to generate the ambient light suppression image of the scene by outputting only the demodulation result relative to the target segment. Therefore, elements not depicted in the target segment can be excluded from the ambient light suppression image of the scene (e.g., if the 3D depth value of the corresponding pixel does not fall within a predetermined range). In other words, in this embodiment, the generated ambient light suppression image of the scene only depicts certain elements that meet certain criteria relative to the 3D depth value.

[0099] Figure 8 The results of ambient light suppression operation according to one embodiment are shown compared with other imaging or processing techniques.

[0100] For reference purposes, a normal image (without image processing or modulation) 810 is provided. As shown in the normal image 810, a first element A, a second element B, and a third element C are depicted. The depicted first to third elements A to C correspond to multiple different distances from the respective elements to the imaging unit, wherein the first element A is closest to the imaging unit and the third element C is furthest from the imaging unit.

[0101] A blurred image 820 is provided next to the normal image 810. The blurred image 820 represents the resulting image after blurring the normal image 810 in an attempt to remove some depicted details. In the example shown in the blurred image 820, image processing is performed to remove details in the background of the normal image 810, namely, the third element C.

[0102] A first ambient light correction (ALC) image 830 is provided alongside the blurred image 820. The first ALC image 830 represents the resulting image after performing wide-illumination ambient light correction processing on the normal image 810 in an attempt to remove certain depicted details. Similar to the blurred image 820 described above, in the example shown in the first ALC image 830, image processing is performed on the modulated image to remove details in the background. It can be seen that wide-illumination ambient light correction is more effective in removing background details (e.g., the third element C) compared to simply performing blurring.

[0103] Furthermore, a second ALC image 840 and a third ALC image 850 are provided next, wherein the second ALC image 840 and the third ALC image 850 represent the resulting images after performing the ambient light suppression operation described in the embodiments herein. Specifically, in this embodiment, when multiple modulated images are captured, the illumination pattern is selectively projected only onto a selected portion of the field of view of the image unit. In the example of the second ALC image 840, the selected portion corresponds to the portion depicting the first element A; and in the example of the third ALC image 850, the selected portion corresponds to the rectangular portion depicting a portion of the first element A. Therefore, the target segment corresponds to a portion of the scene on which the illumination pattern is selectively projected.

[0104] In the two examples shown by the second ALC image 840 and the third ALC image 850, the modulated image is demodulated relative to the corresponding target segment, such that the corresponding generated ambient light suppressed image of the scene depicts only one or more elements included in a selected portion of the field of view of the imaging unit. Accordingly, as shown in the second ALC image 840 and the third ALC image 850, only element A is depicted in the corresponding results, and only a portion of element A is depicted.

[0105] Therefore, an improved system and control method for performing ambient light suppression are provided. Compared to currently known techniques for shallow depth-of-field detection imaging (e.g., including blur) or wide-illumination ambient light correction, the embodiments described herein allow for improved suppression of background elements in an image. This is because the techniques described herein provide a better method for removing out-of-focus portions from an image and an improved way to perform sub-selection of the in-focus portions in an image.

[0106] A computer program product comprising a computer-readable medium having computer-readable code contained therein, the computer-readable code being configured to cause the computer or processor, when executed by a suitable computer or processor, to perform one or more methods described herein. Therefore, it should be understood that this disclosure is also applicable to computer programs, particularly computer programs on or in a carrier suitable for putting embodiments into practice. The program may be in the form of source code, object code, intermediate source code, and object code, such as in a partially compiled form, or any other form suitable for use in implementations of the methods according to the embodiments described herein.

[0107] It should also be understood that such a program can have many different architectural designs. For example, the program code implementing the functionality of the method or system can be subdivided into one or more subroutines. Many different ways of distributing functionality among these subroutines will be apparent to those skilled in the art. Subroutines can be stored together in an executable file to form a self-contained program. Such an executable file can include computer-executable instructions, such as processor instructions and / or interpreter instructions (e.g., Java interpreter instructions). Alternatively, one or more of all subroutines can be stored in at least one external library file and (e.g., at runtime) linked statically or dynamically with the main program. The main program contains at least one call to at least one subroutine. Subroutines can also include function calls to each other.

[0108] One embodiment of the computer program product includes computer-executable instructions corresponding to each processing stage of at least one method described herein. These instructions may be subdivided into subroutines and / or stored in one or more files that can be statically or dynamically linked. Another embodiment of the computer program product includes computer-executable instructions corresponding to each device of at least one of the systems and / or products described herein. These instructions may be subdivided into subroutines and / or stored in one or more files that can be statically or dynamically linked.

[0109] The carrier of a computer program can be any entity or device capable of carrying the program. For example, the carrier can include data storage such as ROM (e.g., CD ROM or semiconductor ROM) or magnetic recording media (e.g., hard disk). Furthermore, the carrier can be a transmissible medium such as electrical or optical signals, which can be transmitted via cables or optical fibers or by radio or other means. When a program is contained within such a signal, the carrier can be constituted by such a cable or other device or apparatus. Alternatively, the carrier can be an integrated circuit in which a program is embedded, the integrated circuit being adapted to perform the relevant method or for the execution of the relevant method.

[0110] Those skilled in the art, upon practicing the principles and techniques described herein, can understand and implement variations of the disclosed embodiments from a study of the accompanying drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude multiple. A single processor or other unit can perform the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not imply that combinations of these measures cannot be advantageously used. Computer programs can be stored or distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. A system (200) for performing ambient light suppression, the system comprising: The image projection unit (210) is configured to project an illumination pattern onto at least a portion of the scene, wherein the illumination pattern is a time-varying spatial modulation pattern having a predetermined spatial frequency. An imaging unit (220) having a field of view is configured to capture multiple images of the scene when the time-varying spatially modulated illumination pattern having the predetermined spatial frequency is projected onto the scene; as well as The processing unit (230) is configured as follows: The plurality of images are demodulated based on the lighting pattern and relative to a target segment in the captured plurality of images, wherein the target segment corresponds to one of the following: When the multiple images are captured, the portion of the scene onto which the lighting pattern is selectively projected. The portion of the scene and the projected lighting pattern that can be distinguished. The portion of the scene having pixel depth values ​​that satisfy a predetermined range; The target segment is part of the field of view of the imaging unit, and An ambient light suppression image of the scene is generated based on the demodulation result.

2. The system (200) of claim 1, wherein the image projection unit (210) is configured to selectively project the illumination pattern only onto a selected portion of the field of view of the imaging unit (220), and the target segment corresponds to the portion of the scene on which the illumination pattern is selectively projected, and The processing unit (230) is configured to demodulate the plurality of images relative to the target segment such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the selected portion of the field of view of the imaging unit.

3. The system (200) of claim 1, wherein the imaging unit (220) is configured to capture the plurality of images of the scene at a predetermined depth of focus, the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit are selected such that the illumination pattern is resolvable only at a distance from the focal point of the imaging unit, and wherein the target segment corresponds to the portion of the scene that is resolvable by the projected illumination pattern, and Furthermore, the processing unit (230) is configured to modulate the plurality of images relative to the target segment such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the field of view and within the focal distance of the imaging unit.

4. The system (200) according to claim 1. The processing unit (230) is configured to analyze the plurality of images to determine 3D depth information of the scene, wherein the 3D depth information includes a depth value for each pixel in the plurality of images, and the target segment is based on the 3D depth information of the scene. The processing unit (230) is configured to generate the ambient light suppression image of the scene by outputting only the demodulation result relative to the target segment.

5. The system (200) of claim 4, wherein the processing unit (230) is configured to determine the target segment by applying a phase mask to the plurality of images.

6. The system (200) according to any one of the preceding claims, wherein the illumination pattern comprises a phase-shifted sinusoidal pattern, and the imaging unit (220) is configured to capture the plurality of images with a predetermined phase difference relative to the phase of the sinusoidal pattern.

7. The system (200) according to claim 6 when dependent on claim 4 or 5, wherein the lighting pattern further includes at least one phase ramp of a predetermined step size.

8. The system (200) of claim 6, wherein the imaging unit (220) is configured to capture three sets of images when the illumination pattern is projected onto the scene, wherein the first set of images corresponds to a 0° phase shift of the sinusoidal pattern, the second set of images corresponds to a 120° phase shift of the sinusoidal pattern, and the third set of images corresponds to a 240° phase shift of the sinusoidal pattern.

9. The system (200) of claim 8, wherein the imaging unit (220) includes a color camera, and each of the first set of images, the second set of images, and the third set of images includes a single image.

10. The system (200) of claim 8, wherein each of the first group of images, the second group of images, and the third group of images comprises three images, wherein the first image in each of the three groups of images corresponds to the red channel, the second image in each of the three groups of images corresponds to the green channel, and the third image in each of the three groups of images corresponds to the blue channel.

11. The system (200) according to any one of claims 1-5, 7-10, wherein the processing unit (230) is configured to demodulate the plurality of images to generate a first image corresponding to the AC component and a second image corresponding to the DC component, wherein the first image is selected as the ambient light suppressed image of the scene.

12. A method for controlling a system to perform ambient light suppression, wherein the system includes an image projection unit, an imaging unit, and a processing unit, the method comprising: The image projection unit projects an illumination pattern (302) onto at least a portion of the scene, wherein the illumination pattern is a time-varying spatial modulation pattern having a predetermined spatial frequency. When the time-varying spatially modulated lighting pattern is projected onto the scene, the imaging unit captures (304) multiple images of the scene; as well as The processing unit demodulates (306) the plurality of images based on the illumination pattern and relative to a target segment in the captured plurality of images, wherein the target segment corresponds to one of the following: When capturing the multiple images, the portion of the scene onto which the lighting pattern is selectively projected. The portion of the scene and the projected lighting pattern that can be distinguished. The portion of the scene having pixel depth values ​​that satisfy a predetermined range; The target segment is part of the field of view of the imaging unit; as well as The processing unit generates (308) an ambient light suppression image of the scene based on the demodulation result.

13. The method of claim 12, wherein projecting the illumination pattern (302) onto at least a portion of the scene comprises selectively projecting the illumination pattern only onto a selected portion of the field of view of the imaging unit, and wherein the target segment corresponds to the portion of the scene on which the illumination pattern is selectively projected, and Demodulation (306) is performed on the plurality of images relative to the target segment such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the selected portion of the field of view of the imaging unit.

14. The method of claim 12, wherein capturing (304) the plurality of images of the scene is performed at a predetermined depth of focus, wherein the predetermined spatial frequency of the illumination pattern and the predetermined depth of focus of the imaging unit are selected such that the illumination pattern is resolvable only at a distance from the focal point of the imaging unit, and wherein the target segment corresponds to the portion of the scene that is resolvable by the projected illumination pattern, and Furthermore, wherein demodulation (306) of the plurality of images is performed relative to the target segment, such that the generated ambient light suppressed image of the scene depicts only one or more elements included in the field of view within the focal distance of the imaging unit.

15. A computer program product comprising a computer-readable medium having computer-readable code contained therein, the computer-readable code being configured to cause, when executed by a suitable computer or processor, the computer or processor to perform the method according to any one of claims 12 to 14.

Images