Optoelectronic module, camera device, optoelectronic system and method for adjusting camera parameters

Abstract

An optoelectronic module (1) for integration in a wearable device (30) is provided. The optoelectronic module (1) comprises an optical sensor (20) that is configured to generate sensor data based on ambient light in a field of view of the optical sensor (20), the ambient light illuminating a scene to be captured by an external camera device (50). The optoelectronic module (1) is configured to transmit the sensor data to the external camera device (50), the sensor data being usable to adjust camera parameters for capturing the scene, and / or for post-processing image or video data of the captured scene. Further, a camera device (50), an optoelectronic system (100) and a method for adjusting camera parameters is provided.

Description

[0001] Description

[0002] OPTOELECTRONIC MODULE , CAMERA DEVICE , OPTOELECTRONIC SYSTEM AND METHOD FOR ADJUSTING CAMERA PARAMETERS

[0003] The present application relates to an optoelectronic module for integration in a wearable device . It further relates to a camera device , an optoelectronic system, and a method for adj usting camera parameters .

[0004] Automatic white balancing, AWB, can be skewed i f the captured scene does not contain a neutral or white obj ect or i f image areas are dominated by non-white colors . Current camera devices , such as high-end smartphone cameras , implement an additional spectral sensor close to the main camera and pointing to the same direction in the scene . Such sensors assist the camera in color saturated environments and closerange images . However, in mixed lighting environments , the AWB algorithms can become confused .

[0005] At least one obj ect of particular embodiments is to improve image or video quality of a captured scene .

[0006] This obj ect is achieved with the subj ect-matter of the independent claims . Embodiments and developments derive from the dependent claims .

[0007] According to at least one embodiment , an optoelectronic module for integration in a wearable device is provided . This can mean that the optoelectronic module is to be integrated in the wearable device . This can further mean that a wearable device comprising the optoelectronic module is provided . The optoelectronic module can form a sensor package . The sensor package is small enough to be placed inside a housing of the wearable device .

[0008] According to at least one embodiment , the optoelectronic module comprises an optical sensor that is configured to generate sensor data based on ambient light in a field of view of the optical sensor . The optical sensor may be implemented as semiconductor chip . The optical sensor may be or may comprise one or more photodetectors , for example photodiodes . The optical sensor may comprise a plurality of photodetectors , which can be arranged in an array . Thus , the optical sensor may form a pixel array . The photodetectors are configured to convert electromagnetic radiation into electrical signals , thus generating the sensor data . The terms "electromagnetic radiation" and " light" may be used as synonyms in the following . Light may refer to the visible and / or invisible spectrum . For example , the optical sensor is configured to detect light in the visible spectrum and / or the infrared spectrum . Ambient light refers to light coming from external light sources , wherein said light sources may be natural or arti ficial . Ambient light illuminates the environment around the optoelectronic module and around the wearable device in which the optoelectronic module is arranged . The optical sensor has a field of view . The field of view can change depending on the orientation within the wearable device and the orientation of the wearable device when used by a user . However, it can be assumed that the optical sensor has a predefined field of view relative to the user i f the user properly uses the wearable device .

[0009] According to at least one embodiment , the ambient light illuminates a scene to be captured by an external camera device . That the camera device is external can mean that the optoelectronic module is located outside the camera device . The external camera device is not part of the optoelectronic module but may communicate with the optoelectronic module . The camera device captures an image or a video of the scene , while the scene is illuminated by the ambient light sensed by the optical sensor . That is , the camera device and the optoelectronic module can be configured to be operated simultaneously . Alternatively, the camera device and the optoelectronic module can be configured to be operated within a common time slot .

[0010] According to at least one embodiment , the optoelectronic module is configured to transmit the sensor data to the external camera device . The sensor data can be transmitted to the external camera device shortly before , during, or shortly after the external camera device captures the scene . The sensor data can be transmitted by means of a transmitter unit that can be part of the optoelectronic module or can be outside the optoelectronic module . A transceiver unit of the external camera module can receive the sensor data . The sensor data can be transmitted as raw data or can be transmitted as pre-processed data . In the latter case , the optoelectronic module may further comprise a pre-processing unit . Pre-processing can comprise , for example , filtering the raw sensor data, extracting speci fic wavelengths , evaluating speci fic frequencies , etc .

[0011] According to at least one embodiment , the sensor data is usable to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene . The camera parameters are parameters of the camera device . Camera parameters may include white balancing parameters , light temperature parameters , color saturation parameters, and / or shutter parameters. This can mean that hardware or software parameters of the camera device are adjusted or modified based on the sensor data. It is also possible that image or video data of the captured scene are adjusted based on the sensor data. In particular, digital parameters of an image or video of the captured scene can be adjusted or modified using the sensor data. For example, pixel values are modified during post-processing. For example, pixel values are modified by adding color components based on the sensor data, thus changing the color temperature. Thus, the sensor data can be used by the camera device before it captures the scene, and / or after it captures the scene. In any case, using the sensor data can improve the image or video quality. That the sensor data is usable to adjust camera parameters can mean that the sensor data is configured to be used for adjusting camera parameters. Camera parameters can be adjusted if sensor data is available and / or if adjusting camera parameters is enabled. For example, the camera device provides options to enable adjusting camera parameters based on the sensor data.

[0012] According to at least one embodiment, an optoelectronic module for integration in a wearable device comprises an optical sensor that is configured to generate sensor data based on ambient light in a field of view of the optical sensor, the ambient light illuminating a scene to be captured by an external camera device, wherein the optoelectronic module is configured to transmit the sensor data to the external camera device, the sensor data being usable to adjust camera parameters for capturing the scene, and / or for post-processing image or video data of the captured scene. The optoelectronic module described here is based on the following considerations , among others .

[0013] A complex lighting scenario may refer to situations in which more than one light source illuminates a scene . By means of the optoelectronic module brightness and color information of light coming from di f ferent directions in such complex lighting scenarios can be captured . The position of each light source can be tracked and used by the camera device to improve the picture quality, including AWB, color fidelity, immunity to flicker, etc .

[0014] Spectral sensors may be integrated in the camera device and may help to detect light properties in a predefined field of view . However, this is done by adding more optical complexity, si ze , and price . The current approach relates to obtaining more information on the light conditions by adding, with low cost and low complexity, optical sensors in wearable devices , such as earphones or earbuds , that by nature can look at di f ferent angles and capture more light sources illuminating the scene .

[0015] Ambient light from light sources at various directions can be detected and identi fied due to the unique position of the optical sensor in the wearable device . Such optical sensors are capable to capture parts of the scene that other sensors that are integrated in the camera device are not able to . Further, integrating the optoelectronic module with the optical sensor in a wearable device of fers flexibility and possibility to be manually turned towards di f ferent directions . Since multiple wearables come with electronic components anyway, the optoelectronic module may reuse their electronic infrastructure , as for example the analog front end. Thus, integration of the optoelectronic module is costefficient .

[0016] Integrating the optoelectronic module in a wearable device can thus support the camera performance of other devices, e.g. smartphones, connected to the wearable device in a sensor fusion fashion.

[0017] According to at least one embodiment, the optical sensor is configured to generate sensor data based on spectral characteristics of the ambient light, and the sensor data is usable to adjust a white balance and / or color fidelity of an image or video of the captured scene. This can mean that the sensor data contain spectral information about the ambient light. In this case, the optical sensor comprises a plurality of channels, each provided with respective color filters to detect predetermined wavelengths or wavelength ranges of the ambient light. This can mean that the optical sensor is or comprises a color sensor or spectral sensor. For example, the optical sensor comprises at least three channels for detecting different wavelengths of the ambient light. For example, the optical sensor comprises at least one channel to detect blue light, at least one channel to detect red light, and at least one channel to detect green light. Thus, spectral information of the ambient light can be determined. Thus, the light sources generating the ambient light can be analyzed and identified. Knowing the spectral composition of the ambient light, one or more light sources generating the ambient light can be analyzed and identified. Thus, it is possible to enhance white balancing and / or color fidelity of the captured image or video. Through the sensor data the external camera device is provided with information about the lighting conditions. Even if the camera device is equipped with its own dedicated spectral sensor, the optical sensor supports the camera device with additional information, in particular with information about light sources illuminating the scene from di f ferent directions .

[0018] According to at least one embodiment , the optical sensor is configured to generate sensor data based on flicker of the ambient light , and the sensor data is usable to reduce flicker arti facts in an image or video of the captured scene . Images and videos can be ruined with unsightly banding distortion ef fects from flickering light sources . In other words , when visible light modulations from flickering lights are captured while a camera device ' s rolling shutter is in operation, the resulting ef fect is an image or video with distorted banding arti facts . The optical sensor may comprise a wideband and / or clear reference channel to incorporate flicker detection . Thus , the optical sensor can detect the presence or absence of 50Hz or 60Hz flicker typically generated from incandescent or fluorescent lights . When flicker is detected, it is captured and reported through the sensor data to the external camera device . Knowing that flicker is present , the camera device ' s video processor may synchroni ze the shutter with the "on" portion of the relative ambient light output in a scene . With this additional information, the video processor can eliminate the distorting banding arti facts and produce images that truly represent the image seen by the user .

[0019] According to at least one embodiment , the field of view of the optical sensor is di f ferent from a field of view of the external camera device and / or from a field of view of another optical sensor integrated in the external camera device . As mentioned above , the camera device may be equipped with dedicated spectral sensors to enable automatic white balancing, AWB . However, such spectral sensors typically have the same or a similar field of view as the camera device . When used in the wearable device , the optical sensor has a di f ferent field of view . In other words , the optical sensor extends the field of view of the camera device . For example , the user of the camera device wears the wearable device in such a way, that a main direction of the field of view of the optical sensor points in a direction di f ferent from a main direction of the field of view of the camera device . Thus , the fields of view of the camera device and of the optoelectronic module within the wearable device relate to normal operating modes of both devices . It is also possible that the user intentionally points the wearable device in a speci fic direction that is di f ferent from a direction in which the camera device points . Since the optical sensor has a di f ferent field of view than the camera device , complex lighting scenarios in which one or more light sources illuminate the scene from di f ferent directions can be handled . The position of each light source can be tracked and used by the camera device to improve the picture quality, including AWB, color fidelity, immunity to flicker, etc .

[0020] According to at least one embodiment , the optoelectronic module is integrated in the wearable device . According to at least one embodiment , the wearable device is one of the following : earphones or earbuds , eyeglasses , a watch, a f itness-tracker , a finger ring, or earring . A separate optoelectronic module can be arranged in each piece of the wearable device . The plurality of separate optoelectronic modules may form an overall optoelectronic module . For example , a respective optoelectronic module is arranged in the left and the right piece of earphones or earbuds, or in the left and the right temple of eyeglasses. In this way it is ensured that the fields of view of respective optical sensors point in different directions. In particular, the fields of view may cover the left and / or the right side of the user. Usually, the camera device captures a scene in front of or behind the user. Thus, the fields of view of the optical sensor and the camera device are different. The optical sensor (s) can "look" under different angles and can directly capture more light sources illuminating the scene. Besides, the wearable device, e.g. earbuds, can be taken off and directed to a specific light source direction, enhancing the advantage of being directional independent on the camera device .

[0021] According to at least one embodiment, the optoelectronic module is arranged in a housing of the wearable device, wherein the housing is at least partially transparent for ambient light to be detected. For example, the housing comprises a plastic material. Further electronic components can be arranged inside the housing, as in the case of earphones or earbuds. The housing has a small thickness through which ambient light can penetrate. In particular, a transmissivity of 1 to 10% of the housing can be sufficient to detect ambient light, such that light sources or light source characteristics can be identified. Thus, the optoelectronic module can easily be integrated behind plastic. It is however also possible that the housing comprises a transparent window through which ambient light can be detected by the optical sensor.

[0022] According to at least one embodiment, the optoelectronic module is configured to communicate with a camera application of a smartphone or smart device . In other words , the camera device can be or can be comprised by a smartphone or smart device . The optoelectronic module can comprise an interface for transmitting data to an interface of the camera device . For example , the optoelectronic module comprises a transmitter unit , and the camera device comprises a transceiver unit . It is also possible that the transmitter unit is arranged outside the optoelectronic module . Information collected by the optical sensor within the optoelectronic module can be transmitted to a main processing unit of the smart device and used to improve the camera performance for AWB, color fidelity, flicker immunity, etc . The additional available direction that is of fered can complement other sensors of the smartphone or smart device and produce a better description of the lighting condition of the scene , which is particularly important in mixed-lighting conditions , i . e . where multiple light sources from di f ferent angles are shining to the obj ect to be captured by the camera . In particular, the additional sensors can help for light source identi fication .

[0023] According to at least one embodiment , the optoelectronic module further comprises an optically based sensor configured to receive force inputs and / or gesture inputs from a user using the wearable device . A force input can be understood as an input caused by the user who applies a force to the housing of the wearable device , so that the housing is bent and / or moved and / or deformed . A gesture input can be an input caused by a user' s hand or finger that is in proximity to the housing . Force and / or gesture inputs can be sensed in the following way, for example : one or more emitters ( comprised or not comprised by the optoelectronic module ) emit light in the visible and / or infrared range . A first portion of said light may be reflected by the housing, and a second portion may be transmitted by the housing . I f a force input is applied, a reflection characteristic, for instance a reflection direction or intensity, of the housing is changed . Said change can be sensed by a photodetector or an array of photodetectors . Gesture inputs can be detected by light that is transmitted by the housing and reflected by the user' s hand or finger . Thus , the optoelectronic module can combine multiple sensing functionality .

[0024] According to at least one embodiment , the optical sensor and the optically based sensor share a common analog front end circuitry of the optoelectronic module . In other words , the optical sensor reuses the available hardware infrastructure of the force / touch or gesture sensor . For example , the optical sensor and the optically based sensor share a common analog-to-digital converter . An optically based force / touch sensor may be implemented anyway in earphones or earbuds . Implementing the optical sensor, e . g . for spectral or flicker sensing, is only associated with the relatively low expense of installing additional sensor channels . By reusing the analog front end circuitry a reduction of the overall chip si ze is achieved .

[0025] According to at least one embodiment , the sensor data of the optical sensor is usable to increase immunity of an optically based sensor to ambient light variations , the optically based sensor being integrated in or being arranged outside of the optoelectronic module . For example , the sensor data generated by the optical sensor can be used to increase immunity of the optically based sensor to flicker noise . Besides , having a broader spectrum than the optically based sensor ( e . g . f orce / gesture sensor ) , and a di f ferent and possibly longer integration time, the optical sensor can provide useful information on an external light source in the environment, i.e. ambient light. This can help rejecting disturbances, e.g. flicker, coming from the light source itself. The optical sensor cannot only support other optically based sensors on chip, but also optically based sensors outside the optoelectronic module.

[0026] According to at least one embodiment, the optoelectronic module further comprises an emitter. The emitter can be configured to emit light in the visible or infrared range during operation. The emitter can be or can comprise a light emitting diode, LED, or laser diode. The optoelectronic module can comprise more than one emitter. The emitter can be arranged on the same plane as the optical sensor. An emission direction of the emitter can at least partially overlap with a field of view of the optical sensor.

[0027] According to at least one embodiment, the optical sensor is configured to generate sensor data based on an intensity of light. For example, the optical sensor can detect changes in intensity .

[0028] According to at least one embodiment, if an intensity of the ambient light sensed by the optical sensor is below a predefined threshold, the emitter is configured to emit a light signal. For example, based on the sensed light by the optical sensor, an LED is activated in dark conditions, providing signaling for warning purposes. Thus, the optoelectronic module can combine multiple sensing functionality . According to at least one embodiment , the emitter is configured to emit light towards human tissue of the user using the wearable device , and the optical sensor is configured to sense said light after reflection or transmission by the human tissue . For example , the user' s finger is placed on a dedicated area of the wearable device . Light transmitted by the housing is at least partially reflected and / or absorbed by the finger . Reflected light can be detected by the optical sensor . I f the optical sensor comprises a plurality of pixels / channels , an image of the fingerprint associated with the finger can be reconstructed . In another example , vital parameters such as the heartrate of the user can be determined based on the absorbance and / or reflectance of the human tissue . Thus , the optoelectronic module can combine multiple sensing functionality .

[0029] In addition, a camera device is provided . The camera device is configured to interact with the optoelectronic module described above . Therefore , features relating to the optoelectronic module are also disclosed for the camera device , i f applicable .

[0030] According to at least one embodiment , the camera device is configured to capture a scene being illuminated by ambient light . According to at least one embodiment , the camera device is further configured to receive sensor data from an external optoelectronic module integrated in a wearable device . According to at least one embodiment , the sensor data contain information about the ambient light that illuminates the scene and is usable to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene . The camera device has the same advantages as described above in connection with the optoelectronic module . In particular, by means of the optoelectronic module brightness and color information of light coming from di f ferent directions in a complex lighting scenario can be captured . The position of each light source can be tracked and used by the camera device to improve the picture quality, including AWB, color fidelity, immunity to flicker, etc .

[0031] In addition, an optoelectronic system is provided . The optoelectronic system comprises the optoelectronic module and the camera device as described above . This means that all features disclosed for the optoelectronic module and the camera device are also disclosed for the optoelectronic system and vice versa . The optoelectronic system has the same advantages as described above in connection with the optoelectronic module and the camera device .

[0032] In addition, a method for adj usting camera parameters is provided . The method is preferably carried out using the optoelectronic module and the camera device as described above . This means that all features disclosed for the optoelectronic module and the camera device are also disclosed for the method for adj usting camera parameters , and vice versa .

[0033] According to at least one embodiment . The method comprises the step of generating, by an optical sensor comprised by an optoelectronic module integrated in a wearable device , sensor data that is based on ambient light in a field of view of the optical sensor, the ambient light illuminating a scene . The optical sensor may comprise one or more photodetectors , which are configured to convert electromagnetic radiation into electrical signals . The sensor data relate to said electrical signals . The sensor data may be raw signals or may be pre- processed signals .

[0034] According to at least one embodiment , the method comprises the step of transmitting the sensor data from the optoelectronic module to a camera device configured to capture the scene , wherein the optoelectronic module is outside the camera device . Any suitable protocol for transmitting can be used . For example , the optoelectronic module communicates with the camera device over a data link according to wireless local area networking ("WLAN" ) standards , such as Wi-Fi or Bluetooth . A wired communication is also possible . The communication can be performed in realtime .

[0035] According to at least one embodiment , the method comprises the step of using the sensor data to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene . This can mean that hardware and / or software parameters of the camera device are adj usted . For example , a setup for white balancing is adj usted . In addition or alternatively, a shutter frequency is adj usted . In this case , the sensor data is used prior to capturing the scene or during capturing the scene . It is however also possible that image data or video data of the captured scene are modi fied using the sensor data . For example , the image or video data is modi fied by adding dedicated color components to pixel values of the image or video .

[0036] According to at least one embodiment , the method comprises the step of capturing, by the camera device , the scene , using the adj usted camera parameters . Capturing the image or the video of the scene and using the sensor data can be conducted simultaneously . The sensor data contain other information than the image or video data . Using the sensor data for adj usting the camera parameters can be performed in realtime .

[0037] According to at least one embodiment , a method for adj usting camera parameter comprises the following steps : generating, by an optical sensor comprised by an optoelectronic module integrated in a wearable device , sensor data that is based on ambient light in a field of view of the optical sensor, the ambient light illuminating a scene ; transmitting the sensor data from the optoelectronic module to a camera device configured to capture the scene , wherein the optoelectronic module is outside the camera device ; and using, by the camera device , the sensor data to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene .

[0038] The method has the same advantages as described above in connection with the optoelectronic module and the camera device . By using the sensor data brightness and color information of light coming from di f ferent directions in a complex lighting scenario can be captured . The sensor data can be used by the camera device to improve the picture quality, including AWB, color fidelity, immunity to flicker, etc .

[0039] According to at least one embodiment of the method, the sensor data is based on spectral characteristics of the ambient light , and using the sensor data comprises adj usting a white balance and / or color fidelity of the image or video of the captured scene . For example , the sensor data contains information about wavelengths of the ambient light . Thus , spectral information of the ambient light can be determined and the light sources generating the ambient light can be analyzed and identi fied . By knowing the spectral composition of the ambient light , it is possible to enhance white balancing and / or color fidelity of the captured image or video .

[0040] Additionally or alternatively, the sensor data is based on flicker of the ambient light , and using the sensor data comprises reducing flicker arti facts in an image or video of the captured scene . For example , when flicker is detected, the camera device ' s video processor synchroni zes the shutter with the "on" portion of the relative ambient light output in a scene . Thus , distorting banding arti facts can be eliminated and images that truly represent the image seen by the user can be produced .

[0041] According to at least one embodiment of the method, the field of view of the optical sensor is di f ferent from a field of view of the camera device and / or from a field of view of another optical sensor integrated in the camera device during operation of the optoelectronic module and of the camera device . In other words , the optical sensor supports the camera device with additional information, in particular with information about light sources illuminating the scene from di f ferent directions .

[0042] The following description of Figures may further illustrate and explain aspects of the optoelectronic module , the camera device , the optoelectronic system, and the method of adj usting camera parameters . Components and parts of the optoelectronic module , the camera device and the optoelectronic system that are functionally identical or have an identical ef fect are denoted by identical reference symbols . Identical or ef fectively identical components and parts might be described only with respect to the Figures where they occur first . Their description is not necessarily repeated in successive Figures .

[0043] Figure 1 shows a use case of generating an image or video of a scene according to an embodiment .

[0044] Figure 2 shows an optoelectronic module integrated in a wearable device and a camera device according to an embodiment .

[0045] Figures 3A and 3B show an optoelectronic module according to an embodiment .

[0046] Figure 4 shows an optoelectronic module according to an embodiment .

[0047] Figures 5A and 5B show wearable devices comprising optoelectronic modules according to an embodiment .

[0048] Figure 6 shows an optoelectronic system comprising an optoelectronic module and a camera device according to an embodiment .

[0049] Figure 7 shows a method of adj usting camera parameters for capturing a scene according to an embodiment .

[0050] In Figure 1 a use case of generating an image or a video of a scene 900 is shown . The scene 900 is viewed by a lens of a camera device 50 . In the shown example , the camera device 50 is or is included in a smartphone . The scene 900 may be captured by the front camera or the rear camera of the smartphone . Thus , the lens can be the lens 51a of the front camera or the lens 51b of the rear camera (not shown in Figure 1 ) . The scene 900 is illuminated by a natural or arti ficial light source 910 . Light 911 from the light source 910 propagates towards the scene 900 , at which a portion 930 of the light 911 is reflected by the scene 900 and propagates towards the camera device 50 . I f said portion 930 of reflected light is within a field of view of the camera device ' s lens 51 , the scene is captured by the camera device 50 . Analyzation and identi fication of the light source 910 may be required for ( automatic ) white balancing of the captured image or video . In the shown example , a further natural or arti ficial light source 920 illuminates the scene 900 , wherein light 921 from the further light source 920 also propagates towards the scene 900 , at which it is partially reflected towards the camera device 50 . In such complex lighting scenarios , in which multiple light sources 910 , 920 from di f ferent directions are involved, each area of a captured image or video may have other backlights with alternative color temperatures . In this case , a global white balance fails . The camera device 50 might be equipped with a spectral sensor to analyze and identi fy a light source in order to perform automatic white balancing, AWB . However, the field of view of such spectral sensors may be limited and may match the field of view of the camera device itsel f . That is , from a perspective of the camera device ' s user, such a spectral sensor may only cover a forward direction and a backward direction, corresponding, to a smartphone ' s front camera and rear camera, for example .

[0051] In Figure 2 another use case of generating an image or a video of a scene is shown, according to an embodiment . An optoelectronic module 1 is integrated in a wearable device 30. In the shown example, the optoelectronic module 1 is integrated in earphones or earbuds, respectively. A user 800 may wear the wearable device 30, e.g. the earphones, during using the camera device 50 for capturing an image or video of the scene. It should be noted that the optoelectronic module 1 is outside the camera device 50. Each piece of the wearable device, e.g. each piece of the earphones, may include a separate optoelectronic module 1. For reasons of readability, however, only one optoelectronic module 1 is referred to below .

[0052] The scene can be, for example, in front of the user 800. Alternatively, the user 800 may capture a scene including the user 800, for example by using a smartphone's front-facing camera, also known as selfie camera. The scene (not explicitly shown in Figure 2) may involve a complex lighting scenario, in which multiple light sources 910, 920 are illuminating the scene 900 from different directions. In the shown example, a first light source 910 and a second light source 920 are located at the left and the right side, respectively, of the user 800. Even if the camera device 50, e.g. the user's smartphone, is equipped with a spectral sensor for AWB, that spectral sensor would have a different field of view that is not aligned with the direction at which the light sources 910, 920 are located. Usually, such a spectral sensor has the same or similar field of view as the field of view 52, 53 of the camera device 50. However, the optoelectronic module 1 comprises an optical sensor 20 (shown in Figures 3A, 3B) having a different field of view 21, 22 than the field of view 52, 53 of the camera device 50 or a spectral sensor within the camera device 50. In particular, the field of view 21, 22 of the optical sensor 20 may be at least partially aligned with the light sources 910 , 920 illuminating the scene 900 . Light from the light sources 910 , 920 may be referred to as ambient light in the following .

[0053] The optical sensor 20 comprised by the optoelectronic module 1 is configured to generate sensor data based on ambient light in the field of view 21 , 22 of the optical sensor 20 . The ambient light illuminates the scene captured or to be captured by the camera device 50 . Since the optoelectronic module 1 is outside the camera device 50 , the camera device 50 may be referred to as external camera device 50 . The optoelectronic module 1 is configured to transmit the sensor data to the external camera device 50 . The sensor data is usable to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene .

[0054] In particular, the optical sensor 20 may be configured to generate sensor data based on spectral characteristics of the ambient light . Thus , spectral information of the ambient light can be determined . Thus , light sources that generate the ambient light can be analyzed and identi fied . Thus , the sensor data is usable to adj ust a white balance and / or color fidelity of an image or video of the captured scene .

[0055] In addition or alternatively, the optical sensor 20 is configured to generate sensor data based on flicker of the ambient light . Thus , a speci fic flicker frequency or flicker noise of the ambient light can be determined . Thus , the sensor data is usable to reduce flicker arti facts in the image or video of the captured scene . Figures 3A and 3B show di f ferent views on an exemplary optoelectronic module 1 . In Figure 3A a top view is shown, while in Figure 3B a perspective view is shown . The optoelectronic module 1 comprises a sensor chip 10 . The sensor chip 10 is mounted on a printed circuit board, RGB 3 , and electrically connected to the RGB 3 via electrical connections 11 . The sensor 10 can be embedded in a clear mold 2 that is arranged on the RGB 3 and the sensor chip 10 . Thus , the optoelectronic module 1 can be formed as clear mold package . However, a di f ferent packaging is also possible . For example , the optoelectronic module 1 can be formed as LID package . The optical sensor 20 may be part of the sensor chip 10 . In the shown example , the sensor chip 10 comprises a pixel array 15 , wherein each pixel of the pixel array 15 forms a respective channel 26 . The optical sensor 20 comprises at least one channel 26 to detect ambient light or ambient light characteristics . It is possible that other pixels of the pixel array 15 form channels 26 ' belonging to a di f ferent sensor functionality .

[0056] For example , a first subset of pixels of the pixel array 15 belongs to the optical sensor 20 and is configured to detect ambient light and to generate sensor data based on the ambient light . In particular, the first subset of pixels can comprise at least three channels 26 to detect spectral characteristics of ambient light and / or it can comprise at least one channel 26 to detect flicker of the ambient light and / or it can comprise at least one channel 26 to evaluate an intensity of ambient light . The optical sensor 20 can form an RGB or XYZ color sensor . A dedicated infrared ( IR) channel of the optical sensor 20 is also possible . Wavelength filters can be arranged on the pixels to form the respective color channels 26 . The first subset of pixels may share a common analog front end circuitry with a second subset of pixels . The second subset of pixels may belong to another sensor functionality, in particular to an optically based sensor 25 comprised by the optoelectronic module 1 .

[0057] For example , the second subset of pixels form an optically based sensor 25 that is configured to receive force inputs and / or gesture inputs from a user using the wearable device 30 . Such optically based sensor 25 may utili ze one or more emitters 18 comprised by the optoelectronic module 1 . The optically based sensor 25 may work as follows : The emitter 18 emit light in the visible and / or infrared range . A first portion of said light may be reflected by the housing of the wearable device 30 , in which the optoelectronic module 1 is arranged . A force input can be understood as an input caused by the user who applies a force to the housing, so that the housing is bent and / or moved and / or deformed . In this way, a reflection characteristic, for instance a reflection direction or intensity, of the housing is changed . Such change can be detected by one or more pixels of the second subset of pixels . A second portion of the light emitted by the emitter 18 may be transmitted by the housing . A gesture input can be an input caused by a user' s hand or finger that is in proximity to the housing . The transmitted light can be reflected by the user' s hand or finger and can be detected by one or more pixels of the second subset of pixels . In addition or alternatively, the user' s hand or finger shadows ambient light . In any case , a change in the detected light intensity can be detected . The optical sensor 20 and the optically based sensor 25 can share a common analog front end circuitry of the optoelectronic module 1 . In addition, the sensor data generated by the optical sensor 20 can be used to increase immunity of the optically based sensor 25 to ambient light variations, e.g. flicker noise.

[0058] It is also possible that the emitter 18 or an additional emitter inside or outside the optoelectronic module 1 is controlled using the sensor data. For example, if an intensity of the ambient light sensed by the optical sensor 20 is below a predefined threshold, the emitter is configured to emit a light signal, for example for warning purposes. In another example, the emitter 18 or an additional emitter inside or outside the optoelectronic module 1 is used to emit light towards human tissue of the user using the wearable device 30. The optical sensor 20, or at least one or more channels 26 of the optical sensor 20, is further configured to sense said light after reflection or transmission by the human tissue. Thus, the optical sensor 20 may enable additional sensor functionality, such as fingerprint recognition and / or vital parameter sensing.

[0059] In Figure 4 it is shown that the optoelectronic module 1 implemented as clear mold package can be soldered on a further PCB 4. The further PCB 4 can comprise additional components, e.g. passive circuit components. The further PCB 4 with the optoelectronic module 1 may fit into a wearable device's housing, e.g. into stems of earphones.

[0060] Figures 5A and 5B show different wearable devices 30, in which the optoelectronic module 1 can be integrated. For example, Figure 5A shows eyeglasses 31, wherein the optoelectronic module 1 is integrated in the temples 31a, 31b of the eyeglasses 31. In particular, a first optoelectronic module la is integrated in the left temple 31a, and a second optoelectronic module lb is integrated in the right temple 31b of the eyeglasses 31 . The first la and the second optoelectronic module lb form a combined optoelectronic module 1 . Figure 5B shows that the wearable device 30 can also be earphones or earbuds 32 . In particular, a first optoelectronic module la can be integrated in the stem 32a of the right earpiece . The left earpiece of the earphones can be utili zed in an analogous way, as shown in Figure 2 . In addition or alternatively, the optoelectronic module 1 can be integrated in a watch, a fitness tracker, a finger ring, an earring, or another wearable device 30 . The optoelectronic module 1 can be arranged inside a housing of the respective wearable device 30 . In the shown examples of Figures 5A and 5B, the housing is formed by the temples 31a, 31b of the eyeglasses 31 and by the stem 32a of the earphones 32 , respectively . The housing may be at least partially transparent for ambient light to be detected by the optical sensor 20 . For example , the housing comprises a plastic material which has a certain transmissivity for light depending on its thickness and on the light ' s wavelength . A transmissivity in the range from 1 to 10% may be suf ficient to detect predefined wavelengths of the ambient light . Alternatively, the housing comprises a window that is transparent for the ambient light to be detected .

[0061] In Figure 6 an optoelectronic system 100 is shown . The optoelectronic system 100 comprises the optoelectronic module 1 and the camera device 50 . Both the optoelectronic module 1 and the camera device 50 are shown schematically . The optoelectronic module 1 comprises the optical sensor 20 generating the sensor data SD based on the ambient light . The optoelectronic module 1 further comprises a pre-processing unit 23 for pre-processing the sensor data SD . The preprocessing unit 23 is optional . The optoelectronic module 1 further comprises a transmitter unit 24 for transmitting the sensor data SD or the pre-processed sensor data, respectively . The transmitter unit 24 is optional . The preprocessing unit 23 and / or the transmitter unit 24 can also be arranged on a separate chip outside the optoelectronic module 1 . It is also possible that the sensor data SD is not pre- processed but transmitted to the camera device 50 as raw data .

[0062] Any suitable transmitting protocol can be used . For example , the optoelectronic module 1 communicates with the camera device 50 over a data link according to wireless local area networking ( "WLAN" ) standards , such as Wi-Fi or Bluetooth . A wired communication is also possible . Thus , i f the camera device 50 is a smartphone or smart device , the optoelectronic module 1 can communicate with a camera application of said smartphone or smart device .

[0063] The sensor data SD or the pre-processed sensor data is transmitted ( as indicated by the arrow) to the camera device 50 , at which a transceiver unit 56 receives it . The camera device 50 further comprises an imaging sensor unit 54 that is used to capture an image or a video of the scene , wherein the scene is illuminated by the ambient light sensed by optical sensor 20 . That is , the optical sensor 20 and the imaging sensor unit 54 may operate simultaneously or at least within a predefined time slot . The optical sensor 20 and the image sensor unit 54 can operate in concert with each other . In one example , image data and / or video data generated by the imaging sensor unit 54 are transmitted to a processing unit 55 of the camera device 50 . Further, the sensor data SD or pre-processed sensor data is transmitted from the transceiver unit 56 to the processing unit 55 . The processing unit 55 is configured to combine the image data or video data with the sensor data SD . As such, the image data or video data is adj usted based on the information about the ambient light contained in the sensor data . In particular, white balancing and / or color fidelity of the image or video can be adj usted . This can mean that the image or video data is post-processed in the processing unit 55 using the sensor data SD .

[0064] In addition or alternatively, the processing unit 55 is implemented as control unit 55 for controlling the imaging sensor unit 54 . In this case , for example , the sensor data SD is used to determine flicker of the ambient light . The imaging sensor unit 54 can be controlled by the control unit 55 to be synchroni zed with the "on" portion of the relative ambient light output in a scene . This is in particular true , i f the camera device 50 is operated in a rolling shutter mode . In another example , a white balance setup of the camera device 50 is adj usted based on the sensor data SD . For example , the sensor data SD indicates that light illuminating the scene is daylight , and / or incandescent light , and / or fluorescent light , and / or halogen light , and / or cool white LED light , and / or warm white LED light . Accordingly, white balancing parameters for capturing the scene can be adj usted .

[0065] In Figure 7 a method for adj usting camera parameters is shown schematically . The method comprises the following steps that are not necessarily carried out in this order but can be carried out in this order .

[0066] In a first step S I , an optical sensor 20 comprised by an optoelectronic module 1 integrated in a wearable device 30 generates sensor data . The sensor data is based on ambient light in a field of view of the optical sensor 20 , wherein the ambient light illuminates a scene .

[0067] In a second step S2 , the sensor data is transmitted from the optoelectronic module 1 to a camera device 50 configured to capture the scene , wherein the optoelectronic module 1 is outside the camera device 50 .

[0068] In a third step S3 , the sensor data is used by the camera device 50 to adj ust camera parameters for capturing the scene , and / or for post-processing image or video data of the captured scene .

[0069] In an optional fourth step S4 , the scene is captured, by the camera device , using the adj usted camera parameters . In addition or alternatively, the scene is captured, and the resulting image or video data is post-processed using the sensor data .

[0070] The embodiments of the optoelectronic module 1 , the camera device 50 , the optoelectronic system 100 , and the method of adj usting camera parameters disclosed herein have been discussed for the purpose of familiari zing the reader with novel aspects of the idea . Although preferred embodiments have been shown and described, many changes , modi fications , equivalents , and substitutions of the disclosed concepts may be made by one having skill in the art without unnecessarily departing from the scope of the claims .

[0071] It will be appreciated that the disclosure is not limited to the disclosed embodiments and to what has been particularly shown and described hereinabove . Rather, features recited in separate dependent claims or in the description may advantageously be combined. Furthermore, the scope of the disclosure includes those variations and modifications, which will be apparent to those skilled in the art and fall within the scope of the appended claims.

[0072] The term "comprising", insofar it was used in the claims or in the description, does not exclude other elements or steps of a corresponding feature or procedure. In case that the terms "a" or "an" were used in conjunction with features, they do not exclude a plurality of such features. Moreover, any reference signs in the claims should not be construed as limiting the scope.

[0073] This patent application claims the priority of German patent application 102024101034.7, the disclosure content of which is hereby incorporated by reference.

[0074] References

[0075] 1 optoelectronic module

[0076] 2 mold

[0077] 3 printed circuit board, PCB

[0078] 4 further PCB

[0079] 10 sensor chip

[0080] 11 electrical connections

[0081] 15 pixel array

[0082] 18 emitter

[0083] 20 optical sensor

[0084] 21 , 22 field of view

[0085] 23 pre-processing unit

[0086] 24 transmitter unit

[0087] 25 optically based sensor

[0088] 26 , 26 ' channel

[0089] 30 wearable device

[0090] 31 eyeglasses

[0091] 31a, 31b temples

[0092] 32 earphones , earbuds

[0093] 32a stem

[0094] 50 camera device

[0095] 51 lens

[0096] 52 , 53 field of view

[0097] 54 imaging sensor unit

[0098] 55 processing / control unit

[0099] 56 transceiver unit

[0100] 100 optoelectronic system

[0101] 900 scene

[0102] 910 light source

[0103] 911 ambient light

[0104] 920 further light source

[0105] 921 ambient light 930 reflected light

[0106] SD sensor data

[0107] S1-S4 step

Claims

Claims1. Optoelectronic module (1) for integration in a wearable device (30) , wherein the optoelectronic module (1) comprises an optical sensor (20) that is configured to generate sensor data based on ambient light in a field of view of the optical sensor (20) , the ambient light illuminating a scene to be captured by an external camera device (50) , wherein the optoelectronic module (1) is configured to transmit the sensor data to the external camera device (50) , the sensor data being usable to adjust camera parameters for capturing the scene, and / or for post-processing image or video data of the captured scene .

2. Optoelectronic module (1) according to the preceding claims, wherein the optical sensor (20) is configured to generate sensor data based on spectral characteristics of the ambient light, the sensor data being usable to adjust a white balance and / or color fidelity of an image or video of the captured scene.

3. Optoelectronic module (1) according to one of the preceding claims, wherein the optical sensor (20) is configured to generate sensor data based on flicker of the ambient light, the sensor data being usable to reduce flicker artifacts in an image or video of the captured scene .

4. Optoelectronic module (1) according to one of the preceding claims, wherein the field of view (21, 22) of the optical sensor (20) is different from a field of view (52, 53) of the external camera device (50) and / or from afield of view of another optical sensor integrated in the external camera device (50) .

5. Optoelectronic module (1) according to one of the preceding claims, wherein the optoelectronic module (1) is integrated in a wearable device (30) , the wearable device being:- earphones or earbuds (32) , or- eyeglasses (31) , or- a watch, or- a f itness-tracker , or- a finger ring or earring.

6. Optoelectronic module (1) according to one of the preceding claims, wherein the optoelectronic module (1) is arranged in a housing of the wearable device (30) , wherein the housing is at least partially transparent for ambient light to be detected.

7. Optoelectronic module (1) according to one of the preceding claims, wherein the optoelectronic module (1) is configured to communicate with a camera application of a smartphone or smart device.

8. Optoelectronic module (1) according to one of the preceding claims, wherein the optoelectronic module (1) further comprises an optically based sensor (25) configured to receive force inputs and / or gesture inputs from a user using the wearable device (30) , and wherein the optical sensor (20) and the optically based sensor (25) share a common analog front end circuitry of the optoelectronic module (1) .

9. Optoelectronic module according to one of the preceding claims, wherein the sensor data of the optical sensor (20) is usable to increase immunity of an optically based sensor (25) to ambient light variations, the optically based sensor (25) being integrated in or being arranged outside of the optoelectronic module (1) .

10. Optoelectronic module (1) according to one of the preceding claims, further comprising an emitter (18) , wherein the optical sensor (20) is configured to generate sensor data based on an intensity of light, and wherein:- if an intensity of the ambient light sensed by the optical sensor (20) is below a predefined threshold, the emitter (18) is configured to emit a light signal; or- the emitter (18) is configured to emit light towards human tissue of a user using the wearable device (30) , and the optical sensor (20) is configured to sense said light after reflection or transmission by the human tissue.

11. Camera device (50) , the camera device (50) being configured to capture a scene being illuminated by ambient light, the camera device (50) further being configured to receive sensor data from an external optoelectronic module (1) integrated in a wearable device (30) , the sensor data containing information about the ambient light that illuminates the scene and being usable to adjust camera parameters for capturing the scene, and / or for post-processing image or video data of the captured scene.

12. Optoelectronic system (100) comprising the optoelectronic module (1) according to one of claims 1 to 10 and the camera device (50) according to claim 11.

13. Method for adjusting camera parameters, the method comprising :- generating, by an optical sensor (20) comprised by an optoelectronic module (1) integrated in a wearable device (30) , sensor data that is based on ambient light in a field of view of the optical sensor (20) , the ambient light illuminating a scene,- transmitting the sensor data from the optoelectronic module (1) to a camera device (50) configured to capture the scene, wherein the optoelectronic module (1) is outside the camera device (50) ,- using, by the camera device (50) , the sensor data to adjust camera parameters for capturing the scene, and / or for post-processing image or video data of the captured scene.

14. Method according to the preceding claim, wherein the sensor data is based on spectral characteristics of the ambient light, and using the sensor data comprises adjusting a white balance and / or color fidelity of the image or video of the captured scene.

15. Method according to claims 13 or 14, wherein, during operation of the optoelectronic module (1) and of the camera device (50) , the field of view of the optical sensor (20) is different from a field of view of the camera device (50) and / or from a field of view of another optical sensor integrated in the camera device (50) .

Images