Virtual Image Signal Processor

Abstract

Equipped with a digital camera: at least one image sensor configured to generate image sensor data; an image signal processor configured to receive and perform a plurality of camera-internal processing operations on the image sensor data, wherein the processing operations are executed sequentially, and wherein such processing operations result in a plurality of device-dependent intermediate processed versions of the image sensor data, which are different from any image type in an exchangeable format, enabling cross-platform viewing of an image associated with the image sensor data; and a communication module configured to wirelessly transmit image source data to a camera-external image signal processor, wherein the image source data includes one of the device-dependent intermediate processed versions of the image sensor data, and wherein such transmission is performed automatically in response to the generation of image sensor data.

Description

SUMMARY

[0001] A digital camera is provided. The digital camera includes at least one image sensor configured to generate image sensor data. The digital camera further includes an image signal processor configured to receive and perform multiple on-camera processing operations on the image sensor data, such processing operations resulting in multiple intermediate processed versions of the image sensor data, which are different from any image type in an exchangeable-format image type, enabling cross-platform viewing of an image associated with the image sensor data.The digital camera further comprises a communication module configured to wirelessly transmit image source data to an off-camera image signal processor, wherein the image source data comprises at least one of the following: (i) the image sensor data and (ii) one of the intermediate processed versions of the image sensor data, wherein such transmission is performed automatically in response to the production of image sensor data. BACKGROUND

[0002] Digital cameras use various image sensors (CCD, CMOS, etc.) to convert light received by optical components (e.g., lenses) into electronic signals. These electronic signals are then sent through on-camera processing pipelines to generate exchangeable-format images, which are stored on the camera. These images can be displayed on the camera's screen and can also be manually transferred to another device, such as a desktop computer, laptop, tablet, or other computing device. Exchangeable-format images are in a device-independent format that is shared between computing devices, such as JPEG, GIF, etc. The processing pipeline includes an image signal processor (ISP), which is primarily implemented in hardware within the camera.The dedicated hardware implementation is necessary due to the high computational intensity of the processing operations within the ISP and enables the relatively rapid generation of high-quality, exchange-format images. The camera's internal processing operations can include area correction, defect pixel correction (DPC), lens shadow correction, demosaicing, etc.

[0003] The inventor identified several disadvantages of the typical camera-integrated ISP discussed above. For example, implementing the ISP via hardware components reduces its adaptability and prevents performance improvements, updates, and processing enhancements. Additionally, developing improved hardware for camera-integrated (on-board) ISPs can be time-consuming, extending the iterative improvement period and increasing the time it takes for new innovations to reach the market. Furthermore, it may be desirable to display an image on the camera with minimal delay after exposure and low power consumption. Conversely, producing a high-quality image may also be desirable, which can increase the complexity of the on-board systems and consume more power.The aforementioned goals (i.e., latency, power consumption, and image quality) are usually at odds. Therefore, generating images via an on-camera ISP or camera-internal ISP can involve trade-offs between latency, power consumption, and image quality.

[0004] Methods for image sensor data processing in image recording devices are known from US 6 535 243 B1, US 8 471 924 B2, and US 8 237 801 B2. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 and Fig. Figure 2 shows schematic representations of a digital photography system; Fig. Figure 3 shows an example of a processing operations pipeline used in the digital photography system in Fig. 1 and Fig. 2 is shown, includes; and Fig. Figures 4A to 4B show a method for image processing in a digital photography system. DETAILED DESCRIPTION

[0005] This document describes a digital photography system for automatically transferring camera-specific image source data to a camera-external image signal processor (ISP) for processing. Offloading processing to a remote resource provides greater flexibility in the ISP and allows it to utilize a larger amount of computing power for image processing. As a result, the image quality of the processed images is improved when using a camera-external ISP compared to the image quality achievable with an internal camera ISP. Furthermore, when using a camera-external ISP, processing operations can be updated based on data (e.g., metadata) received from the camera, set by the factory, and / or based on user preferences to further enhance image quality.Additionally, the camera-external ISP can generate inbound control signals and send inbound control signals to the digital camera to improve image quality and overall camera performance.

[0006] Fig. 1 and Fig. Figure 2 are schematic representations showing aspects of a digital photography system that uses automatic remote ISP processing. Fig. Figure 3 shows a plurality of processing operations that can be performed by both an internal and an external camera ISP, which are included in the digital photography system described in Fig. 1 and Fig. 2 is shown. Fig. Figures 4A to 4B show a method for image processing in a digital photography system.

[0007] Fig. Figure 1 shows a digital photography system 10 configured to generate digital images from exposures captured by a digital camera 12, among other things. The digital photography system 10 comprises the digital camera 12 in electronic communication with an off-camera processing subsystem 14, which includes an off-camera ISP 16. It is estimated that the digital camera 12 may be located remotely from the off-camera processing subsystem 14 in some embodiments. A network, such as the internet, can provide electronic communication between the digital camera 12 and the off-camera processing subsystem 14. In this way, the off-camera ISP 16 can be accessed by a cloud computing platform, thus increasing the accessibility of the ISP.Therefore, the camera-external ISP 16 can be accessed via the digital camera at any location where the digital camera 12 has connectivity to the Internet.

[0008] A communication module 18, which is included in the digital camera 12, is configured to provide wireless connectivity to the camera-external processing subsystem 14 and, in particular, to the camera-external ISP 16. It is estimated that the wireless connectivity can be achieved through one or more suitable communication platforms, such as a mobile communication network (e.g., 3G, 4G, etc.), Bluetooth, Wi-Fi, etc. Furthermore, in some examples, the communication module 18 may include an application programming interface (API). The camera-external subsystem 14 may also include a communication module 20 to enable electronic communication. In other embodiments, however, the electronic communication functionality of the camera-external processing subsystem 14 may be integrated into the camera-external ISP 16.

[0009] The digital camera 12 can also include an actuator 22. Examples of actuators include a shutter button, a touch interface, an electronic switch, a motion sensor, a timer, a communication signal, etc. The actuator 22 can be actuated via an actuation input 24, which causes the digital camera 12 to perform at least one exposure or another suitable image acquisition operation. In one example, the actuator can cause the camera to take multiple exposures. Furthermore, in another example, the exposures can be frames in a video sequence. The actuation input 24 can be a user input and / or an input signal from a controller 26 integrated into the digital camera 12. Thus, the actuation input can be manual or automatic.When an exposure is recorded, light passes through one or more optical components 28 and is directed onto an image sensor 30. The optical components 28 can include lenses, filters, mirrors, etc. It is estimated that the digital camera can include multiple image sensors in other embodiments.

[0010] The image sensor 30 generates image sensor data 32 in response to receiving the light projected through its optical components. The image sensor data can include data corresponding to one or more exposures and / or a video sequence comprising multiple frames. The image sensor data 32 can be RAW data. In the illustrated embodiment, the image sensor data 32 is transmitted through an analog-to-digital converter 34, which converts analog data from the image sensor into digital data. In other embodiments, however, the output of the image sensor can be digital, or the converter can be integrated into the image sensor 30. Instead of the signals being RAW format data directly from the sensor, RAW format data can result from a relatively minimal amount of in-camera processing.In any case, ROH data is device-dependent and is not in an exchange format that can be shared and viewed across a variety of different devices.

[0011] Continuing with the figure, the image sensor data 32 is transferred to an on-camera ISP 36. The on-camera ISP 36 is configured to perform a plurality of processing operations 38 on the image sensor data 32. The processing operations can be performed sequentially, and each processing operation can result in an intermediate processed version. The intermediate processed versions are device-dependent and may not be in an exchange format. For example, the device-dependent intermediate processed versions can be exchanged within the system in a proprietary format. Furthermore, the intermediate processed versions are distinct from any image type in an exchange format that allows cross-platform viewing of an image associated with exposure. Exchange formats can include JPEG, GIF, etc.

[0012] The camera's internal ISP 36 is also configured to perform the majority of the processing operations 38, culminating in an image 40 (e.g., an interchangeable image). The image 40 is sent to a display 42 in the digital camera 12 for viewing. In this way, a user can preview an image on an on-board display after an exposure has been taken.

[0013] The communication module 18 is further configured to send image source data 44, received from the camera's internal ISP 36 and / or the image sensor 30, to the camera's external ISP 16. The image source data 44 can include at least one of the image sensor data 32 and one of the intermediate processed versions in the processing pipeline, in which processing operations could be executed sequentially. The processing operations are described in greater detail with regard to Fig. 3 discussed herein. The image source data 44 is device-dependent. This means that the image source data 44 must not be, or need not be, interchangeable with other computing devices, digital cameras, etc., which are configured to view exchange format images such as JPEGs, GIFs, etc. Additionally, the image source data 44 can be compressed by the camera's internal ISP 36 before transmission. Therefore, the camera's external ISP 16 can also be functionally configured to compress the image source data 44. Compressing the image source data can reduce the time required to transmit the image source data from the digital camera to the camera's external ISP.

[0014] Additionally, the communication module 18 is configured to send on-camera metadata 46 to the camera-external ISP 16. It is anticipated that in other embodiments, the metadata 46 can be combined with the image source data 44 and simultaneously transmitted to the camera-external ISP. The on-camera metadata 46 can include at least one type of calibration data, identification data (e.g., hardware specification data, factory information data), and local setting data specific to the digital camera. For example, the identification data can include an identification number and / or text associated with the camera model, hardware within the camera such as the image sensor, and / or optical components. The identification data can also include tags (e.g., descriptive keywords) associated with the exposure.Local setting data can include shutter speed, ISO setting data, sensor gain data, in-camera processing operation data (e.g., processing operations performed internally within the camera, the order of processing operations performed within the camera, etc.), defective pixel data, lens shading data, lens focusing data, and / or lens aperture. Calibration data can include defective pixel lists, dark signal data, the lens shading correction function, and spectral sensitivity characteristics. This allows the identification of camera-specific hardware and settings for use in external image processing.In one example, a camera-specific identification number and / or image sensor identification number can be included in the metadata, enabling the camera-external ISP 16 to configure image processing based on the identification numbers.

[0015] The camera-external ISP 16 is configured to perform a plurality of processing operations 48. These camera-external processing operations can be implemented in hardware and / or software. In some examples, at least some of the processing operations 48 are executed via software modules on a computing device (e.g., a remote server). Therefore, in some examples, at least part of the camera-external ISP may be stored in memory as instruction code, which can be executed by a processor. In this way, the processing operations can be virtualized to increase the flexibility and adaptability of the camera-external processing operations. As a result, the camera-external processing operations can be based on data from the digital camera, user preferences, camera manufacturer updates, etc.The camera-external ISP 16 can be updated and / or adjusted to enhance the image quality of an image generated outside the camera and / or to provide greater custom processing, if desired. Additionally, the camera-external ISP 16 can have greater processing capacity than the camera-internal ISP 36, enabling further image quality enhancement, if desired. The camera-external processing subsystem 14 can also include a storage device 50 for storing any of the data received by the camera-external processing subsystem described above. In the event that camera-external changes are implemented as hardware modifications / updates, the benefit of these changes can be obtained without any changes to the local hardware (e.g., digital camera 10).

[0016] Fig. Figure 2 shows the digital photography system 10, after the image source data 44 and the metadata 46, shown in Fig. 1, received via the camera-external ISP 16. The camera-external ISP 16 is configured to perform at least some of the processing operations 48 on the image source data. The image processing operations are further performed in Fig. 3 specified.

[0017] In some examples, the processing operations are based on the metadata 46, which is in Fig. 1 are shown, controlled, or configured. For example, an image sensor identification number may be included in the metadata. In such an example, the processing operations can be configured based on image sensor data associated with the identified image sensor stored in the camera-external processing subsystem 14 (e.g., configured for a specific sensor type or a sensor for a specific manufacturer). Therefore, the camera-external processing subsystem 14 may have camera-external metadata 204 stored within it (e.g., cloud-stored data) and may be associated with specific image sensors, digital cameras, etc. The camera-external metadata may be provided by camera suppliers and components (e.g., sensor manufacturers) or acquired during calibration at the factory.In this way, the processing operations can be configured and adjusted (tuned) based on the camera model, image sensor type, etc., thereby increasing the adaptability of the camera-external ISP 16. Furthermore, offline processing can be configured based on selections made at the camera. For example, based on the exposure measurement reported by the camera or based on a user selection, a more detailed and longer reconstruction could be automatically chosen, which would direct the image to a specific set of servers and resources capable of performing such an operation.

[0018] Furthermore, in one example, the processing operations can be controlled or set based on the image source data 44. For example, an intermediate processed version included in the image source data can trigger the processing operations in the pipeline to be executed downstream of the processing operation preceding the intermediate processed version, which is executed via the camera's internal ISP. If only the image sensor data 32, shown in Fig. 1, to which camera-external ISP 16 are sent, in another example all of the processing operations 48 can be selected for execution.

[0019] Furthermore, in some examples, the camera-external ISP can receive camera-external metadata from external sources. For example, the camera-external metadata can be crowdsourced. Specifically, in one example, the user might want the image content to be recognized, in which case the images can be directed to offline resources capable of pattern recognition using, for example, computer vision, or a crowdsourcing service staffed by real people who look at the image and type in a description.

[0020] The processing operations 48, which are performed on the image signal data via the camera-external ISP 16, can generate an exchangeable image or exchange image 200. As shown, the image 200 can be stored in the storage device 50, which is part of the camera-external processing subsystem. The image can be accessed by a user at a later time via the digital camera, a desktop computer, a laptop computer, a tablet, etc.

[0021] Additionally, image 200 can be transmitted to the digital camera 12 and, in particular, to the communication module 18. In one example, image 200 might be a file. In other examples, however, the image data is organized in a different suitable way. Thus, the communication module 18 is configured to receive image 200. Subsequently, image 200 can be displayed and / or saved on-board or on the digital camera.

[0022] The camera-external ISP 16 can also be configured to generate a control signal 202 and send the control signal 202 to the communication module 18 in the digital camera 12. Therefore, the communication module 18 can be configured to receive the incoming control signal 202.

[0023] In another example, the control signal 202 can be based on the image source data 44 and / or the metadata 46, shown in Fig. 1. A control signal can be generated. For example, the image source data and / or metadata can be analyzed to determine whether the digital camera setting can be replicated to improve image quality. If it is determined that this camera setting would improve image quality, a control signal can be generated.

[0024] The control 26 can be operational, using the incoming control signal 202 to control an operation of the digital camera for a subsequent exposure. Controlling the operation of the digital camera can include setting at least one focus setting, one exposure control setting, one sensor gain setting, one white balance setting, and one flash control setting for the subsequent exposure. It is estimated that the control signal can be configured to change any non-reversible process or setting within the digital camera.

[0025] Additionally, an update source 250, such as the camera manufacturer, image sensor manufacturer, etc., can provide a processing operation update 252 to the camera-external ISP 16. The processing operation update 252 can be configured to modify one or more of the camera-external processing operations 48. In this way, the camera-external processing operations can be updated periodically to iteratively improve the processing operations. The update source 250, or another suitable external source, can also provide metadata to the camera-external ISP, in some examples.

[0026] Fig. Figure 3 shows an illustration of a plurality of processing operations 300 that can be performed using the camera's internal ISP 36 and / or the camera's external ISP 16. As discussed above, each of the processing operations can generate (render) an intermediate processed version 302 of the image sensor data obtained by the ISP. The processing operations 300 can be performed sequentially in the order shown. However, other sequential arrangements of the processing operations are also considered. Furthermore, in other embodiments, only a subset of the processing operations can or needs to be performed using one or both of the ISPs. Additionally, two or more of the processing operations can be combined (e.g., algorithmically combined) to form a single processing operation. In this way, the processing efficiency of the ISPs can be increased.Thus, the operations can be executed in a processing pipeline. The processing pipeline can conceptually be divided into a frontend and a backend. The frontend of the pipeline is in . Fig. Figure 3 shows that the front-end processing operations can be configured to generate an exchangeable image defined in objective terms. The pipeline's back end can be configured to encode, compress, render, and / or color-change the processed data to produce an image in an exchangeable format, such as GIF, JPEG, etc. It is estimated that the aforementioned processing operations may be included in the camera's internal ISP and the camera's external ISP in some embodiments.

[0027] As above with reference to Fig. 1 and Fig. As discussed in section 2, both the camera's internal and external ISPs (36 and 16, respectively) can receive image sensor data. The camera's external ISP can also receive an intermediate processed version of the image sensor data. In particular, in the Fig. In the embodiment shown in Figure 3, the image sensor data 304 is sent to a range correction operation 306, which is included in the processing operations 300. The range correction operation 306 is performed on the image sensor data 304. The range correction operation can adjust numerical ranges in the image sensor data to desired values ​​and generate an intermediate processed version of the image sensor data. It is estimated that each of the intermediate processed versions 302 generated after each operation may be dissimilar to another. Furthermore, the intermediate processed versions may be dissimilar to the interchangeable image 200, which is displayed in Figure 3. Fig. 2 is shown. Furthermore, the intermediate processed versions can be different from or distinct from any image type in an exchangeable format that allows cross-platform viewing of an image associated with exposure. It is estimated that at least some of the in Fig. The three operations shown do not necessarily have to be performed sequentially in the order shown, nor do they need to be in other embodiments. Furthermore, in some embodiments, at least some of the operations can be performed at overlapping time intervals.

[0028] Next, a Defect Pixel Correction (DPC) operation 308 is performed on an intermediate processed version of the image sensor data, and another intermediate processed version of the data is generated. The DPC operation 308 can correct defective pixels in the image sensor 30, as shown in Fig. 1 and Fig. 2. Identify and modify the image sensor data to correct the defective pixels. In addition, in one embodiment, DPC operation 308 can also include crosstalk correction and / or non-linearity correction. The DPC operation can also be referenced as a sensor interface block. DPC operation 308, which is described in Fig. As shown in 3, another intermediate processed version can be generated.

[0029] Continuing with Fig. 3. A lens shading correction operation 310 is performed on an intermediately processed version of the image sensor data, and another intermediately processed version of the image sensor data is generated. The lens shading correction operation can correct variations in the amount of light passing through lenses in the camera. Additionally, circuits, glass plates, plastic plates, etc., may be positioned in front of the image sensor. Therefore, the lens shading correction can also correct shading caused by circuits in the camera that obscure the lenses or other optical components. In particular, an algorithm in the operation can fit the surface of a lens to the pixel values. A Bessel interpolator can be used to implement this functionality. In one embodiment, the Bessel interpolator can be executed in real time.

[0030] Next, a white balance operation 302 is performed on the intermediately processed version of the image sensor data, and another intermediately processed version of the image sensor data is generated by performing this operation. The white balance operation 312 can be configured to normalize red, blue, and green (RGB) values ​​in the data. For example, the image sensor 30 might have unequal RGB values ​​for a white object, which could be caused by the transmittance of the filter and / or the color of the light. The white balance operation can also include a gain operation, which can be configured to adjust the camera's sensitivity to light.

[0031] Next, a demosaicing operation 314 is performed on an intermediate processed version of the image sensor data, and another intermediate processed version of the image sensor data is generated by executing the operation. The demosaicing operation can be configured to interpolate a set of RGB values ​​for each point in the image sensor data. Thus, the demosaicing operation can modify the data based on human luminescence perception. In one example, the demosaicing operation can be interpolated based on a Bayer pattern. Furthermore, in some examples, the interpolation can be implemented by convolution or direct computation.

[0032] Next, a Color Conversion Matrix (CCM) operation 316 is performed on an intermediate processed version of the image sensor data, and another intermediate processed version of the image sensor data is generated by executing the operation. The CCM operation 316 can be configured to correct for variations in camera filters. For example, the characteristics of camera filters may differ between manufacturers, production runs, etc. Additionally, a filter standard may be used for certain objects. Therefore, a 3-to-3 transformation can be used to generate predictable RGB values ​​in the CCM operation in some implementations.

[0033] Next, a sharpening operation (320) is performed on an intermediate processed version of the image sensor data, and another intermediate processed version of the image sensor data is generated by executing the operation. The sharpening operation can be used to sharpen and / or blur various aspects of the image data.

[0034] Next, a noise reduction operation 322 is performed on an intermediate processed version of the image sensor data, and another intermediate processed version of the image sensor data is generated by executing the operation. The noise reduction operation can be configured to remove artifacts from the data to reduce noise in the data and improve image quality.

[0035] Next, a gamma correction operation 324 is performed on an intermediate processed version of the image sensor data. The gamma correction operation 322 can be configured to apply a logarithmic function to the image sensor data to encode luminance based on human perception of luminance, thereby reducing the overall size of the image data by improving how the existing bits are allocated to distinguish light levels with respect to perception. In either the camera's internal ISP and / or camera's external ISP, after the gamma correction operation is implemented, the data can be encoded in an exchangeable format and / or compressed to produce an exchangeable format image.

[0036] Fig. 4A and Fig. Figure 4B shows a method 400 for image processing in a digital photography system. As shown, the method is implemented via a digital camera and a camera-external ISP. The digital camera and the camera-external ISP shown in method 400 can be similar to the digital camera 12 and the camera-external ISP 16 included in the digital photography system 10, which were described above with reference to or in relation to Fig. 1 to 3 are discussed in some embodiments. In other embodiments, the digital camera and the camera-external ISP may be included in a different digital photography system.

[0037] The procedure includes, at 402, what in Fig. Figure 4A shows the actuation or activation of an actuator. Next, the method described in Figure 404 involves taking an exposure in response to the actuation of the actuator. Taking an exposure may involve allowing light to pass through a plurality of optical components (e.g., lenses, filters, etc.) to reach an image sensor.

[0038] The next step in procedure 406 involves generating image sensor data in response to the exposure being recorded. It is estimated that step 406 can be performed using an image sensor. In step 408, the procedure involves transferring the image sensor data to an external ISP. Furthermore, in some examples, the image sensor data may be converted from analog to digital data prior to step 408.

[0039] In step 410, the procedure involves performing multiple camera-internal processing operations to produce multiple intermediate processed versions of the image sensor data. As discussed above, these intermediate processed versions are distinct from any image type of an exchangeable format, allowing cross-platform viewing of an image associated with exposure. As shown, steps 402-410 are implemented in the digital camera.

[0040] In 412, the method comprises an automatic transfer of the image source data to a camera-external image signal processor (ISP) in response to the acquisition of the exposure and / or the generation of the image sensor data. Automatic transmission of image source data to the camera-external ISP may include sending the image source data to the camera-external ISP from the digital camera and receiving the image source data at the camera-external ISP from the digital camera.

[0041] The next step in the procedure, step 414, may involve transmitting metadata from the digital camera to the camera-external ISP. It is estimated that step 414 may be implemented automatically in response to exposure capture and / or image sensor data production. Wireless transmission of metadata from the camera-external ISP to the camera-external ISP may involve sending metadata from the digital camera to the camera-external ISP and receiving the metadata from the digital camera at the camera-external ISP.

[0042] At 416, which is in Fig.As illustrated in Figure 4B, the method can involve performing a number of camera-internal processing operations on the image source data via the camera's internal ISP to generate an interchangeable format image. In this way, a preview image can be generated for display on the camera. Next, the method described in Figure 418 involves displaying the interchangeable format image. Steps 416-418 are implemented via the digital camera.

[0043] At 420, the method can include controlling one or more of the camera-external processing operations based on metadata. In this way, the camera-external processing operations can be selected and / or set based on the specific characteristics of the digital camera, such as the type of image sensor, the type of optical components, defective pixels within the image sensor, and setting data (e.g., shutter speed, flash setting, gain settings, and / or focus settings). The gain settings (e.g., ISO setting) can include the sensor gain applied to data from the sensor to adjust the sensor's sensitivity to light.

[0044] Next, the method described in 422 can include controlling one or more of the camera-external processing operations based on the image sensor data. For example, the camera-external processing operations that are selectively applied to the image source data can be chosen based on the intermediate processed version of the image sensor data transmitted to the camera-external ISP. In particular, in one embodiment, the processing operations that sequentially follow an intermediate processed version contained in the image source data sent to the camera-external ISP can be selected for execution by the camera-external ISP.

[0045] Next, the method described in 424 involves performing multiple camera-external processing operations on the image source data to generate an exchangeable image. In some embodiments, the exchangeable image may be in a proprietary format. In other examples, however, the exchangeable image may be in a standard format. It is estimated that the image may be of higher quality than the image generated on-board by the camera due to the greater amount of processing capacity or power available in the camera-external ISP, as well as the adaptability of the camera-external ISP. Thus, the strengths of cloud computing can be leveraged to provide a higher-quality image to the user of a digital camera.Furthermore, in some embodiments, the camera-external processing operations can be updated before being performed. For example, the digital camera manufacturer can periodically update the camera-external processing algorithms to iteratively improve the processing operations.

[0046] Next, in step 426, the procedure can include storing the exchangeable image locally or on a remote server / resource. This allows the image to be saved by a user for remote viewing at a later time. For example, a user can access the image via a personal computer, such as a desktop computer, laptop, and / or tablet. Steps 420-426 are implemented via the camera's external ISP.

[0047] The next step in step 428 may involve transmitting an interchangeable format image (e.g., an interchangeable standard format image) to the digital camera. It is estimated that step 428 can be implemented automatically in response to the generation of the interchangeable format image. Subsequently, the image can be viewed by the user via an internal camera display. In some examples, if there is a high-speed connection between the digital camera and the camera's external ISP, the interchangeable format image can be displayed on the digital camera as a preview image for the corresponding exposure and / or image sensor data.

[0048] At 430, the procedure can involve generating a control signal. This control signal can be configured to control various aspects of the digital camera, such as focus setting, exposure control setting (e.g., shutter speed and aperture), sensor gain setting, white balance setting, and / or flash control setting for subsequent exposures. It is estimated that the control signal can adjust any non-reversible process or setting within the digital camera. In this way, the camera's external ISP can adjust settings for subsequent exposures taken with the digital camera, thus enabling subsequent exposure settings to improve the quality of the images produced by those exposures.

[0049] The next step in step 432 involves transmitting the control signal to the digital camera. In step 434, the method involves controlling an operation of the digital camera during the recording of a subsequent exposure based on the control signal. Step 434 can be implemented via a control circuit included in the digital camera. Furthermore, controlling an operation of the digital camera can include setting one or more exposure control settings (e.g., shutter speed and aperture), sensor gain settings, white balance settings, focus settings, and flash control settings for the subsequent exposure. It is anticipated that such interactive control between offline components and a camera can also be extended to a set of cameras.For example, after a camera at a specific location provides a valid light level reading, the same values ​​can be used by other cameras with sufficient temporal and spatial proximity.

[0050] Embodiments of the present invention include the following concepts: Concept 1. Digital camera comprising: at least one image sensor configured to generate image sensor data; an image signal processor configured to receive and perform a plurality of camera-internal processing operations on the image sensor data, wherein such processing operations result in a plurality of intermediate processed versions of the image sensor data, which are different from any image type in an exchangeable format, enabling cross-platform viewing of an image associated with the image sensor data;and a communication module configured to wirelessly transmit image source data to a camera-external image signal processor, wherein the image source data comprises at least one of: (i) the image sensor data and (ii) one of the intermediate processed versions of the image sensor data, wherein such transmission is performed automatically in response to the generation of image sensor data. Concept 2. Digital camera according to Concept 1, wherein the camera's internal processing operations are configured to produce an interchangeable format image, and wherein the digital camera further comprises a display configured to display the interchangeable format image. Concept 3. Digital camera according to Concept 1, wherein the communication module is configured to receive an interchangeable format image from the camera's external image signal processor, which is unlike any of the intermediate processed versions. Concept 4. Digital camera according to Concept 1, wherein the communication module is operable to receive an incoming control signal from the camera-external image signal processor, which is generated by the camera-external image signal processor based on the image source data, wherein the digital camera further comprises a control which is operable to use that incoming control signal to control an operation of the digital camera for a subsequent exposure. Concept 5. Digital camera according to Concept 4, wherein the control is operational to use the incoming control signal to set at least one of a focus setting, an exposure control setting, a sensor gain setting, a white balance setting and a flash control setting for the subsequent exposure. Concept 6. Digital camera according to Concept 1, wherein the communication module is configured to wirelessly transmit metadata to the camera's external image signal processor. Concept 7. Digital camera according to Concept 6, wherein the metadata includes at least one of calibration data, identification data and local setting data corresponding to the digital camera. Concept 8. Digital camera according to Concept 6, wherein the communication module is operational to receive an incoming control signal from the camera-external image signal processor, which is generated by the camera-external image signal processor based on the metadata, wherein the digital camera further comprises a control which is operational to use the incoming control signal to control the operation of the digital camera for a subsequent exposure. Concept 9. Digital camera according to Concept 8, wherein the incoming signal, which is generated by and received by the camera-external image signal processor, is further based on the image source data, such that the subsequent exposure is controlled based on both the image source data and the metadata. Concept 10. Digital photography system comprising: a camera-external image signal processor; and a digital camera, comprising: an image sensor configured to generate image sensor data in response to the digital camera taking an exposure; an image signal processor configured to receive and perform a plurality of camera-internal processing operations on the image sensor data, wherein such processing operations result in a plurality of intermediate processed versions of the image sensor data and an exchangeable format image based on the image sensor data, wherein the intermediate processed versions of the image sensor data are different from any image type in an exchangeable format that allows cross-platform viewing of an image associated with the exposure; a display configured to display the exchangeable format image;and a communication module configured to wirelessly transmit image source data to the camera-external image signal processor, wherein the image source data comprises at least one of the following: (i) the image sensor data and (ii) one of the intermediate processed versions of the image sensor data, wherein such transmission is performed automatically in response to the acquisition of the exposure. Concept 11. Digital photography system according to Concept 10, wherein the camera-external image signal processor is configured (i) to perform a plurality of camera-external processing operations which result in an interchangeable format image and (ii) to send the interchangeable format image to the digital camera. Concept 12. Digital photography system according to Concept 11, wherein the camera-external image signal processor is configured to receive a processing operation update from an update source, wherein the processing update updates one or more of the camera-external processing operations. Concept 13. Digital photography system according to Concept 11, wherein the camera-external processing operations and the camera-internal processing operations both include at least one of an area correction operation, a defect pixel correction operation, a lens shading correction operation, a white balance enhancement operation, a color conversion matrix correction operation, a focusing operation, a noise reduction operation, and a gamma correction operation. Concept 14. Digital photography system according to Concept 10, wherein the camera-external image signal processor is configured to associate camera-external metadata with the image source data. Concept 15. Digital photography system according to Concept 10, wherein the communication module is further configured to wirelessly transmit camera-internal metadata associated with the image source data to the camera-external image signal processor, wherein the camera-internal metadata includes at least one of calibration data, identification data, and local setting data corresponding to the digital camera. Concept 16. Digital photography system according to Concept 15, wherein the camera-external image signal processor is configured to perform a plurality of camera-external processing operations on the image source data, which are controlled at least in part based on the camera-internal metadata. Concept 17. A method for image processing in a digital photography system, comprising: in the digital camera, generating image sensor data via an image sensor; transmitting the image sensor data to an in-camera image signal processor; performing a plurality of in-camera processing operations to produce a plurality of intermediate processed versions of the image sensor data, wherein the intermediate processed versions are different from any image type in an exchangeable format that enables cross-platform viewing of an image associated with the image sensor data; in response to the generation of the image sensor data, automatically transmitting image source data to an external image signal processor, wherein the image source data comprises at least one of the following: (i) the image sensor data and (ii) one of the intermediate processed versions of the image sensor data;and in the case of the camera-external image signal processor, performing a plurality of camera-external processing operations on the image source data to produce an exchangeable format image.; Concept 18. Method according to Concept 17, further comprising: (i) transmitting metadata from the digital camera to the camera-external image signal processor; and (ii) at the camera-external image signal processor, generating one or more of the camera-external processing operations based on the metadata. Concept 19. Method according to Concept 17, further comprising: (i) wirelessly transmitting metadata from the digital camera to the camera-external image signal processor; (ii) at the camera-external image signal processor, generating a control signal based on the metadata; (iii) transmitting the control signal to the digital camera; and (iv) controlling the operation of the digital camera during the recording of a subsequent exposure based on the control signal. Concept 20. Method according to Concept 17, further comprising: (i) in the camera-external image signal processor, generating a control signal based on the image source data; (ii) transmitting the control signal to the digital camera; (iii) controlling an operation of the digital camera during the recording of a subsequent exposure based on the control signal. Concept 21. Method according to Concept 20, wherein controlling the operation of the digital camera during the recording of the subsequent exposure comprises controlling one or more of a focus setting, an exposure control setting, a white balance setting, a sensor gain setting, and a flash control setting for a subsequent exposure based on the control signal.

[0051] It is estimated that the methods described herein are provided for illustrative purposes only and are not intended to be limiting. Accordingly, it is estimated that in some embodiments the methods described herein may include additional or alternative processes, while in other embodiments the methods described herein may include some processes that can be rearranged, performed in parallel, or omitted without departing from the scope of this disclosure. Furthermore, it is estimated that the methods described herein can be performed using any suitable software and hardware, including the specific examples described herein.

[0052] This written description uses examples to disclose the invention, including the best mode of use, and also to enable a person skilled in the art to put the invention into practice, including manufacturing and using any devices or systems and carrying out any incorporated methods. The patentable scope of the invention is defined in the claims and may include other examples as understood by a person skilled in the art. Such other examples are intended to be within the scope of the claims.

Claims

[1] Equipped with a digital camera: at least one image sensor configured to generate image sensor data; an image signal processor configured to receive and perform a plurality of camera-internal processing operations on the image sensor data, wherein the processing operations are executed sequentially, and wherein such processing operations result in a plurality of device-dependent intermediate processed versions of the image sensor data, which are different from any image type in an exchangeable format, enabling cross-platform viewing of an image associated with the image sensor data; and a communication module configured to wirelessly transmit image source data to a camera-external image signal processor, wherein the image source data includes one of the device-dependent intermediate processed versions of the image sensor data, and wherein such transmission is performed automatically in response to the generation of image sensor data. [2] Digital camera according to claim 1, wherein the camera's internal processing operations are configured to produce an interchangeable format image, the digital camera further comprising a display configured to display the interchangeable format image, the communication module being configured to receive an interchangeable format image from the camera's external image signal processor, which is unlike any of the intermediate processed versions. [3] Digital camera according to claim 1, wherein the communication module is operable to receive an incoming control signal from the camera-external image signal processor, which is generated by the camera-external image signal processor based on the image source data, wherein the digital camera further comprises a controller which is operable to use that incoming control signal to control the operation of the digital camera for a subsequent exposure, wherein the controller is operable to use that incoming control signal to set at least one of a focus setting, an exposure control setting, a sensor gain setting, a white balance setting and a flash control setting for the subsequent exposure. [4] Digital camera according to claim 1, 2 or 3, wherein the communication module is configured to wirelessly transmit metadata to the camera-external image signal processor. [5] Digital camera according to claim 4, wherein the metadata includes at least one of calibration data, identification data and local setting data corresponding to the digital camera. [6] Digital camera according to claim 4, wherein the communication module is operational to receive an incoming control signal from the camera-external image signal processor, which is generated by the camera-external image signal processor based on the metadata, wherein the digital camera further comprises a control which is operational to use the incoming control signal to control the operation of the digital camera for subsequent exposure. [7] Digital camera according to claim 6, wherein the incoming signal, which is generated by and received by the camera-external image signal processor, is further based on the image source data, such that the subsequent exposure is controlled based on both the image source data and the metadata. [8] Method for image processing in a digital photography system, comprising: In digital cameras, generating image sensor data via an image sensor; Transmitting image sensor data to a camera-internal image signal processor; Performing a plurality of camera-internal processing operations to produce a plurality of device-dependent intermediate processed versions of the image sensor data, wherein the processing operations are performed sequentially, wherein the intermediate processed versions are different from any image type in an exchangeable format that enables cross-platform viewing of an image associated with the image sensor data; In response to the generation of image sensor data, automatic transmission of image source data to a camera-external image signal processor, wherein the image source data includes at least one of the device-dependent intermediate processed versions of the image sensor data; and at the camera-external image signal processor, performing a plurality of camera-external processing operations on the image source data to produce an interchangeable format image. [9] Method according to claim 8, further comprising: (i) transmitting metadata from the digital camera to the camera-external image signal processor; and (ii) at the camera-external image signal processor, generating one or more of the camera-external processing operations based on the metadata. [10] Method according to claim 8 or 9, further comprising: (i) in the camera-external image signal processor, generating a control signal based on the image source data; (ii) transmitting the control signal to the digital camera; (iii) controlling the operation of the digital camera during the recording of a subsequent exposure based on the control signal, wherein controlling the operation of the digital camera during the recording of the subsequent exposure comprises controlling one or more of a focus setting, an exposure control setting, a white balance setting, a sensor gain setting, and a flash control setting for a subsequent exposure based on the control signal.

Images