Image Difference Generation

Abstract

Methods, systems, and apparatus for image difference generation. In some implementations, raw image data is obtained for an image. Compressed image data is obtained for the image. An encoded frame is generated from the compressed image data. Pixel value differences between at least a portion of the raw image data and the decoded frame are calculated. An image file is generated that includes both the compressed image data and a representation of the pixel value differences.

Description

[Technical field]

[0001] The present disclosure relates to systems and devices for image authentication. [Background technology]

[0002] Image authentication is the application of trusted image encoding and decoding to determine whether a problematic image accurately represents the original data captured by an image sensor, e.g., a camera. Image authentication can be applied to any device or system that receives and displays images, such as camera applications on mobile devices, property management and security devices, any social media application that displays images, etc.

[0003] Image authentication plays a key role in risk identification and fraud identification for various use cases. For example, when an image of a damaged car is sent to an insurance company during an insurance claim, the insurance company may need to determine whether the image of the damaged car is authentic. As another example, a bank may need to determine whether an image on a deposit check has been tampered with. A government may want to check whether an image circulated on social media is a fake image spreading fake news. A user of a social media platform may want to check whether an image is authentic or overly distorted.

[0004] Some image authentication techniques involve generating an authentication image in a specialized image format, which is primarily focused on proving that the raw image data was captured by a camera sensor. The generated authentication image may include (i) the raw image data, (ii) a digital signature over a hash of the raw image data, (iii) a processed image of the raw image data in a compressed image format, and (iv) a digital signature over a hash of the processed image of the raw image data. An example of a specialized image format is the Digital Negative Image Format (DNG). The origin of an image can also be proven using a watermark. For example, an image may include a watermark that identifies the creator or owner of the image.

[0005] One drawback of traditional image authentication is that it can be spoofed: for example, if an image has two separate signatures for the raw and processed images, one of the images, along with its corresponding signature, can be maliciously altered or replaced.

[0006] In one general aspect, a method includes obtaining raw image data for an image, obtaining processed image data for the image, generating a decoded frame from the processed image data, calculating pixel value differences between at least a portion of the raw image data and the decoded frame, and generating an image file that includes both the processed image data and a representation of the pixel value differences.

[0007] Implementations include one or more of the following features. For example, in some implementations, generating a decoded frame from the processed image data includes identifying a frame in the processed image data and reconstructing a red, green, blue (RGB) frame from the frame, where the RGB frame is the decoded frame.

[0008] In some implementations, the RGB frame includes a set of color channels for each pixel in the RGB frame, and calculating the pixel value difference includes identifying, for each pixel in the RGB frame, a color channel of the pixel that matches a color of a corresponding pixel in at least a portion of the raw image data, and discarding, for each pixel in the RGB frame, other color channels that do not match a color of a corresponding pixel in at least a portion of the raw image data, such that the pixel includes a single color channel.

[0009] In some implementations, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame includes identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame, and taking the difference between the value and the second value for each pixel of the at least a portion of the raw image data.

[0010] In some implementations, calculating the pixel value difference between at least a portion of the raw image data and the decoded frame includes determining a scale of at least a portion of the raw image data and scaling the decoded frame to the scale of at least a portion of the raw image data, and identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame includes, for each pixel of the at least a portion of the raw image data, identifying a value of the pixel and a second value of a corresponding pixel in the decoded frame after the decoded frame is scaled.

[0011] In some implementations, the value of the pixel and the second value of the corresponding pixel are values ​​representing color intensity.

[0012] In some implementations, generating an image file that includes both the processed image data and a representation of the pixel value differences includes encoding and compressing the pixel value differences.

[0013] In some implementations, the representation of the pixel value difference is an encoded and compressed pixel value difference.

[0014] In some implementations, the pixel value differences are a difference image, and encoding and compressing the pixel value differences includes partitioning the difference image into one or more macroblocks, encoding the partitioned image to generate an encoded image, and compressing the encoded image to generate an encoded and compressed image.

[0015] In some implementations, generating the image file includes generating a digital signature using a representation of the processed image data and the pixel value differences, and the image file includes the digital signature.

[0016] Other embodiments of these aspects include corresponding systems, apparatus, and computer programs encoded on computer storage devices configured to perform the actions of the methods. One or more computer systems may be so configured by software, firmware, hardware, or combinations thereof installed on the system that, during operation, cause the system to perform the actions. One or more computer programs may be so configured by having instructions that, when executed by a data processing device, cause the device to perform the actions.

[0017] Furthermore, conventional authentication images can have large file sizes because all of the raw data may be included in the authentication image, in addition to a processed version of the image data. Summary of the Invention

[0018] Techniques for image authentication are described herein. The image authentication techniques described below use output generated in an authenticated image format that includes a post-processed image and additional information that allows an image rendering device to reconstruct a previous version of the image. For example, a system for image authentication can include a first computing device that generates an output from a compressed image, such as JPEG, and a previous or original version of the image, such as a raw image (e.g., Bayer RGGB). The output can include the compressed image as well as the additional information. The first computing device can send the output to a second computing device of the system, which can reconstruct the raw image from the output compressed image and the additional information. The second computing device can present the reconstructed image to a user of the second computing device, who can, for example, compare it to the compressed image to authenticate the image or identify changes made to the compressed image during processing.

[0019] The additional information may include a difference or a representation of the difference between the post-processed image and the previous version of the image. These differences may be or represent light intensity differences between the post-processed image and the previous version of the image. As an example, when the previous version of the image is raw image data, the system may first convert the post-processed image data (e.g., JPEG) into a decoded frame, such as a red-green-blue (RGB) reconstructed image, and may proceed to calculate, for each pixel of the raw image data and the corresponding pixel of the RGB reconstructed image, a light intensity difference value between the raw pixel and the corresponding pixel of the RGB reconstructed image. Thus, the calculated difference may include a value for each pixel in the decoded frame. The calculated difference may be referred to or represented in terms of delta.

[0020] Calculating the difference may include selecting a color channel of the decoded frame that corresponds to a pixel color of the previous version image. For example, an RGB reconstructed image may include a red, green, and blue channel for each pixel, each having its own light intensity (e.g., color intensity) value. For each pixel of the RGB reconstructed image, the first computing device may determine which channel value to use by identifying the color of the corresponding pixel in the raw image. More specifically, if the first pixel of the Bayer RGGB raw image is a green pixel, the first computing device may select a value of the green channel for the corresponding pixel of the RGB reconstructed image.

[0021] These differences can be used to reconstruct the previous version of the image. For example, the second computing device can proceed to convert the output post-processed image data (e.g., JPEG) into decoded frames, such as red-green-blue (RGB) reconstructed images, and add the differences to the decoded frames to reconstruct the raw image.

[0022] The differences can be compressed and included in an output image file that also includes the post-processed image data. The output image file can be sent to a computing device configured to use the output image file to reconstruct a previous version of an image from the differences and the post-processed image (e.g., JPEG). For example, by compressing the differences instead of the entire previous version of an image, more efficient compression can be achieved and the size of the output image file can be reduced. For example, it is desirable for the output image file to include the post-processed image, such as a compressed image that a common image viewer can display. However, an output image file that includes both the post-processed image and the previous version of an image, such as a raw image, would be very large. Instead, by calculating the differences and including them in the output image file after compression instead of the previous version of an image, the size of the output image file can be significantly reduced while still providing the ability to reconstruct the previous version of an image.

[0023] As mentioned above, the previous version of image data used to calculate the delta can be raw image data. The raw image data can be, but is not necessarily, sensor data captured by a camera before any processing has been performed. As an example, the camera can be an on-board camera of the first device, or a camera of a different device that communicates with the first computing device. In some cases, raw image data refers to data from a previous stage of an image processing pipeline that can be used to calculate the delta. Furthermore, the term image data refers to any suitable data that can be used to generate an image rendering, and thus includes actual image formats, as well as other binary information or intermediate representations that can be used for the same purpose.

[0024] The subject matter described herein may be implemented in certain embodiments to achieve one or more of the following advantages: Rather than generating two separate signatures for the raw image data and the processed image data as in conventional image authentication formats, the digital signature of the authenticated image format is generated based on the complete content, i.e., both the processed image data and a representation of the delta, allowing the output generated in the authenticated image format to be authenticated. That is, the joint signature can ensure that the authentication is based on the entire content of the generated output. The authenticated image format allows a representation of the delta between the decoded frame of the processed image data and the raw image data to be stored as part of the generated output, instead of storing the entire raw image data, thereby reducing file size. Thus, compared to conventional image authentication formats, the authenticated image format reduces costs associated with storing and transmitting the output generated in the authenticated image format.

[0025] After receiving a request to generate raw image data from the input data in the authenticated image format, the image rendering computer can reconstruct the raw image data from a representation of the delta between the decoded frames of the processed image data and the raw image data. The image rendering computer can present the processed image data and the raw image via a user interface so that a user can compare the reconstructed raw image data with the processed image data and the user can determine whether the input data in the authenticated image format is authentic, e.g., whether the processed image data is an authentic representation of the raw image data. In some implementations, the image rendering computer or another computer can receive the processed image data and the raw image as input and can use, e.g., a machine learning algorithm to generate a likelihood score indicative of whether the processed image data is an authentic representation of the raw image data.

[0026] Furthermore, raw image data obtained directly from the sensor may not be the best data for image authentication in some use cases. In many use cases, post-processing on the raw image data may be useful to improve the image quality of the raw image data without compromising the authenticity of the image. For example, image enhancements such as noise removal, motion compensation, etc. may help improve image quality. In some cases, multiple raw images may be used to create one processed image data, such as high dynamic range (HDR) imaging, low light imaging, etc. Thus, the final post-processed image may be associated with multiple raw images instead of a single raw image. These post-processed images, even though they are different from the raw image data, may still be considered as authentic images. The authenticated image format may enable authentication of a single raw image or a processed image generated from a series of multiple raw images.

[0027] Additionally, the authenticated image format may include a digital certificate that includes a public key of a public-private key pair of a device that generated the input data in the authenticated image format. The image rendering computer may verify whether the device that generated the input data is trustworthy by verifying the public key of the digital certificate, for example, by verifying a signature on the certificate with a Certificate Authority (CA) public key. In some implementations, the image rendering computer may retrieve a device public key stored on a trusted server and may use the retrieved device public key to verify the digital signature. This ensures that an untrusted device cannot generate an unauthenticated public-private key pair and claim that an unauthenticated public-private key pair corresponds to another trusted device.

[0028] The details of one or more embodiments of the subject matter herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 illustrates an example system for generating output in an authenticated image format. [Diagram 2] FIG. 1 is a diagram of an exemplary computing device. [Diagram 3] FIG. 1 illustrates an example system for rendering an image in an authenticated image format. [Figure 4] 1 is a flowchart of an exemplary process for generating output in an authenticated image format. [Diagram 5] 1 is a flow chart of an exemplary process for reconstructing raw image data from input data in an authenticated image format. [Figure 6A] FIG. 1 illustrates an exemplary system for generating output in an authenticated image format and reconstructing a previous version of an image from the output. [Figure 6B]FIG. 1 illustrates an exemplary system for generating output in an authenticated image format and reconstructing a previous version of an image from the output. [Figure 7] FIG. 13 illustrates an exemplary authenticated image format of the output. [Figure 8] 1 is a flowchart of an exemplary process for generating output in an authenticated image format. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Like reference symbols and designations in the various drawings indicate like elements. FIG. 1 is a diagram of an example system 100 for generating output in an authenticated image format.

[0031] The system 100 includes a sensor 102 , an image processing pipeline 106 , a decoder 120 , a differential encoder 124 , and a certified image format generator 128 .

[0032] The sensor 102 can capture the raw image data 104. The sensor 102 can be a camera of the system 100, or any other type of imaging sensor. The raw image data 104 can include a raw image, or multiple raw images in a series of frames captured within a period of time. The raw image data 104 can include color images or grayscale images. The raw image data 104 is not processed by any image post-processing methods.

[0033] The system 100 can perform post-processing on the raw image data 104 using an image processing pipeline 106. The image processing pipeline 106 can include a series of image processing steps to generate processed image data 108 with improved image quality and / or reduced file size. Each image processing step can use one or more image processing algorithms. In some implementations, the image processing pipeline 106 can include steps that can generate enhanced images that improve the appearance of the raw image data, such as noise removal, motion compensation, contrast enhancement, etc. In some implementations, the image processing pipeline 106 can include steps to generate an enhanced image from multiple raw images in multiple frames, such as HDR / HDR+ photography, low light photography, etc. In some implementations, the image processing pipeline 106 can include steps to arrange multiple raw images in multiple frames and combine the multiple raw images into a single image, which can be further enhanced and compressed by other steps of the image processing pipeline 106.

[0034] In some implementations, the image processing pipeline 106 may include an encoding step that compresses the raw image data 104 or the enhanced image to generate processed image data 108 in a compressed image format, such as JPEG, PNG, TIFF, etc. The processed image data 108 may include a processed image or other format of processed data generated from the raw image data 104. The processed image data 108 in a compressed image format may have a smaller file size than the raw image of the raw image data 104.

[0035] Some image processing pipelines 106 can generate processed image data 108 that is a true representation of the raw image data 104. For example, the image processing pipelines 106 can generate the processed image data 108 using one or more of low light enhancement, HDR photography, noise reduction, deblurring, etc., and the processed image data 108 can still be considered a true representation of the raw image data 104.

[0036] However, the image processing pipeline 106 may generate processed image data 108 that is not considered to be an authentic representation of the raw image. The image processing pipeline 106 may, for example, change the shape, size, position, or texture of objects in the image, and / or add or remove objects in the image. For example, a fake image may be generated by a mobile application that excessively changes the content of an image, such as a fake image of a car with damaged parts of the car altered to submit to an insurance company. As another example, an image may be edited by computer software such that the processed image includes one or more objects that do not belong in the raw image, such as a fake image of a check to a bank with an altered bank account number.

[0037] The decoder 120 can decode the processed image data 108 and generate a decoded frame 122. The decoder 120 can be deterministic and can be specified. The same decoder can later be used in an image rendering computer to reconstruct a raw image from the authentication image. The decoded frame 122 includes image data equivalent to the format of the raw image data 104. For example, the decoded frame 122 can be in an uncompressed image format. In some implementations, the decoder 120 can be a JPEG image decoder capable of generating a decoded image from a JPEG encoded image.

[0038] The differential encoder 124 may receive as input the decoded frame 122 of the processed image data and the raw image data 104 and may generate a representation of the delta 126 between the decoded frame 122 of the processed image data 108 and the raw image data 104. The differential encoder 124 may calculate the delta, e.g., a difference image, by subtracting the decoded frame 122 from a raw image of the raw image data 104. The differential encoder 124 may generate the representation of the delta 126 based on a compression algorithm that measures the difference between image frames. For example, the compression algorithm may be similar to a video compression algorithm that measures and compresses the difference between image frames of a video, such as an MPEG-2 algorithm, an MPEG-4 part 10 / H.264, or an H.265 algorithm. The video compression algorithm may be adapted to measure and compress the difference between the decoded frame 122 and the raw image data 104.

[0039] In some implementations, when the raw image data 104 includes more than one raw image frame, the differential encoder 124 can compare the decoded frame with a raw image selected from the multiple raw image frames. In some implementations, the differential encoder 124 can select a raw image that corresponds to the same or closest point in time as the decoded frame 122.

[0040] The authenticated image format generator 128 can combine the processed image data 108 and the representation of the decoded frames and deltas 126 of the raw image data into a single output image file 130 in an authenticated image format. Unlike conventional image authentication methods that store the raw image data 104 in the output image file, the system 100 can include a much smaller compressed representation of the deltas 126 of the decoded frames 122 of the processed image data 108 and the raw image data 104 in the output image file 130. Thus, the authenticated image format output image file 130 can have a smaller size than image data generated using conventional image authentication methods.

[0041] In response to the request, the image rendering device may reconstruct the raw image data from the output image file 130 received by the image rendering device. The reconstructed raw image data may be compared to the processed image data 108 included in the output image file 130 to determine whether the processed image data 108 is an authentic representation of the reconstructed raw image data. Reconstructing and rendering images in an authenticated image format is described in more detail below in conjunction with FIG.

[0042] In some implementations, the output image file may include a digital signature based on both the processed image data 108 and a representation of the delta 126 between the decoded frame 122 and the raw image data 104. The joint signature may be based on the complete content of the authenticated image format, i.e., on both the processed image data 108 and the representation of the delta 126, and may ensure the authenticity of the output image file 130. For example, a digital signature of a conventional authenticated image format may be applied to a digital signature of a portion of the conventional authenticated image format without unauthorized modification being identified, since the digital signature of the conventional authenticated image format is based only on either the raw image data or the processed image data. In contrast, a digital signature of a conventional authenticated image format is generated based on the complete content of the authenticated image format, and therefore unauthorized modification cannot be applied to the digital signature of a portion of the authenticated image format. That is, the joint signature may ensure that the entire content of the generated output 130 is authentic.

[0043] 2 is a diagram of an exemplary computing device 200 capable of generating a camera image in an authenticated image format. The computing device 200 includes a system-on-chip (SoC) 204 capable of generating an authenticated image file, e.g., an output image file 130 in the authenticated image format. That is, generating the authenticated image file can be performed on the device via the SoC 204. FIG. 2 and its description use a SoC as an exemplary device. However, the computing device 200 can be any general-purpose processing device.

[0044] The SoC 204 includes a camera control 208 that interacts with the camera sensor 202 of the computing device 200. The camera control 208 is connected to a central processing unit (CPU) 222 of the SoC via an SoC bus 234. The camera control 208 can communicate with a camera driver 216 on the CPU 222. In some implementations, the SoC 204 can include camera firmware 206 that can communicate with the camera driver 216 via the camera control 208.

[0045] The SoC 204 includes an ML accelerator 212. The ML accelerator 212 is connected to the CPU 222 via the SoC bus 234. The ML driver 218 can perform post-processing on images captured by the camera sensor 202. The ML driver 218 can communicate with the ML accelerator 212 during the process of image post-processing to expedite one or more steps of the post-processing pipeline 106. In some implementations, the SoC 204 can include ML accelerator firmware 210 that can communicate with the ML driver 218 via the ML accelerator 212.

[0046] The SoC 204 includes an encryption engine 214, which can be a software or hardware module capable of generating a digital signature for a hash of subject data. An encryption driver (crypto driver) 220 can communicate with the encryption engine 214 via the SoC bus 234. The crypto driver 220 can control the encryption engine 214 and generate a digital signature of the complete contents of the data in the authenticated image format, i.e., the processed image data 108 and a representation of the delta 126 between the decoded frame 122 of the processed image data 108 and the raw image data 104.

[0047] The encryption engine 214 can access the hardware key 228 via the SoC bus 234. The hardware key 228 can include a public-private key pair associated with the computing device 200. The public-private key pair can indicate whether the computing device 200 can be trusted. The encryption engine 214 can generate a digital signature of the representation of the processed image data 108 and the delta 126 using the private key of the hardware key 228 of the computing device 200. For example, the encryption engine 214 can generate the digital signature by encrypting a hash value of the representation of the processed image data 108 and the delta 126 using the private key of the hardware key 228.

[0048] The SoC 204 includes a memory control 224 that controls the memory device 230 of the computing device 200. The memory control 224 is connected to the CPU 222 via a SoC bus 234. The memory device 230 can include both volatile and non-volatile memory, such as random access memory (RAM) and flash RAM. For example, after the camera sensor 202 captures the raw image data 104, the raw image data 104 can be stored in a shared buffer in the memory device 230. The CPU can later access the raw image data 104 stored in the memory device 230 and perform post-processing on the raw image data 104 using the ML accelerator 212. The processed image data 108 and a representation of the delta 126 can also be stored in a shared buffer in the memory device 230 controlled by the memory control 224.

[0049] The SoC 204 includes a storage control 226 that controls a storage device 232 of the computing device 200. The storage device 232 may include, for example, one or more of a hard disk device, an optical disk device, a solid-state memory device, etc. The storage control 226 is connected to the CPU 222 via a SoC bus 234. After the SoC 204 generates the output image file 130 in the certified image format, the output image file 130 may be saved to the storage device 232 controlled by the storage control 226.

[0050] An exemplary process for generating an authentication image file on the computing device 200 is as follows.

[0051] A camera application on the computing device 200 can trigger the camera sensor 202 to capture an image. The camera firmware 206 can communicate with a camera driver 216 on the CPU 222 to store the raw image data 104 in a shared buffer in the memory device 230.

[0052] The camera application can invoke an image post-processing component of the SoC 204 to perform post-processing on the raw image data to generate processed image data, e.g., generate a processed image using HDR+ post-processing. For example, the camera application can invoke the ML driver 218 on the CPU 222 to perform post-processing on the raw image data 104 to generate processed image data 108. The CPU 222 can compare the processed image data 108 and the raw image data 104 to generate a difference, i.e., a delta, between a decoded frame 122 of the processed image data 108 and the raw image data 104. In some implementations, the ML driver 218, in conjunction with the ML accelerator firmware 210, can execute an image post-processing pipeline to generate a delta between the two images.

[0053] The CPU 222 may perform differential encoding using the differential encoder 209 to generate a compressed delta of the difference between the raw image data and the decoded frame of post-processed image data. For example, the differential encoding may be entropy encoding, Huffman encoding, run-length encoding, and / or any other suitable technique. After generating the compressed delta, the CPU 222 may apply the compressed delta to the processed image data.

[0054] The camera application can call the encryption driver 220 to control the encryption engine 214 to generate a digital signature of the processed image data 108 and a representation of the delta 126 between the decoded frame 122 of the processed image data 108 and the raw image data 104. In some implementations, the encryption engine 214 can have access to a hardware key 228 of the computing device and can generate the digital signature using the private key of the hardware key 228.

[0055] The camera application can generate an output image file 130 in an authenticated image format. The output image file 130 can include a representation of the processed image data 108 and the deltas 126. The output image file 130 can further include a digital signature generated based on the representation of the processed image data 108 and the deltas 126. The camera application can save the output image file 130 to a storage device 232.

[0056] 3 is a diagram of an example system 300 for rendering an image in an authenticated image format. The process of rendering an image in an authenticated image format may be implemented in the operating system of the image rendering computer, may be implemented as part of a stand-alone image viewing or editing application of the computing device, or may be implemented as part of web browser software, or the like.

[0057] The image rendering computer 304 can receive the input data 302 in an authenticated image format. The image rendering computer 304 can be a different computer than the computing device that generated the input data 302. As described in Figure 1, the input data 302 in the authenticated image format can be the output image file 130 generated by the on-device image authentication process. The input data 302 can include processed image data 108 and a representation of the delta 126 between a decoded frame of the processed image data and the raw image data.

[0058] The image-rendering computer 304 may receive a request 306 from, for example, a user 316 of the image-rendering computer 304 to view a raw image contained in the input data 302 .

[0059] For example, a vehicle owner may use a mobile device to capture an image of a damaged vehicle. Because the image is captured at night, the mobile device may perform low light correction and JPEG encoding, and may generate a processed encoded image of the raw image of the damaged vehicle. The mobile device may generate an image file in an authenticated image format that includes the low light corrected processed encoded image of the damaged vehicle and a representation of the delta between the processed image data and the raw image captured by the camera. The vehicle owner may upload the image file in the authenticated image format to an insurance company's computer during the process of claiming compensation for damage to the vehicle. The insurance company's computer, i.e., the image rendering computer 304, may receive the uploaded image file in the authenticated image format. An insurance agent 316 may send a request to the computer to view the raw image included in the uploaded image file in the authenticated image format. For example, the insurance agent 316 may want to visually compare the raw image and the processed image to determine whether the post-processed image is an authentic representation of the damaged vehicle. In some implementations, the insurance agent 316 may send a request to the computer and instruct the computer to verify a digital signature included in the uploaded image file.

[0060] Upon receiving a request to view raw image 306, image rendering computer 304 can generate processed image 308 and raw image 310 in an authenticated image format from input data 302. Image rendering computer 304 can generate processed image 308 by reading the processed image included in input data 302. Image rendering computer 304 can generate a decoded image of processed image 308 by decoding the processed image using, for example, the same decoder as decoder 120 during generation of input data 302. Image rendering computer 304 can generate raw image 310 by combining the decoded image with a representation of delta 126 included in input data 302.

[0061] For example, the insurance company's computer can generate a processed image of the damaged vehicle, e.g., a JPEG encoded image of the damaged vehicle, using a JPEG decoder to decode the JPEG image. The computer can generate a raw image corresponding to the processed image by combining a representation of delta 126 included in input data 302 with a decoded image of processed image 308.

[0062] The image rendering computer 304 can include a display device 314, such as a monitor of a desktop computer, a screen of a mobile device, a VR and / or AR device, etc. In some implementations, as shown in FIG. 3, the image rendering computer 304 can present the processed image 308 and the raw image 310 side-by-side on the display device 314. In some implementations, not currently shown in FIG. 3, the image rendering computer 304 can present a user interface element that allows the user 316 to toggle between the processed image 308 and the raw image 310. In some implementations, not currently shown in FIG. 3, the image rendering computer 304 can present a user interface element that displays a difference between the processed image 308 and the raw image 310, e.g., a difference image. For example, the difference between the two images can be calculated by determining the difference between each pixel of each image and generating an image based on the difference between each pixel of each image.

[0063] In some implementations, the image rendering computer 304 may receive a request 306 to verify a digital signature included in the input data 302. In some implementations, the image rendering computer 304 may verify the digital signature before generating a raw image from the input data 302. If the digital signature verification is successful, the image rendering computer 304 may proceed to reconstruct the raw image from the input data 302 in an authenticated image format, and the image rendering computer 304 may inform the user 316 that the computing device that generated the input data 302 is a trusted device. If the digital signature verification is not successful, the image rendering computer 304 may not be able to proceed to reconstruct the raw image, and the image rendering computer 304 may inform the user 316 that the computing device that generated the input data 302 is not a trusted device.

[0064] In some implementations, the image rendering computer 304 can display the device identity 312 on a display device 314. The device identity 312 can be included in a digital certificate along with the device's public key. The digital certificate can be signed by a private key of a certificate authority (CA) at the device manufacturer, for example via a sub-CA. The device identity 312 can include an identification of the device that generated the input data 302 or a verification result of the digital signature, for example, "device verified" or "device not verified." A user 316 can review the digital signature verification result included in the device identity 312 and determine whether the device that generated the input data 302 can be trusted.

[0065] In some implementations, the image rendering computer 304 may include a general purpose image reader, e.g., a JPEG decoder, that is not specialized to process the input data 302 in an authenticated image format. The image rendering computer 304 may further decode and render the processed image, e.g., render the processed JPEG image as if it were a normal JPEG image. In some implementations, the image rendering computer 304 may discard, e.g., a representation of the delta between the decoded frame of the processed image and the raw image, and discard the remainder of the data in the input data 302.

[0066] In some implementations, the image rendering computer 304 can receive input data that is not in an authenticated image format, for example, a post-processed image in JPEG format. The image rendering computer 304 can still generate the post-processed image, for example by performing a JPEG decoding process, and display the post-processed image on the display device 314 without generating the raw image 310. Thus, the image rendering computer 304 is backward compatible to display input data that is not in an authenticated image format.

[0067] Figure 4 is a flow chart of an exemplary process for generating output in an authenticated image format. For convenience, the process is described as being performed by a system for generating an output image in an authenticated image format, such as system 100 of Figure 1. The system may include components of the computing device 200 described with reference to Figure 2, including one or more camera sensors, one or more SoCs, one or more memory devices, and one or more storage devices, or any combination thereof.

[0068] The system captures raw image data with an image sensor (402). In some implementations, the raw image data can include a single image of a single raw frame captured by the camera sensor. In some implementations, the raw image data can include multiple images of a series of raw frames captured by the camera sensor. The system can perform a registration and merging process to generate a single image as the raw image representative of what is being captured by the camera.

[0069] The system generates 404 processed image data from new image data captured by the image sensor by a processing pipeline having one or more processing elements. The processing pipeline can include one or more image processing routines or algorithms capable of generating processed image data with enhanced image quality. The processed image data can include a processed image or other format of processed data generated from the raw image data. In some implementations, the processed image can be in a compressed image format, such as JPEG, PNG, TIFF, or another compressed image format.

[0070] The system generates 406 decoded frames from the processed image data generated from the processing pipeline. In some implementations, the decoded frames can be in an uncompressed image format. For example, when the processed images are in JPEG format, the system can generate the decoded frames from the processed images using a JPEG image decoder.

[0071] The system generates 408 output in an authenticated image format. The output includes the processed image data and a representation of the delta between a decoded frame of the processed image data and the raw image data. That is, instead of storing the raw image data, the system generates the difference between the raw image data and the decoded frame of the processed image data. The system then encodes the output image difference in the authenticated image format so that an image reader can later properly reconstruct the raw image data upon request.

[0072] The system can reliably generate and encode raw image frames regardless of the number and type of processing steps in the processing pipeline used to generate the processed image data. An image reader configured to process an output file in the certified image format can then reconstruct and display the raw image frames.

[0073] In some implementations, the representation of the delta between the decoded frame of processed image data and the raw image data can be generated based on a compression algorithm that measures the difference between image frames. The compression algorithm is efficient at handling differences between image frames, e.g., differences that have many zeros. When two image frames are sufficiently similar to one another, the difference between two image frames can be efficiently compressed. For example, the system can generate the output using a video encoding algorithm that can encode the difference between frames in a video, such as a video MPEG encoder.

[0074] In some implementations, the output may include a digital signature based on both the processed image data and a representation of the delta between the decoded frame of the processed image data and the raw image data, i.e., the system may sign and authenticate both the processed encoded frames as well as the difference information, i.e., the delta between the decoded frame of the processed image data and the raw image data, in the same output file.

[0075] In some implementations, the system can generate a hash value based on the representation of the processed image data and the delta between the decoded frame of the processed image data and the raw image data, and the system can generate a digital signature based on the hash value. A hash value is a fixed-length number that uniquely identifies data. A hash value can represent large amounts of data as a much smaller number.

[0076] In some implementations, the system may generate a digital signature by encrypting the hash value using the private key of the device's public-private key pair. The system may access a hardware key associated with the system's computing device, such as hardware key 228 of communication device 200 of FIG. 2. The public-private key pair may indicate whether the computing device is trustworthy. The image rendering computer may then obtain the public key of the private key pair associated with the device and verify the digital signature of the output file in the authenticated image format to determine whether the device that generated the output file is trustworthy.

[0077] In some implementations, the output may include a digital certificate that includes the public key of the device's public-private key pair. In some implementations, the output may further include a device identifier, an expiration date, and a digital signature generated by the CA or sub-CA. The image rendering computer may then use the public key to verify the digital certificate in the certified image format output file to determine whether the device that generated the output file is trustworthy.

[0078] 5 is a flow chart of an exemplary process for reconstructing raw image data from input data in an authenticated image format. For convenience, the process is described as being performed by a system for image rendering, such as system 300 of FIG. 3. The system may include an image rendering computer or device that may be configured to analyze input data in the authenticated image format and reconstruct raw image data from the input data in the authenticated image format.

[0079] The system receives input data in an authenticated image format (502). The input data includes processed image data and a representation of a delta between a decoded frame of the processed image data and the raw image data. The processed image data may include the processed image or other format of processed data generated from the raw image data. For example, the input data may include a post-processed image in JPEG format and a delta between a decoded frame of the post-processed image and a raw image captured by a camera sensor.

[0080] The system receives a request to generate raw image data from the input data 504. For example, the system may receive a request 306 from a user of the image rendering system to display the raw image 310 and the processed image 308 so that the user can verify the authenticity of the input data 302.

[0081] The system reconstructs the raw image data from the representation of the delta between the decoded frames of the processed image data and the raw image data (506). In some implementations, reconstructing the raw image data from the representation of the delta between the decoded frames of the processed image data and the raw image data may include generating the raw image data by combining the processed image data with the representation of the delta between the decoded frames of the processed image data and the raw image data.

[0082] In some implementations, the input data in the authenticated image format can include a digital signature generated based on the complete content, i.e., both the processed image data and a representation of the delta between the decoded frame of the processed image data and the raw image data. The digital signature generated on the complete content can ensure the authenticity of the input data. That is, the digital signature can ensure that the representation of the processed image data and the delta cannot be modified after the digital signature is calculated.

[0083] In some implementations, the digital signature included in the input data can be generated by (i) generating a hash value based on a representation of the processed image data and a delta between a decoded frame of the processed image data and the raw image data, and (ii) encrypting the hash value using a private key of a public-private key pair associated with a device that generates the input data in the authenticated image format. In some implementations, the public-private key pair can be a hardware key associated with the device that generated the input data, such as hardware key 228 of computing device 200. The digital signature can indicate whether the input data was generated by a trusted device that is authenticated to have access to the private key. In some implementations, the system can obtain a public key of a public-private key pair associated with a device that generates the input in the authenticated image format, and the system can verify the digital signature of the input data using the public key.

[0084] In some implementations, the input data in the authenticated image format can include a digital certificate that can be used to verify whether the input data was generated by a trusted device. The system can verify the digital certificate of the input data before verifying the digital signature of the input data in the authenticated image format, e.g., verify that the input data was generated by a trusted device. Any device can provide an authenticated image with a valid signature, but only a legitimate device can certify its device's public key, e.g., by including the device's public key in a valid digital certificate. Invalid device manufacturers cannot have their public keys certified, e.g., by a CA or a secure server.

[0085] Responsive to successful verification of the digital certificate, the system may extract the device's public key from the digital certificate. The system may then proceed to verify the validity of the input data in the authenticated image format by verifying the digital signature using the device's public key. If the input data is valid, the system may reconstruct raw image data from the input data in the authenticated image format and display the reconstructed raw image to a user so that the user can determine whether the processed image accurately represents the reconstructed raw image. In some implementations, the system may provide the reconstructed raw image and the processed image to a computer-implemented algorithm, e.g., a machine learning algorithm, which may generate a matching score between the reconstructed raw image and the processed image, where the matching score may represent the likelihood that the processed image accurately represents the reconstructed raw image.

[0086] In some implementations, the device's public key can be provided in a digital certificate. The digital certificate can be certified by a Certificate Authority (CA) and can be included in the input data in the authenticated image format. The digital certificate can be generated external to the device and can be provisioned into the device at the time of manufacture. The device's public key associated with the device's private key can be certified by the CA. The system can verify the digital certificate included in the input data with the CA root's public key to ensure the authenticity of the device that generated the input data in the authenticated image format. For example, the system can obtain the digital certificate by accessing metadata included in the input data, and the system can verify the digital certificate with the CA root's public key.

[0087] In some implementations, the device's public key can be stored in a trusted server or trusted database accessible to the system, and the system can retrieve the device's public key stored in the trusted server and use the device's public key to verify the digital signature of the input data. For example, the system can retrieve the device's public key stored in the trusted server based on an identification (ID) provided in the metadata of the input data. The system can verify the digital signature using the device's public key retrieved from the trusted server.

[0088] In some implementations, the system may present the processed image data and the reconstructed raw image data side-by-side on a display device of the system. In some implementations, the system may present a user interface element that toggles between the processed image data and the reconstructed raw image data. For example, a user of the system may use a user interface element to toggle between the processed image and the reconstructed raw image.

[0089] In some implementations, the system or another system can receive the processed image data and the reconstructed raw image data as inputs and generate a prediction result indicative of whether the processed image data is an authentic representation of the reconstructed raw image data. In some implementations, the system or another system can use a trained machine learning model to generate a prediction result indicative of the likelihood that the processed image data is an authentic representation of the reconstructed raw image data.

[0090] 6A and 6B are diagrams of an example system 600 for generating output in an authenticated image format and reconstructing a previous version of an image from the output.

[0091] 6A is a diagram of an example system 600 for generating output in an authenticated image format. System 600 can be system 100 described above with respect to FIG 1. System 600 can include computing device 200 described above with respect to FIG 2 and can include image rendering computer 304 with display 314 described above with respect to FIG 3.

[0092] The computing device 200 can be a computing device such as a server, a desktop or laptop computer, a smartphone, a tablet computing device, a PDA, etc.

[0093] The image rendering computer 304 can be a computing device such as a server, a desktop or laptop computer, a smartphone, a tablet computing device, a PDA, etc. The display 314 can be an integral display such as an LED, OLED, LCD, or other display type screen of a laptop computer, smartphone, tablet computing device, PDA, etc. The display 314 can alternatively be an external display such as an LED, OLED, LCD, or other display type monitor electronically connected to the computing device.

[0094] The computing device 200 may communicate with the image-rendering computer 304 over a network, such as a wireless network. For example, the computing device 200 may communicate with the image-rendering computer 304 over a cellular network or the Internet.

[0095] The computing device 200 may generate the raw image data 104. For example, the computing device 200 may generate the raw image data 104 using a camera sensor 202 as described above with respect to Figure 2. Alternatively, the computing device 200 may receive the raw image data 104 from a different device over a network.

[0096] The computing device 200 may generate processed image data 108 from the raw image data 104 as described above with respect to Figure 1. For example, the computing device 200 may convert the raw image data 104 into an image in a compressed image format having a smaller file size (e.g., a JPEG image, a PNG image, etc.) using the image processing pipeline described above with respect to Figure 1.

[0097] Alternatively, the computing device 200 can receive the processed image data 108 from a different device over a network. For example, the computing device 200 can obtain both the raw image data 104 and the processed image data 108 from a smartphone that uses a camera sensor to generate the raw image data 104 and the processed image data 108 from the raw image data 104.

[0098] The raw image data 104 can be in a Bayer format. The raw image data 104 can be a Bayer RGGB image in which a group of four pixels includes a top left pixel that is a red (R) color pixel, a top right pixel that is a green (G) color pixel, a bottom left pixel that is also a green (G) color pixel, and a bottom right pixel that is a blue (B) color pixel. However, the raw image data 104 can be in different formats, such as Bayer BGGR, Bayer RGBG, Bayer GRBG, or other formats with different colors and / or arrangements depending on the camera sensor used to generate the raw image data.

[0099] As an example, the raw image data 104 may include a grouping of four pixels. The group may include a red pixel (Raw Pixel 1) having a color intensity value of 180 (e.g., an 8-bit intensity value), a first green pixel (Raw Pixel 2) having a color intensity value of 89, a second green pixel (Raw Pixel 3) having a color intensity of 78, and a blue pixel (Raw Pixel 4) having a color intensity of 40. Although a simplified four pixel example is shown and described, it should be understood that the raw image data 104 or the portion of the raw image data 104 used to generate the output image file 130 may include any number of pixels.

[0100] Although the example shown in FIG. 6A illustrates the use of 8-bit color (e.g., 8-bit RGB image size or depth) in both the raw image data 104 and the decoded frame 122, various other scenarios are possible. For example, the raw image data 104 may have a color depth of 10 bits, 12 bits, 14 bits, or 16 bits. Typically, the decoded frame 122 has an 8-bit RGB image size because the processed image data 108 is typically a JPEG or PNG image with an 8-bit color depth. However, the processed image data 108 and the decoded frame may have a different color depth, such as 10 bits or 12 bits. As described in more detail below, when the color depth of the raw image data 104 is different from the color depth of the decoded frame 122, the differential encoder 124 may convert the pixel color intensity values ​​of the less significant bits of the decoded frame 122 to match the bit size of the raw image data 104 to generate the delta 126.

[0101] The raw image data 104 may represent all of the image data acquired from the camera sensor or a portion of the image data acquired from the camera sensor. For example, the raw image data 104 may be a selected frame from a video captured by the camera sensor.

[0102] The processed image data 108 may also be generated from all image data acquired from the camera sensor, or a portion of the image data acquired from the camera sensor. For example, if the raw image data 104 is a single frame from a video data object, the processed image data 108 may be generated by the computing device 200 using the same frame from the video data object. As another example, if the raw image data 104 includes raw image data for an entire video data object such that it includes multiple image frames, the processed image data 108 may similarly include multiple image frames generated by the computing device 200 using multiple frames of the raw image data 104.

[0103] The decoded frames 122 generated by the decoder 120 may represent all or a portion of the processed image data 108. As an example, if the processed image data 108 includes multiple image frames, the decoder 120 may select a particular frame from the processed image data 108 to use to generate the decoded frame 122. The decoder 120 may continually generate decoded frames from the processed image data 108, for example, in response to generating a decoded frame, the differential encoder 124 obtaining a previously generated decoded frame, processing resources becoming available, etc. The decoder 120 may continue to generate decoded frames until all frames of the processed image data 108 have been converted into decoded frames. The computing device 200 may then provide (i) the decoded frames 122 representing a portion of the processed image data 108, and (ii) the raw image data 104 or a corresponding portion of the raw image data 104 as input to the differential encoder 124.

[0104] The decoded frame 122 may represent all of the processed image data 108. For example, the decoded frame 122 can represent all of the processed image data 108 when the processed image data 108 comprises a single image frame. Alternatively, the decoded frame 122 may include multiple image frames generated by the decoder 120 from multiple frames of the processed image data 108. The computing device 200 can then provide (i) the decoded frame 122 representing all of the processed image data 108, and (ii) the raw image data 104 or a corresponding portion of the raw image data 104, as inputs to the differential encoder 124.

[0105] The decoded frame 122 may be a red-green-blue (RGB) reconstructed image. For example, the decoder 120 may convert the processed image data 108 into an RGB image, with each pixel of the resulting decoded frame 122 having a red channel, a green channel, and a blue channel. Each channel of each pixel of the decoded frame 122 may have its own color or light intensity value. For example, pixel 1 of the decoded frame 122 may include a red channel having a color intensity value of 57 out of 255 (e.g., 8-bit color depth), a green channel having a color intensity value of 205 out of 255, and a blue channel having a color intensity value of 242 out of 255.

[0106] The computing device 200 may provide the decoded frame 122 and the raw image data 104 or a portion of the raw image data 104 corresponding to the decoded frame 122 as inputs to a differential encoder 124. The differential encoder may include a differencing module 602 configured to generate a delta 126 from the decoded frame 122 and the raw image data 104 by calculating a difference between a selected channel intensity value 604 of the decoded frame 122 and a corresponding intensity value of the raw image data 104. The differencing module 602 may be a software module that includes one or more models for generating the delta 126. These one or more models may include static or machine learning models.

[0107] The differencing module 602 may select a color channel for pixels of the decoded frame 122. For example, the differencing module 602 may select a particular color channel for each pixel of the decoded frame 122.

[0108] The difference module 602 can use the raw image data 104 to determine which color channel to select for different pixels of the decoded frame 122. For example, the difference module 602 can identify a corresponding pixel in the raw image data 104 for each pixel of the decoded frame 122 and determine the color of the pixel in the raw image data 104 (e.g., a Bayer color filter). The difference module 602 can then select a color channel for that particular pixel of the decoded frame 122 that matches the color of the corresponding pixel in the raw image data 104. For example, based on pixel 1 of the decoded frame 122 corresponding to raw pixel 1 of the raw image data 104 having a red color, the difference module 602 can select a red channel for pixel 1. The difference module 602 can similarly select a green channel for pixel 2 of the decoded frame 122, a green channel for pixel 3 of the decoded frame 122, and a blue channel for pixel 4 of the decoded frame 122 based on the color of the corresponding pixel in the raw image data 104.

[0109] Using the techniques described above, the differencing module 602 may obtain selected channel intensity values ​​604 from the decoded frame 122. For example, the selected channel values ​​may include a red intensity value of 57 out of 255 for pixel 1 of the decoded frame 122, a green intensity value of 255 out of 255 for pixel 2 of the decoded frame 122, a green intensity value of 255 out of 255 for pixel 3 of the decoded frame 122, and a blue intensity value of 40 out of 255 for pixel 4 of the decoded frame 122.

[0110] After obtaining the selected channel intensity values ​​604, the difference module 602 may calculate delta 126 by taking the difference between the color intensity values ​​of the raw image data 104 (or a portion of the raw image data 104) and the corresponding values ​​of the selected channel intensity values ​​604. For example, the difference module 602 may subtract red intensity value 57 of pixel 1 from red intensity value 180 of raw pixel 1 of raw image data 104 to calculate color intensity value 123 of delta 126. The difference module 602 may similarly calculate other intensity values ​​of delta 126.

[0111] In some implementations, when the color depth of the raw image data 104 does not match the color depth of the decoded frame 122, the difference module 602 changes the color depth of the decoded frame 122 or the selected channel intensity values ​​604 to match the depth of the raw image data 104. For example, if the raw image data 104 uses a 12-bit color depth, the difference module 602 may convert the selected channel intensity values ​​604 from 8 bits to 12 bits by scaling the decoded frame 122 to a 12-bit size or by multiplying the channel intensity values ​​604 by 16. The difference module 602 may then take the difference between the converted channel values ​​and the corresponding 12-bit values ​​of the raw image data 104 to generate the delta 126.

[0112] Delta 126 can be an image, such as an RGB image or a grayscale image. For example, the first pixel of delta 126 can be a pixel having a red channel with intensity value 123, a green channel with intensity value 0, and a blue channel with intensity value 0. Alternatively, the first pixel of delta 126 can be a grayscale pixel with intensity value 123.

[0113] The difference module 602 can generate metadata for the delta 126. For example, the difference module 602 can generate metadata that indicates whether the light or color intensity values ​​of the pixels of the delta 126 should be applied as positive and / or negative values ​​during raw image reconstruction, which is described in more detail below with respect to FIG. 6B. For example, the metadata can identify all pixels whose intensity values ​​should be applied as negative values.

[0114] Difference module 602 may instead indicate whether a pixel of delta 126 contains a positive and / or negative value by adding a bit to each pixel value. For example, difference module 602 may generate delta 126 to have a size or depth of 9 bits instead of a size or depth of 8 bits, with the added bit indicating the sign of the pixel intensity value. For example, a ninth bit having a value of "0" may indicate that the pixel contains a positive value, and a ninth bit having a value of "1" may indicate that the pixel contains a negative value.

[0115] After generating the deltas 126, the differencing module 602 may provide the deltas 126 to an encoding and compression engine 610 configured to encode and compress the deltas 126. As an example, the encoding and compression engine 610 may divide the deltas 126 into macroblocks of a particular size (e.g., 8x8 pixels, 16x16 pixels, 32x32 pixels, etc.). In some implementations, the engine 610 may use half-pixel and / or quarter-pixel matching during the division.

[0116] After dividing out the delta 126, in some implementations, the engine 610 may apply a discrete cosine transform (DCT), an inverse DCT (IDCT), or another type of DCT.

[0117] After splitting the delta 126, in some implementations, the engine 610 may apply entropy coding. For example, after splitting the delta 126 and applying the DCT, the engine 610 may apply Huffman entropy coding to generate the encoded and / or compressed delta 612.

[0118] After splitting the delta 126, in some implementations, the engine 610 can apply run-length encoding. For example, after splitting the delta 126, applying the DCT, and applying Huffman entropy encoding, the engine 610 can apply run-length encoding to generate the encoded and / or compressed delta 612.

[0119] After splitting the delta 126, the engine 610 may apply one or more compression steps. For example, after splitting the delta 126, applying a DCT, applying Huffman entropy coding, and applying run-length coding, the engine 610 may apply one or more compression steps to generate an encoded and / or compressed delta 612.

[0120] The engine 610 of the differential encoder 124 provides the encoded and / or compressed delta 612 as an input to the image format generator 128. As described in more detail below with respect to FIGURE 7, in response to receiving the encoded and / or compressed delta 612, the image format generator 128 uses the encoded and / or compressed delta 612 and the processed image data 108 to generate an output image file 130. The output image file 130 may include the encoded and / or compressed delta 612 and the processed image data 108, such as an image in a compressed image format (e.g., JPEG format).

[0121] For example, by encoding and / or compressing the delta 126 instead of the raw image data 104, more efficient compression may be achieved and the size of the output image file 130 may be relatively small. For example, the output image file may desirably include processed image data 108 that may be viewed by a typical image viewer, as described in more detail below with respect to FIG. 7. However, an output image file that includes both the processed image data 108 (e.g., JPEG) and the raw image data 104 may be very large. Instead, by calculating the delta 126 and including it in the output image file 130 after encoding and / or compression instead of the raw image data 104, the size of the output image file 130 may be significantly reduced while still having the ability to reconstruct the raw image data 104, as described in more detail below with respect to FIG. 6B.

[0122] This is because the resulting encoded and / or compressed delta 612 is typically smaller in size than the raw image data 104 or the encoded and / or compressed raw image data 104. This is at least in part because the difference between pixel intensity values ​​of the raw image data 104 and a selected channel of a decoded frame 1222 (e.g., a reconstructed RGB image) is typically small, allowing for more efficient compression of the delta 126 compared to the raw image data 104.

[0123] 6B is a diagram of a portion of a system 600 for reconstructing an earlier version of an image from output received in an authenticated image format. As described above, the system 600 includes an image rendering computer 304 configured to reconstruct an earlier version of an image from output in the authenticated image format, and a display 314 used to present the reconstructed image data next to the processed image data 108.

[0124] The image rendering computer 304 may use an image rendering module 620 configured to reconstruct a previous version of an image, such as raw image data 104, from the output image file 130. The image rendering module 620 may be a software module including one or more models configured to perform the techniques described below and elsewhere, as discussed above with respect to FIG.

[0125] In response to receiving the output image file 130, the image rendering computer 304 can use an image rendering module 620 to decode and / or decompress the encoded and / or compressed deltas 612 of the output image file 130. For example, by decoding and decompressing the encoded and / or compressed delta 612 data, the image rendering module 620 can obtain the deltas 126.

[0126] In response to receiving the output image file 130, the image rendering computer 304 may use an image rendering module 620 to generate a decoded frame 622 from the processed image data 108 of the output image file 130. For example, the computer 304 may also include a decoder that provides the functionality of the decoder 120 described above and may use the decoder to generate a reconstructed RGB image from the compressed image of the output image file 130. The decoded frame 622 may be the same as the decoded frame 122, as shown.

[0127] In some implementations, decoded frame 622 differs from decoded frame 122. For example, decoded frame 622 may differ from decoded frame 122 based on differences in how computer 304 generates the decoded frame (e.g., the RGB reconstruction methodology used). Ideally, however, computing device 200 and computer 304 use the same method to generate a reconstructed RGB image from, for example, processed image data 108.

[0128] After generating the decoded frame 622, the image rendering module 620 may select a color channel for a pixel of the decoded frame 622 to obtain a selected channel intensity value 624. The process of selecting a color channel for a pixel of the decoded frame 622 to match the color of a corresponding pixel of the delta 126 is described in more detail above with respect to the encoded frame 122 of FIG.

[0129] After obtaining the selected channel intensity value 624, the image rendering module 620 may add the selected channel intensity value 624 to the delta 126 to reconstruct the raw image data 104 or a portion of the raw image data 104. For example, the module 620 may add the red channel intensity value 57 of pixel 1 of the decoded frame 622 to the intensity value 123 of the corresponding pixel of the delta 126 to reconstruct raw pixel 1 of the raw image data 104 having color intensity 180. The module 620 may repeat this process for other pixels of the decoded frame 622 and the delta 126 to reconstruct raw pixel 2, raw pixel 3, and raw pixel 4 of the raw image data 104.

[0130] After reconstructing the raw image data 104 or a portion of the raw image data 104, the image rendering module 620 can perform demosaicing on the raw image data 104 or a portion of the raw image data 104 to generate a full color image 626. The full color image 626 is not mosaiced in that it does not include a Bayer pattern image. Instead, each pixel of the full color image 626 can be a full color pixel, such that the pixel has a red channel value, a green channel value, and a blue channel value. That is, the pixels of the full color image 626 are not limited to a single color channel or filter, unlike, for example, the raw image data 104.

[0131] In generating the full color image 626, the module 620 can apply a demosaicing algorithm. For example, the module 620 can generate the full color image 626 by interpolating missing color channel values ​​at all pixels of the raw image data 104. For example, by interpolation, the module 620 can generate green and blue channel values ​​for raw pixel 1 of the raw image data 104 that includes only a red intensity value. The demosaicing algorithm can be, for example, linear interpolation, bilinear interpolation, bicubic interpolation, a uniform directional demosaicing algorithm, or higher order interpolation, etc.

[0132] After generating the full-color image 626, the computer 304 can present the full-color image 626 on the interface 630 of the display 314. For example, the computer 304 can present the full-color image 626 next to the processed image data 108. By presenting the full-color image 626 next to the processed image data 108, a user of the computer 304 can easily identify differences between the processed image data 108 and the raw image data 104 represented by the full-color image 626. The full-color image 626 can serve as a ground truth frame that is independent of the potential multiple and types of finishing steps performed on the image to generate the processed image data 108. In this manner, a user of the computer 304 can quickly identify edits made to the image by another user or automatically by the device, for example, when applying a filter or enhancing the image (e.g., low light correction).

[0133] As an example, the image rendering computer 304 may first present the processed image data 108 from the output image file 130 on the interface 630. After presenting the processed image data 108, the computer 304 may present a full color image 626 in response to receiving a request from a user to view the raw image data 104.

[0134] For example, a vehicle owner may use the camera of the computing device 200 to capture an image of the damaged vehicle. Because the image is captured at night, the computing device 200 may perform low-light correction and JPEG encoding, and may generate a processed encoded image of the raw image of the damaged vehicle. The computing device 200 may generate an image file in an authenticated image format that includes the low-light corrected processed encoded image of the damaged vehicle and a representation of the delta between the processed image data and the raw image captured by the camera of the computing device 200. The vehicle owner may upload the image file in the authenticated image format to a computer of the insurance company during the process of claiming compensation for damage to the vehicle. The insurance company's computer, e.g., the image rendering computer 304, may receive the uploaded image file in the authenticated image format. A user of the computer 304 (e.g., the insurance agent 316 as described above with respect to FIG. 3) may send a request to the computer 304 to view the raw image included in the uploaded image file in the authenticated image format. For example, the insurance agent 316 may want to visually compare the raw image and the processed image to determine whether the post-processed image is an authentic representation of the damaged vehicle. In some implementations, the insurance agent 316 can send a request to the computer 304 instructing the computer 304 to verify a digital signature included in the uploaded image file.

[0135] In some implementations, computer 304 additionally or alternatively displays delta 126 or a representation of delta 126 on interface 630. For example, computer 304 may perform demosaicing on delta 126 to generate a differential full-color image and then display the differential full-color image on interface 630.

[0136] 7 is a diagram illustrating an exemplary authenticated image format for output image file 130. The authenticated image format may specify that output image file 130 includes a standard header 702 with a new flag, processed image data 108, a difference image header 704 of the encoded and / or compressed deltas 612, the encoded and / or compressed deltas 612, and a digital signature 706 applied to the standard header 702, the processed image data 108, the header 704, and the encoded and / or compressed deltas 612.

[0137] The standard header 702 can be the standard image header of the processed image data 108 with a new flag added to the standard image header to indicate the presence of additional image data (e.g., the encoded and / or compressed delta 612). The flag can indicate that the additional image data appears after the processed image data 108 in the output image file 130.

[0138] The certified image format may specify that the difference image data is added to the output image file after the compressed image data. For example, the certified image format may specify that the encoded and / or compressed delta 612 is added to the output image file 130 after the processed image data 108.

[0139] In some implementations, the header 704 may indicate the encoding used to generate the difference image data. For example, the header 704 may indicate that Huffman entropy encoding was used to generate the encoded and / or compressed delta 612. The computer 304 may identify the type of encoding performed on the difference image data from the header 704 and use this information to decode the encoded and / or compressed delta 612.

[0140] In some implementations, the header 704 may indicate the compression used to generate the difference image data. For example, the header 704 may indicate the type of compression used to generate the encoded and / or compressed delta 612. The computer 304 may identify the type of compression performed on the difference image data from the header 704 and use this information to decompress the encoded and / or compressed delta 612.

[0141] The certified image format may be backward compatible to allow legacy image readers to read and / or present the processed image data in an output image file that is in the certified image format. For example, the standard header 702 and the processed image data 108 may be processed by a standard or legacy viewer for compressed images. More specifically, if the processed image data 108 is a compressed image such as JPEG, the certified image format allows a standard JPEG viewer to ignore the new flag, read the standard header 702, and present the processed image data 108 in a display interface. However, this standard JPEG viewer cannot process, for example, the header 704 and the encoded and / or compressed deltas 612. Thus, the standard JPEG viewer in this example cannot reconstruct the raw image data 104 from the processed image data 108 and the encoded and / or compressed deltas 612.

[0142] The secure or dedicated image viewer may be configured to process the entirety of the output image file 130 in an authenticated image format. For example, the secure image viewer may be installed on the computer 304 described above or may be accessed by the computer 304 via a web browser and may be used by the computer 304 to extract the encoded and / or compressed deltas 612 from the output image file 130. The secure image viewer may also be used for additional functions, such as the functions of module 620 described above.

[0143] As an example, when opening the output image file 130, the secure image viewer can recognize a new flag in the standard header 702 and use it to determine that the output image file 130 contains difference image data (e.g., encoded and / or compressed deltas 612). The secure image viewer can read the header 704 and use the information in the header 704 to obtain the difference image data, present the difference image data, reconstruct raw image data 104 from the difference image data and the processed image data 108, and / or generate a full color image from the reconstructed raw image data.

[0144] The secure image viewer can also perform the functions of a legacy or standard image viewer, as described above. For example, the secure image viewer can read the standard header 702 and use the information in the standard header 702 to present the processed image data 108.

[0145] The authenticated image format may feature joint authentication. For example, hashing and signing may be performed on the entire data bundle including the standard header 702, the processed image data 108, the header 704, and the encoded and / or compressed deltas 612. The resulting digital signature 706 is included in the output image file 130 according to the authenticated image format and may be used by the computer 304 to authenticate the output image file 130.

[0146] 8 is a flow chart of an exemplary process 800 for generating output in an authenticated image format. Process 800 may be performed by system 100 described above. Process 800 may be performed by system 600 described above with respect to Figures 6A-6B. For example, process 800 may be performed by computing device 200 described above with respect to Figures 2 and 6A-6B.

[0147] The process 800 includes acquiring raw image data for an image (802). For example, the computing device 200 can acquire the raw image data 104 from an on-board camera sensor or from an external device over a wireless network. The raw image data 104 can be in a Bayer image format, such as Bayer RGGB.

[0148] The process 800 includes obtaining processed image data for the image (804). For example, the computing device 200 may generate the processed image data 108 from the raw image data 104. Alternatively, the computing device 200 may obtain the processed image data from an external device over a wireless network. The processed image data may be, for example, a JPEG file, a PNG file, or another image file in a compressed and / or encoded image format.

[0149] In some implementations, the processed image data is encoded image data. For example, the processed image data 108 can be generated by encoding the raw image data 104, as shown in FIG. 6A.

[0150] In some implementations, the processed image data is compressed image data. For example, the processed image data 108 can be generated by compressing the raw image data 104, as shown in FIG. 6A.

[0151] In some implementations, the processed image data is encoded and compressed image data. For example, the processed image data 108 can be generated by encoding and compressing the raw image data 104, as shown in FIG. 6A.

[0152] The process 800 includes generating a decoded frame from the processed image data (806). For example, with reference to FIGS. 1 and 6A, the computing device 200 may generate a decoded frame 122 from a processed image, such as the processed image data 108, using the decoder 120. The decoded frame may be a red-green-blue (RGB) reconstructed image, where each pixel of the decoded frame may have three color channels: a red channel having a red intensity value, a green channel having a green intensity value, and a blue channel having a blue intensity value. The decoded frame may be generated from the entire processed image data, such that the decoded frame and the processed image data share the same number of pixels. Alternatively, the decoded frame may be generated from a portion of the processed image data.

[0153] The process 800 includes calculating 808 a difference in pixel values ​​between at least a portion of the raw image data and the decoded frame. For example, with reference to Figures 1 and 6A, the computing device 200 may use difference module 602 of differential encoder 124 to calculate a difference between the decoded frame 122 and at least a portion of the raw image data 104. Because each pixel of the decoded frame may have a corresponding pixel in at least a portion of the raw image data, the number of pixels of the decoded frame may be equal to the number of pixels of the at least a portion of the raw image data.

[0154] In some implementations, the at least a portion of the raw image data includes all of the raw image data, which may be the case, for example, when a decoded frame is generated using the entirety of the processed image data.

[0155] In some implementations, at least a portion of the raw image data includes a portion of the raw image data, which may be the case, for example, when a decoded frame is generated using a portion of the processed image data.

[0156] In some implementations, calculating the pixel value difference includes, for each pixel of the decoded frame, identifying a color channel of the pixel that matches a color of a corresponding pixel in at least a portion of the raw image data, and discarding, for each pixel of the decoded frame, other color channels that do not match a color of the corresponding pixel in at least a portion of the raw image data, such that the pixel includes a single color channel. For example, with respect to FIG. 6A , the computing device 200 can use the difference module 602 to select a particular color channel for each pixel of the decoded frame 122 based on the color of the corresponding pixel in the raw image data 104 to obtain a selected channel intensity value 604.

[0157] In some implementations, calculating the pixel value difference between at least a portion of the raw image data and the decoded frame includes identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame, and, for each pixel of the at least a portion of the raw image data, taking the difference between the value and the second value. For example, with respect to Figure 6A, the computing device 200 may use the difference module 602 to generate a difference image (e.g., delta 126) by subtracting values ​​of selected channel intensity values ​​604 for pixels in the decoded frame 122 from corresponding color intensity values ​​of at least a portion of the raw image data 104.

[0158] In some implementations, calculating pixel value differences between at least a portion of the raw image data and the decoded frame includes determining a scale of at least a portion of the raw image data and scaling the decoded frame to the scale of at least a portion of the raw image data. Identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame may include, for each pixel of the at least a portion of the raw image data, identifying a value of the pixel and a second value of a corresponding pixel in the decoded frame after the decoded frame is scaled.

[0159] 6A , the difference module 602 of the computing device 200 may scale the values ​​of the decoded frame 122 or selected channel intensity values ​​604 to the scale of the raw image data 104. More specifically, if the raw image data 104 is in a 12-bit scale or color depth, the difference module 602 may, for example, convert the values ​​of the selected channel intensity values ​​604 from an 8-bit scale or color depth to a 12-bit scale or color depth by multiplying them by 16.

[0160] In some implementations, the value of the pixel and the second value of the corresponding pixel are values ​​that represent color intensity. For example, with reference to Figure 6A, a value of 57 in the red channel of pixel 1 of the decoded frame may represent a red intensity of 57 out of 255 on an 8-bit scale.

[0161] The process 800 includes generating 810 an image file that includes both the processed image data and a representation of the pixel value differences. For example, with reference to FIG. 6A , the computing device 200 can use the encoding and compression engine 610 of the differential encoder 124 to encode and / or compress the pixel value differences (e.g., deltas 126). The computing device 200 can then use the image format generator 128 to generate the output image file 130 from the resulting encoded and / or compressed deltas 612 and the processed image data 108.

[0162] In some implementations, generating the image file including both the processed image data and a representation of the pixel value differences includes encoding and compressing the pixel value differences. For example, with reference to FIG. 6A , the computing device 200 may use the encoding and compression engine 610 of the differential encoder 124 to encode the pixel value differences (e.g., deltas 126) and then apply one or more compression steps to the encoded pixel value differences. Applying the encoding may include the engine 610 applying entropy encoding and / or run-length encoding.

[0163] In some implementations, the representation of the pixel value difference is an encoded and compressed pixel value difference. For example, as shown in FIG 7, the output image file 130 may include encoded and / or compressed deltas 612.

[0164] In some implementations, the pixel value differences are a difference image, and encoding and compressing the pixel value differences includes partitioning the difference image into one or more macroblocks, encoding the partitioned image to generate an encoded image, and compressing the encoded image to generate an encoded and compressed image. For example, with respect to Figure 6A, engine 610 of computing device 200 may partition delta 126 (e.g., the difference image) into a number of 8x8 pixel macroblocks, apply Huffman entropy coding to the partitioned difference image, and apply one or more compression steps to the encoded image.

[0165] In some implementations, generating the image file includes generating a digital signature using a representation of the processed image data and the pixel value differences, and the image file includes the digital signature. For example, with reference to FIG. 7, the output image file 130 can include the processed image data 108, the difference image data (e.g., the encoded and / or compressed delta 612), a standard header 702, and a digital signature 706 applied to the header 704.

[0166] The subject matter embodiments and actions and operations described herein may be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware, including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. The subject matter embodiments described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a tangible, non-transitory storage medium for execution by or control of the operation of a data processing device. Alternatively, or in addition, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiving device for execution by the data processing device. The computer storage medium may be, or may be a part of, a machine-readable storage device, a machine-readable storage substrate, a random access memory device, or a serial access memory device, or a combination of one or more thereof. The computer storage medium is not a propagating signal.

[0167] A computer program (which may also be called or expressed as a program, software, software application, app, module, software module, engine, script, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and a computer program may be deployed in any form, such as as a stand-alone program or as a module, component, engine, subroutine, or other unit suitable for execution in a computing environment, which may include one or more computers interconnected by a data communications network in one or more locations.

[0168] A computer program may, but need not, correspond to a file in a file system: a computer program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple associated files (e.g., files that store one or more modules, subprograms, or portions of code).

[0169] To interact with a user, embodiments of the subject matter described herein may be executed on or configured to communicate with a computer having a display device, such as an LCD (liquid crystal display) monitor, for displaying information to the user, and an input device, such as a keyboard, and a pointing device, such as a mouse, trackball, or touchpad, by which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user. For example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, spoken language, or tactile input. Additionally, the computer may interact with the user by sending documents to and receiving documents from a device used by the user. For example, the computer may interact with the user by sending a web page to a web browser on the user's device in response to a request received from the web browser, or by interacting with an app running on the user's device (e.g., a smartphone or electronic tablet). The computer may also interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smart phone running a messaging application) and then receiving reply messages from the user.

[0170] An embodiment of the subject matter described herein may be implemented in a computing system including a back-end component, e.g., a data server, or a computing system including a middleware component, e.g., an application server, or a front-end component, e.g., a computing system including a client device having a graphical user interface, web browser, or app through which a user can interact with an embodiment of the subject matter described herein, or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), e.g., the Internet.

[0171] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interconnected through a communications network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server sends data (e.g., HTML pages) to a user device, for example for the purpose of displaying the data to and receiving user input from a user interacting with the device functioning as a client. Data generated at the user device (e.g., results of user interaction) can be received at the server from the device.

[0172] In addition to the above embodiments, the following embodiments are also inventive. Embodiment 1 is a computer-implemented method comprising: Obtaining raw image data for an image; obtaining processed image data for the image; generating a decoded frame from the processed image data; calculating a pixel value difference between at least a portion of the raw image data and the decoded frame; generating an image file containing both the processed image data and a representation of the pixel value differences.

[0173]

[0026] Embodiment 2 is the method of embodiment 1, wherein generating the decoded frame from the processed image data comprises: identifying a frame of the processed image data; and reconstructing a red, green, blue (RGB) frame from said frame, said RGB frame being said decoded frame.

[0174]

[0026] Example 3 is the method of example 2, wherein the RGB frame includes a set of color channels for each pixel of the RGB frame; Calculating the difference in pixel values for each pixel of the RGB frame, identifying a color channel of the pixel that matches a color of a corresponding pixel of the at least a portion of the raw image data; and for each pixel of the RGB frame, discarding other color channels that do not match the color of the corresponding pixel of the at least a portion of the raw image data such that the pixel contains a single color channel.

[0175] A fourth embodiment is a method according to any one of the first to third embodiments, wherein the step of calculating a difference in pixel values ​​between the at least a portion of the raw image data and the decoded frame comprises: identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame; and calculating a difference between the value and the second value for each of the at least some pixels of the raw image data.

[0176] A fifth embodiment is the method of the fourth embodiment, wherein calculating a pixel value difference between the at least a portion of the raw image data and the decoded frame comprises: determining a scale of the at least a portion of the raw image data; scaling the decoded frame to the scale of the at least a portion of the raw image data; Identifying, for each pixel of the at least a portion of the raw image data, the value of the pixel and the second value of the corresponding pixel in the decoded frame includes, for each pixel of the at least a portion of the raw image data, identifying the value of the pixel and the second value of the corresponding pixel in the decoded frame after the decoded frame is scaled.

[0177] A sixth embodiment is the method according to any one of the fourth to fifth embodiments, wherein the value of the pixel and the second value of the corresponding pixel are values ​​representing color intensity.

[0178] Embodiment 7 is a method according to any one of embodiments 1 to 6, wherein generating an image file including both the processed image data and a representation of the pixel value difference includes encoding and compressing the pixel value difference.

[0179] Example 8 is the method of Example 7, wherein the representation of the pixel value difference is an encoded and compressed pixel value difference.

[0180] A ninth embodiment is the method according to any one of the eighth to ninth embodiments, wherein the pixel value difference is a difference image; The encoding and compressing of the pixel value difference includes: Dividing the difference image into one or more macroblocks; encoding the segmented image to generate an encoded image; and compressing the encoded image to generate an encoded and compressed image.

[0181] Embodiment 10 is a method according to any one of embodiments 1 to 9, wherein generating the image file includes generating a digital signature using the representation of the processed image data and the pixel value difference, and the image file includes the digital signature.

[0182] Embodiment 11 is a system including one or more computers and one or more computer-readable media, the computer-readable media storing instructions that, when executed, cause the one or more computers to perform a method according to any one of embodiments 1 to 10.

[0183] Embodiment 12 is one or more non-transitory computer-readable media, the non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform a method according to any one of embodiments 1 to 10.

[0184] Although the specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or what is or may be claimed, but rather as descriptions of features that may be specific to certain embodiments of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described as a single embodiment may also be implemented in multiple embodiments, either separately or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a particular combination and originally claimed as such, one or more features from the claimed combination may in some cases be deleted from the combination, and the claims may cover subcombinations or variations of the subcombinations.

[0185] Similarly, although operations may be illustrated in the figures and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequential order illustrated, or that all of the illustrated operations be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0186] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still produce desirable results. By way of example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. 1. A computer-implemented method comprising: Obtaining raw image data for an image; obtaining processed image data for the image; generating a decoded frame from the processed image data; calculating a difference in pixel values ​​between at least a portion of the raw image data and the decoded frame; generating an image file including both the processed image data and a representation of the pixel value differences.

2. Generating the decoded frame from the processed image data includes: identifying a frame of the processed image data; and reconstructing a red, green, blue (RGB) frame from the frame, the RGB frame being the decoded frame.

3. the RGB frame includes a set of color channels for each pixel of the RGB frame; Calculating the difference in pixel values identifying, for each pixel of the RGB frame, a color channel of the pixel that matches a color of a corresponding pixel of the at least a portion of the raw image data; and for each pixel of the RGB frame, discarding other color channels that do not match the color of the corresponding pixel of the at least some of the raw image data such that the pixel contains a single color channel.

4. Calculating a difference in pixel values ​​between the at least a portion of the raw image data and the decoded frame includes: identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame; The method according to any one of claims 1 to 3, further comprising: calculating a difference between the value and the second value for each pixel of the at least some of the raw image data.

5. Calculating a difference in pixel values ​​between the at least a portion of the raw image data and the decoded frame includes: determining a scale of the at least a portion of the raw image data; scaling the decoded frame to the scale of the at least a portion of the raw image data; 5. The method of claim 4, wherein identifying, for each pixel of the at least a portion of the raw image data, the value of the pixel and the second value of the corresponding pixel in the decoded frame includes identifying, for each pixel of the at least a portion of the raw image data, the value of the pixel and the second value of the corresponding pixel in the decoded frame after the decoded frame is scaled.

6. The method according to claim 4 or 5, wherein the value of the pixel and the second value of the corresponding pixel are values ​​representing color intensity.

7. A method according to any preceding claim, wherein generating an image file containing both the processed image data and a representation of the pixel value differences comprises encoding and compressing the pixel value differences.

8. The method of claim 7 , wherein the representation of the pixel value difference is the encoded and compressed pixel value difference.

9. the pixel value difference is a difference image; The encoding and compressing of the pixel value difference includes: Dividing the difference image into one or more macroblocks; encoding the segmented image to generate an encoded image; and compressing the encoded image to produce an encoded and compressed image.

10. generating the image file includes generating a digital signature using the representation of the processed image data and the pixel value differences; The method of any one of claims 1 to 9, wherein the image file includes the digital signature.

11. 1. A system comprising: one or more computers; one or more computer-readable media having instructions stored thereon that, when executed, cause the one or more computers to perform operations; The operation includes: Obtaining raw image data for an image; obtaining processed image data for the image; generating a decoded frame from the processed image data; calculating a difference in pixel values ​​between at least a portion of the raw image data and the decoded frame; generating an image file including both the processed image data and a representation of the pixel value differences.

12. Generating the decoded frame from the processed image data includes: identifying a frame of the processed image data; and reconstructing a red, green, blue (RGB) frame from the frame, the RGB frame being the decoded frame.

13. the RGB frame includes a set of color channels for each pixel of the RGB frame; Calculating the difference in pixel values identifying, for each pixel of the RGB frame, a color channel of the pixel that matches a color of a corresponding pixel of the at least a portion of the raw image data; and for each pixel of the RGB frame, discarding other color channels that do not match the color of the corresponding pixel of the at least a portion of the raw image data such that the pixel contains a single color channel.

14. Calculating a difference in pixel values ​​between the at least a portion of the raw image data and the decoded frame includes: identifying, for each pixel of the at least a portion of the raw image data, a value of the pixel and a second value of a corresponding pixel in the decoded frame; The system according to any one of claims 11 to 13, further comprising: calculating a difference between the value and the second value for each pixel of the at least a portion of the raw image data.

15. Calculating a difference in pixel values ​​between the at least a portion of the raw image data and the decoded frame includes: determining a scale of the at least a portion of the raw image data; scaling the decoded frame to the scale of the at least a portion of the raw image data; 15. The system of claim 14, wherein identifying, for each pixel of the at least a portion of the raw image data, the value of the pixel and the second value of the corresponding pixel in the decoded frame includes identifying, for each pixel of the at least a portion of the raw image data, the value of the pixel and the second value of the corresponding pixel in the decoded frame after the decoded frame is scaled.

16. 16. The system of claim 14 or 15, wherein the value of the pixel and the second value of the corresponding pixel are values ​​representing color intensity.

17. 17. The system of claim 11, wherein generating an image file containing both the processed image data and a representation of the pixel value differences comprises encoding and compressing the pixel value differences.

18. 20. The system of claim 17, wherein the representation of the pixel value difference is the encoded and compressed pixel value difference.

19. the pixel value difference is a difference image; The encoding and compressing of the pixel value difference includes: Dividing the difference image into one or more macroblocks; encoding the segmented image to generate an encoded image; and compressing the encoded image to produce an encoded and compressed image.

20. One or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more computers, cause the one or more computers to perform operations, including: Obtaining raw image data for an image; obtaining processed image data for the image; generating a decoded frame from the processed image data; calculating a difference in pixel values ​​between at least a portion of the raw image data and the decoded frame; and generating an image file including both the processed image data and a representation of the pixel value differences.

Images