Image difference generation

Abstract

This document provides methods, systems, and devices for image recognition. [Solution] A method for generating output in an authenticated image format, comprising: acquiring raw image data for an image; acquiring compressed image data for the same image; generating an encoded frame from the compressed image data; calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame; and generating an image file that includes both the compressed image data and a representation of the difference in pixel values.

Description

[Technical Field]

[0001] The present disclosure relates to systems and devices for image authentication. [Background Art]

[0002] Image authentication is an application of encoding and decoding trusted images for determining whether a problematic image accurately represents original data captured by an image sensor, such as a camera. Image authentication can be applied to devices or systems that receive and display images, such as camera applications on mobile devices, property management and security devices, and any social media application that displays images.

[0003] Image authentication plays an important role in risk identification and fraud identification for various use cases. For example, when an image of a damaged vehicle is sent to an insurance company during an insurance claim, the insurance company may need to determine whether the image of the damaged vehicle is authentic. As another example, a bank may need to determine whether an image of a check for deposit has been tampered with. A government may want to check whether an image circulated on social media is a fake image that spreads fake news. Users of social media platforms may want to confirm whether an image is authentic or excessively distorted.

[0004] Some image authentication technologies involve generating authentication images in special image formats, which primarily focus on proving that raw image data was captured by a camera sensor. The generated authentication image may include (i) raw image data, (ii) a digital signature on the hash of the raw image data, (iii) a processed image of the raw image data in a compressed image format, and (iv) a digital signature on the hash of the processed image of the raw image data. An example of a special image format is the Digital Negative Image Format (DNG). The origin of an image can also be proven using a watermark. For example, an image may include a watermark that identifies the creator or owner of the image.

[0005] One of the drawbacks of traditional image authentication is that it can be spoofed. For example, if an image has two separate signatures—one for the raw image and one for the processed image—one of the images, along with its corresponding signature, could be maliciously altered or replaced.

[0006] In one general embodiment, the method includes obtaining raw image data of an image, obtaining processed image data of an image, generating a decoded frame from the processed image data, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame, and generating an image file containing both the processed image data and a representation of the difference in pixel values.

[0007] The embodiments include one or more of the following features. For example, in some embodiments, generating a decoded frame from processed image data includes identifying a frame in the processed image data and reconstructing red, green, and blue (RGB) frames from the frame, where the RGB frame is the decoded frame.

[0008] In some embodiments, an RGB frame contains a set of color channels for each pixel, and calculating the difference of pixel values ​​involves identifying the color channel of the pixel that matches the color of the corresponding pixel in at least a portion of the raw image data for each pixel in the RGB frame, and ensuring that the pixel contains a single color channel within the RGB frame. This includes discarding, for each pixel, other color channels in the raw image data that do not match the color of the corresponding pixel in at least a portion of the image data.

[0009] In some embodiments, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame includes, for each pixel in at least a portion of the raw image data, identifying the pixel value and a second value for the corresponding pixel in the decoded frame, and for each pixel in at least a portion of the raw image data, taking the difference between the value and the second value.

[0010] In some embodiments, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame includes determining the scale of at least a portion of the raw image data and scaling the decoded frame to the scale of at least a portion of the raw image data; identifying the pixel value and a second value for the corresponding pixel in the decoded frame for at least a portion of the pixel data, and identifying the pixel value and a second value for the corresponding pixel in the decoded frame for at least a portion of the raw image data, after the decoded frame has been scaled.

[0011] In some embodiments, the pixel value and the corresponding second value of the pixel are values ​​representing color intensity.

[0012] In some embodiments, generating an image file that includes both processed image data and a representation of the difference in pixel values ​​involves encoding and compressing the difference in pixel values.

[0013] In some embodiments, the representation of the difference in pixel values ​​is the difference between encoded and compressed pixel values.

[0014] In some embodiments, the difference in pixel values ​​is a difference image, and encoding and compressing the difference in pixel values ​​includes dividing the difference image into one or more macroblocks, encoding the divided images to generate an encoded image, and compressing the encoded image to generate an encoded and compressed image.

[0015] In some embodiments, generating an image file involves generating a digital signature using a representation of the difference between the processed image data and pixel values, and the image file includes the digital signature.

[0016] Other embodiments of these aspects include computer programs encoded in corresponding systems, devices, and computer storage devices, configured to perform actions of the method. One or more computer systems can be configured in this way by software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform actions during operation. One or more computer programs can be configured in this way by having instructions that cause the device to perform actions when executed by a data processing device.

[0017] Furthermore, conventional authentication images can have large file sizes because, in addition to the processed version of the image data, all raw data may be included in the authentication image. [Overview of the project]

[0018] This specification describes techniques for image authentication. The image authentication techniques described below use output generated in an authenticated image format that includes a post-processed image and additional information that allows an image rendering device to reconstruct a previous version of the image. For example, A system for image authentication may include a first computing device that generates output from compressed images such as JPEG, and from previous or original versions of images such as raw images (e.g., Bayer RGGB). The output may include the compressed image and additional information. The first computing device may send the output to a second computing device of the system, which can reconstruct the raw image from the compressed image and additional information of the output. The second computing device may present the reconstructed image to a user of the second computing device, who can, for example, compare it to the compressed image to authenticate the image or identify any changes made to the compressed image during processing.

[0019] Additional information may include the difference between the post-processed image and the previous version of the image, or a representation of that difference. These differences may be, or represent, the difference in light intensity between the post-processed image and the previous version of the image. For example, if the previous version of the image is raw image data, the system can first convert the post-processed image data (e.g., JPEG) into a decoded frame such as a red-green-blue (RGB) reconstructed image, and then proceed to calculate the difference in light intensity between each corresponding pixel in the RGB reconstructed image. Thus, the calculated difference may include a value for each pixel in the decoded frame. The calculated difference may be referred to or represented by a delta.

[0020] Calculating the difference may involve selecting the color channel of the decoded frame that corresponds to the pixel color of the previous version of the image. For example, an RGB reconstructed image may contain red, green, and blue channels for each pixel, each having its own light intensity (e.g., color intensity) value. For each pixel of the RGB reconstructed image, the first computing device can determine which channel value to use by identifying the color of the corresponding pixel in the raw image. More specifically, if the first pixel in the Bayer RGGB raw image is a green pixel, the first computing device can select the green channel value for the corresponding pixel in the RGB reconstructed image.

[0021] These differences can be used to reconstruct previous versions of the image. For example, a second computing device can proceed to convert the output post-processed image data (e.g., JPEG) into a decoded frame, such as a red-green-blue (RGB) reconstructed image, and then add the differences to the decoded frame to reconstruct the raw image.

[0022] The difference can be compressed and included in the output image file, along with post-processing image data. The output image file can be sent to a computing device configured to use the output image file to reconstruct previous versions of the image from the difference and post-processing images (e.g., JPEG). For example, compressing the difference instead of all previous versions of the image can achieve more efficient compression and reduce the size of the output image file. For example, it is desirable that the output image file includes post-processing images, such as compressed images that can be displayed by a common image viewer. However, an output image file containing both post-processing images and previous versions of the image, such as the raw image, can be very large. Instead, by calculating the difference and including it in the output image file after compression instead of the previous versions of the image, the size of the output image file can be significantly reduced while still providing the ability to reconstruct previous versions of the image.

[0023] As described above, the previous version of image data used to calculate the delta can be raw image data. Raw image data is what the camera has done before any processing has been done. The data may, but does not necessarily, be sensor data acquired by the first device. For example, the camera may be an onboard camera of the first device or a camera of a different device communicating with the first computing device. In some cases, raw image data means data from a previous stage of the image processing pipeline that can be used to calculate the delta. Furthermore, the term image data means any suitable data that can be used to generate an image rendering, and therefore includes the actual image format, and other binary information or intermediate representations that can be used for the same purpose.

[0024] The subject matter described herein can be implemented in specific embodiments to achieve one or more of the following advantages: Rather than generating two separate signatures for raw image data and processed image data as in conventional image authentication formats, the digital signature of the authenticated image format is generated based on the complete content, i.e., both the processed image data and the delta representation, enabling authentication of the output generated in the authenticated image format. That is, the joint signature can ensure that the authentication is based on the entire content of the generated output. The authenticated image format reduces file size because the decoded frame of the processed image data and the delta representation of the raw image data can be stored as part of the generated output rather than storing the entire raw image data. Therefore, compared to conventional image authentication formats, the authenticated image format reduces the costs associated with storing and transmitting the output generated in the authenticated image format.

[0025] After receiving a request to generate raw image data from input data in a certified image format, the image rendering computer can reconstruct the raw image data from a delta representation of the decoded frame of the processed image data and the raw image data. The image rendering computer can present the processed image data and the raw image data via a user interface so that the user can compare the reconstructed raw image data with the processed image data and determine whether the input data in a certified image format is authentic, for example, whether the processed image data is an authentic representation of the raw image data. In some embodiments, the image rendering computer or another computer can receive the processed image data and the raw image data as input and, for example, use a machine learning algorithm to generate a likelihood score indicating whether the processed image data is an authentic representation of the raw image data.

[0026] Furthermore, raw image data acquired directly from sensors may not be optimal for image authentication in some use cases. In many use cases, post-processing of raw image data can be useful in improving the image quality without compromising the authenticity of the image. For example, image enhancements such as noise reduction and motion correction can help improve image quality. In some cases, multiple raw images can be used to create a single processed image data, such as high dynamic range (HDR) imaging or low-light imaging. Therefore, the final post-processed image can be associated with multiple raw images instead of a single raw image. These post-processed images can still be considered authentic images, even if they differ from the raw image data. Authenticated image formats can enable authentication of processed images generated from a single raw image or a series of multiple raw images.

[0027] Furthermore, the authenticated image format may include a digital certificate containing the public key of the public-private key pair of the device that generated the input data in the authenticated image format. The image rendering computer can verify whether the device that generated the input data is trustworthy by verifying the public key of the digital certificate, for example, by verifying the signature on the certificate with a certificate authority (CA) public key. In some embodiments, the image rendering computer may obtain a device public key stored in a trusted server, and verify the digital signature using the obtained device public key . This ensures that an untrusted device cannot generate an unauthenticated public-private key pair and cannot claim that an unauthenticated public-private key pair corresponds to another trusted device.

[0028] The details of one or more embodiments of the subject matter of the present specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. [BRIEF DESCRIPTION OF THE DRAWINGS]

[0029] [Figure 1] FIG. 1 is a diagram of an exemplary system for generating output in an authenticated image format. [Figure 2] FIG. 2 is a diagram of an exemplary computing device. [Figure 3] FIG. 3 is a diagram of an exemplary system for rendering an image in an authenticated image format. [Figure 4] FIG. 4 is a flowchart of an exemplary process for generating output in an authenticated image format. [Figure 5] FIG. 5 is a flowchart of an exemplary process for reconstructing raw image data from input data in an authenticated image format. [Figure 6A] FIG. 6 is a diagram of an exemplary system for generating output in an authenticated image format and reconstructing a previous version of an image from the output. [Figure 6B] This is a diagram illustrating an exemplary system for generating output in a certified image format and reconstructing a previous version of the image from that output. [Figure 7] This figure shows an exemplary certified image format for output. [Figure 8] This is an illustrative process flowchart for generating output in a certified image format. [Modes for carrying out the invention]

[0030] Similar reference symbols and designations in various drawings refer to the same components. Figure 1 is a diagram of an exemplary system 100 for generating output in a certified image format.

[0031] System 100 includes a sensor 102, an image processing pipeline 106, a decoder 120, a differential encoder 124, and a certified image format generator 128.

[0032] Sensor 102 can capture raw image data 104. Sensor 102 can be the camera of system 100 or any other type of imaging sensor. Raw image data 104 can include a raw image in a series of frames captured within a certain period, or multiple raw images. Raw image data 104 can include color images or grayscale images. Raw image data 104 is not processed by any image post-processing method.

[0033] The system 100 can perform post-processing on the raw image data 104 using the image processing pipeline 106. The image processing pipeline 106 may include a series of image processing steps that can produce processed image data 108 with improved image quality and / or reduced file size. Each image processing step may use one or more image processing algorithms. In some embodiments, the image processing pipeline 106 may include steps that can produce enhanced images that improve the appearance of the raw image data, such as denoising, motion correction, and contrast enhancement. In some embodiments, the image processing pipeline 106 may include steps that generate a single enhanced image from multiple raw images in multiple frames, such as HDR / HDR+ shooting or low-light shooting. In some embodiments, the image processing pipeline 106 may include steps that generate a single enhanced image from multiple raw images in multiple frames The process may include arranging multiple raw images and combining multiple raw images into a single image, which can then be further enhanced and compressed by other steps in the image processing pipeline 106.

[0034] In some embodiments, the image processing pipeline 106 may include an encoding step that compresses the raw image data 104 or the enhanced image and generates the processed image data 108 in a compressed image format, such as JPEG, PNG, or TIFF. The processed image data 108 may include the processed image or other formats of processed data generated from the raw image data 104. The processed image data 108 in a compressed image format may have a smaller file size than the raw image data 104.

[0035] Several image processing pipelines 106 can generate processed image data 108, which is an authentic representation of the raw image data 104. For example, an image processing pipeline 106 can generate processed image data 108 using one or more of the following: low-light enhancement, HDR shooting, noise reduction, blur reduction, etc., and the processed image data 108 can still be considered an authentic representation of the raw image data 104.

[0036] However, the image processing pipeline 106 can generate processed image data 108 that is not considered an authentic representation of the raw image. The image processing pipeline 106 can, for example, change the shape, size, position, or texture of objects in the image, and / or add or remove objects in the image. For example, a fake image may be generated by a mobile application that excessively alters the content of an image, such as a fake image of a car with altered damaged parts to submit to an insurance company. As another example, an image may be edited by computer software so that the processed image includes one or more objects that do not belong to the raw image, such as a fake image of a bank check with an altered bank account number.

[0037] Decoder 120 can decode the processed image data 108 and generate a decoded frame 122. Decoder 120 is deterministic and can be specified. The same decoder can later be used in an image rendering computer to reconstruct the raw image from the authenticated image. The decoded frame 122 contains image data equivalent to the format of the raw image data 104. For example, the decoded frame 122 can be an uncompressed image format. In some embodiments, decoder 120 can be a JPEG image decoder that can generate a decoded image from a JPEG encoded image.

[0038] The differential encoder 124 can accept the decoded frame 122 and raw image data 104 of the processed image data as input and can generate a representation of the delta 126 of the decoded frame 122 and raw image data 104 of the processed image data 108. The differential encoder 124 can calculate the delta, for example, the difference image, by subtracting the decoded frame 122 from the raw image of the raw image data 104. The differential encoder 124 can generate a representation of the delta 126 based on a compression algorithm that measures the difference between image frames. For example, the compression algorithm may be similar to a video compression algorithm that measures and compresses the difference between image frames of a video, such as the MPEG-2 algorithm, MPEG-4 part 10 / H.264, or H.265 algorithm. The video compression algorithm can be adapted to measure and compress the difference between the decoded frame 122 and the raw image data 104.

[0039] In some embodiments, when the raw image data 104 includes two or more raw image frames, the difference encoder 124 can compare the decoded frame with a raw image selected from the multiple raw image frames. In some embodiments, the difference encoder 124 can decode You can select a raw image that corresponds to the same or closest point in time as frame 122.

[0040] The authenticated image format generator 128 can combine the processed image data 108 and the delta 126 representations of the decoded frame and raw image data into a single output image file 130 in the authenticated image format. Unlike conventional image authentication methods that store the raw image data 104 in the output image file, the system 100 can include in the output image file 130 a much smaller compressed representation of the delta 126 of the decoded frame 122 of the processed image data 108 and the raw image data 104. Therefore, the output image file 130 in the authenticated image format can have a smaller size than image data generated using conventional image authentication methods.

[0041] Upon request, the image rendering device can reconstruct raw image data from the output image file 130 received by the image rendering device. The reconstructed raw image data can be compared with the processed image data 108 contained in the output image file 130 to determine whether the processed image data 108 is an authentic representation of the reconstructed raw image data. Reconstructing and rendering images in a certified image format will be described in more detail below with reference to Figure 3.

[0042] In some embodiments, the output image file may include a digital signature based on both the processed image data 108 and the delta 126 representation of the decoded frame 122 and the raw image data 104. The joint signature can be based on the complete content of the authenticated image format, i.e., on both the processed image data 108 and the delta 126 representation, and can guarantee the authenticity of the output image file 130. For example, since the digital signature of a conventional authenticated image format is based on either the raw image data or the processed image data, it may be possible to apply a fraudulent modification to some digital signatures of a conventional authenticated image format without identifying it. In contrast, since the digital signature of an authenticated image format is generated based on the complete content of the authenticated image format, it is not possible to apply a fraudulent modification to some digital signatures of the authenticated image format. That is, the joint signature can guarantee that the entire content of the generated output 130 is authentic.

[0043] Figure 2 shows an exemplary computing device 200 capable of generating camera images in an authenticated image format. The computing device 200 includes a system-on-a-chip (SoC) 204 capable of generating authenticated image files, for example, output image files 130 in an authenticated image format. That is, generating authenticated image files can be performed on the device via the SoC 204. Figure 2 and its description use an SoC as an exemplary device. However, the computing device 200 can be any general-purpose processing device.

[0044] The SoC 204 includes a camera control 208 that interacts with the camera sensor 202 of the computing device 200. The camera control 208 is connected to the SoC's central processing unit (CPU) 222 via the SoC bus 234. The camera control 208 can communicate with a camera driver 216 on the CPU 222. In some embodiments, the SoC 204 may include camera firmware 206 that can communicate with the camera driver 216 via the camera control 208.

[0045] The SoC204 includes an ML accelerator 212. The ML accelerator 212 is connected to the CPU 222 via the SoC bus 234. The ML driver 218 can perform post-processing on images captured by the camera sensor 202. The ML driver 218 accelerates one or more steps of the post-processing pipeline 106. The SoC 204 can communicate with the ML accelerator 212 during the image post-processing process. In some embodiments, the SoC 204 may include ML accelerator firmware 210 that can communicate with the ML driver 218 via the ML accelerator 212.

[0046] The SoC204 includes an encryption engine 214, which may be a software or hardware module capable of generating a digital signature for a hash of the target data. An encryption driver (crypto driver) 220 can communicate with the encryption engine 214 via the SoC bus 234. The crypto driver 220 can control the encryption engine 214 and generate a digital signature for the complete content of the data in the authenticated image format, i.e., the processed image data 108 and the delta 126 representation of the decrypted frame 122 of the processed image data 108 and the raw image data 104.

[0047] The cryptographic engine 214 can access the hardware key 228 via the SoC bus 234. The hardware key 228 may contain a public-private key pair associated with the computing device 200. The public-private key pair can indicate whether the computing device 200 is trustworthy. The cryptographic engine 214 can use the private key of the hardware key 228 of the computing device 200 to generate a digital signature of the processing image data 108 and the delta 126 representation. For example, the cryptographic engine 214 can generate a digital signature by using the private key of the hardware key 228 to encrypt the hash value of the processing image data 108 and the delta 126 representation.

[0048] The SoC 204 includes a memory control 224 that controls the memory device 230 of the computing device 200. The memory control 224 is connected to the CPU 222 via the SoC bus 234. The memory device 230 can include both volatile and non-volatile memory, such as random access memory (RAM) and flash RAM. For example, after the camera sensor 202 captures raw image data 104, the raw image data 104 can be stored in a shared buffer of the memory device 230. The CPU can later access the raw image data 104 stored in the memory device 230 and perform post-processing on the raw image data 104 using the ML accelerator 212. The processed image data 108 and the delta 126 representation can also be stored in a shared buffer of the memory device 230 controlled by the memory control 224.

[0049] The SoC204 includes a storage control 226 that controls the storage device 232 of the computing device 200. The storage device 232 can include one or more of the following: a hard disk device, an optical disk device, a solid-state memory device, etc. The storage control 226 is connected to the CPU 222 via the SoC bus 234. After the SoC204 generates an output image file 130 in a certified image format, the output image file 130 can be saved to the storage device 232 controlled by the storage control 226.

[0050] The following is an example process for generating an authentication image file on computing device 200.

[0051] The camera application of the computing device 200 can trigger the camera sensor 202 to capture an image. The camera firmware 206 can communicate with the camera driver 216 on the CPU 222 to save the raw image data 104 to a shared buffer in the memory device 230.

[0052] The camera application can call the image post-processing component of the SoC204 to perform post-processing on the raw image data to generate processed image data, for example, by using HDR+ post-processing to generate the processed image. For example, the camera application can call the ML driver 218 on the CPU222 to perform post-processing on the raw image data 104 to generate processed image data 108. The CPU222 can compare the processed image data 108 with the raw image data 104 to generate the difference, i.e., the delta between the decoded frame 122 of the processed image data 108 and the raw image data 104. In some embodiments, the ML driver 218 can work with the ML accelerator firmware 210 to execute an image post-processing pipeline and generate the delta between the two images.

[0053] The CPU 222 can perform differential coding using the differential encoder 209 to generate a compressed delta of the difference between the raw image data and the decoded frame of the post-processed image data. For example, the differential coding can be entropy coding, Huffman coding, run-length coding, and / or any other suitable technique. After generating the compressed delta, the CPU 222 can add the compressed delta to the processed image data.

[0054] The camera application can invoke the encryption driver 220 to control the encryption engine 214 and generate a digital signature of the processed image data 108 and a delta 126 representation of the decrypted frame 122 of the processed image data 108 and the raw image data 104. In some embodiments, the encryption engine 214 can access the hardware key 228 of the computing device and generate a digital signature using the secret key of the hardware key 228.

[0055] The camera application can generate an output image file 130 in an authenticated image format. The output image file 130 may include a representation of the processed image data 108 and the delta 126. The output image file 130 may further include a digital signature generated based on the representation of the processed image data 108 and the delta 126. The camera application can save the output image file 130 to a storage device 232.

[0056] Figure 3 shows an exemplary system 300 for rendering images in a certified image format. The process of rendering images in a certified image format can be implemented in the operating system of an image rendering computer, as part of a standalone image display or editing application on a computing device, or as part of web browser software, etc.

[0057] The image rendering computer 304 can receive input data 302 in an authenticated image format. The image rendering computer 304 may be a different computer from the computing device that generated the input data 302. As shown in Figure 1, the input data 302 in an authenticated image format may be an output image file 130 generated by an on-device image authentication process. The input data 302 may include processed image data 108 and a delta 126 representation of the decoded frame and raw image data of the processed image data.

[0058] The image rendering computer 304 can, for example, receive a request 306 from a user 316 of the image rendering computer 304 to view the raw image contained in the input data 302.

[0059] For example, a vehicle owner can use a mobile device to capture images of the damaged vehicle. Since the images are captured at night, the mobile device may perform low-light correction and JPEG encoding, which can generate a processed encoded image of the raw image of the damaged vehicle. The mobile device can generate an image file in a certified image format that includes the low-light corrected processed encoded image of the damaged vehicle and a delta representation of the processed image data and the raw image captured by the camera. The vehicle owner can upload the image file in the certified image format to the insurance company's computer during the process of claiming compensation for damage to the vehicle. The insurance company's computer, i.e., the image rendering computer 304, can receive the uploaded image file in the certified image format. The insurance agent 316 can send a request to the computer to view the raw image contained in the uploaded image file in the certified image format. For example, the insurance agent 316 may want to visually compare the raw image and the processed image to determine whether the post-processed image is an authentic representation of the damaged vehicle. In some embodiments, the insurance agent 316 can send a request to the computer instructing it to verify the digital signature contained in the uploaded image file.

[0060] Upon receiving a request to view the raw image 306, the image rendering computer 304 can generate the processed image 308 and the raw image 310 from the input data 302 in an authenticated image format. The image rendering computer 304 can generate the processed image 308 by reading the processed image contained in the input data 302. The image rendering computer 304 can generate a decoded image of the processed image 308 during the generation of the input data 302, for example, by decoding the processed image using the same decoder as decoder 120. The image rendering computer 304 can generate the raw image 310 by combining the decoded image with the representation of delta 126 contained in the input data 302.

[0061] For example, an insurance company's computer can generate a processed image of a damaged vehicle, such as a JPEG encoded image of the damaged vehicle, using a JPEG decoder to decode the JPEG image. The computer can generate a raw image corresponding to the processed image by combining the representation of delta 126 contained in the input data 302 with the decoded image of the processed image 308.

[0062] The image rendering computer 304 may include a display device 314 such as a desktop computer monitor, a mobile device screen, or a VR and / or AR device. In some embodiments, as shown in Figure 3, the image rendering computer 304 may present the processed image 308 and the raw image 310 side by side on the display device 314. In some embodiments, although not shown in Figure 3, the image rendering computer 304 may present a user interface element that allows a user 316 to toggle between the processed image 308 and the raw image 310. In some embodiments, although not shown in Figure 3, the image rendering computer 304 may present a user interface element that displays the difference between the processed image 308 and the raw image 310, for example, a difference image. For example, the difference between two images can be calculated by finding the difference between each pixel of each image and generating an image based on the difference between each pixel of each image.

[0063] In some embodiments, the image rendering computer 304 can receive a request 306 to verify the digital signature contained in the input data 302. In some embodiments, the image rendering computer 304 can verify the digital signature before generating a raw image from the input data 302. If successful, the image rendering computer 304 can proceed to reconstruct the raw image from the input data 302 in the authenticated image format, and the image rendering computer 304 can inform the user 316 that the computing device that generated the input data 302 is a trusted device. If the digital signature verification is unsuccessful, the image rendering computer 304 may not be able to proceed to reconstruct the raw image, and the image rendering computer 304 can inform the user 316 that the computing device that generated the input data 302 is not a trusted device.

[0064] In some embodiments, the image rendering computer 304 can display the device identity 312 on the display device 314. The device identity 312 can be included in a digital certificate along with the device's public key. The digital certificate can be signed by the device manufacturer using the private key of a Certificate Authority (CA), for example, via a sub-CA. The device identity 312 may include identification of the device that generated the input data 302, or the result of digital signature verification, for example, "device verified" or "device not verified". A user 316 can review the digital signature verification result included in the device identity 312 and determine whether the device that generated the input data 302 is trustworthy.

[0065] In some embodiments, the image rendering computer 304 may include a general-purpose image reader, such as a JPEG decoder, that is not specialized in processing the input data 302 in a certified image format. The image rendering computer 304 may further decode and render the processed image, for example, rendering the processed JPEG image as if it were a regular JPEG image. In some embodiments, the image rendering computer 304 may discard the remaining data of the input data 302, for example, by discarding the delta representation between the decoded frame of the processed image and the raw image.

[0066] In some embodiments, the image rendering computer 304 can receive input data that is not in an authenticated image format, for example, a post-processed image in JPEG format. The image rendering computer 304 can still generate the post-processed image by, for example, performing a JPEG decoding process, and display the post-processed image on the display device 314 without generating a raw image 310. Thus, the image rendering computer 304 is backward compatible with displaying input data that is not in an authenticated image format.

[0067] Figure 4 is a flowchart of an exemplary process for generating output in a certified image format. For convenience, the process is described as being performed by a system for generating output images in a certified image format, for example, system 100 in Figure 1. The system may include components of the computing device 200 described with reference to Figure 2, and may include one or more camera sensors, one or more SoCs, one or more memory devices, and one or more storage devices, or any combination thereof.

[0068] The system captures raw image data by an image sensor (402). In some embodiments, the raw image data may include a single image of a single raw frame captured by a camera sensor. In some embodiments, the raw image data may include multiple images of a series of raw frames captured by a camera sensor. The system may perform alignment and merging processes to produce a single image as a raw image representing what has been captured by the camera.

[0069] The system generates processed image data from new image data captured by the image sensor through a processing pipeline having one or more processing elements (404). The processing pipeline may include one or more image processing routines or algorithms that can generate processed image data with improved image quality. The processed image data may include processed images or other forms of processed data generated from raw image data. In some embodiments, the processed image may be in a compressed image format, such as JPEG, PNG, TIFF, or another compressed image format.

[0070] The system generates a decoded frame from the processed image data generated from the processing pipeline (406). In some embodiments, the decoded frame may be in an uncompressed image format. For example, when the processed image is in JPEG format, the system may use a JPEG image decoder to generate a decoded frame from the processed image.

[0071] The system generates output in a certified image format (408). The output includes the processed image data and a delta representation of the decoded frame of the processed image data and the raw image data. That is, instead of saving the raw image data, the system generates the difference between the raw image data and the decoded frame of the processed image data. The system then encodes the difference of the output image in a certified image format so that an image reader can later appropriately reconstruct the raw image data upon request.

[0072] The system can reliably generate and encode raw image frames, regardless of the number and type of processing steps in the processing pipeline used to generate processed image data. An image reader configured to process the output file in a certified image format can then reconstruct and display the raw image frames.

[0073] In some embodiments, the delta representation of the decoded frame of the processed image data and the raw image data can be generated based on a compression algorithm that measures the difference between image frames. The compression algorithm is efficient in processing the difference between image frames, for example, a difference with many zeros. When the two image frames are sufficiently similar to each other, the difference between the two image frames can be efficiently compressed. For example, the system can generate the output using a video encoding algorithm that can encode the difference between frames in a video, such as a video MPEG encoder.

[0074] In some embodiments, the output may include a digital signature based on both the processed image data and a representation of the delta between the decoded frame of the processed image data and the raw image data. That is, the system can sign and authenticate both the processed encoded frame and the differential information, i.e., both the decoded frame of the processed image data and the raw image data, in the same output file.

[0075] In some embodiments, the system can generate a hash value based on the processed image data and the delta representation of the decoded frame of the processed image data and the raw image data, and the system can generate a digital signature based on the hash value. The hash value is a fixed-length numerical value that uniquely identifies the data. Hash values ​​can represent large amounts of data as much smaller numbers.

[0076] In some embodiments, the system can generate a digital signature by encrypting a hash value using the private key of a public-private key pair of the device. The system can access hardware keys associated with the computing devices of the system, for example, the hardware key 228 of the communication device 200 in Figure 2. Yes, it is possible. A public-private key pair can indicate whether a computing device is trustworthy. The image rendering computer can then retrieve the public key of the private key pair associated with the device and verify the digital signature of the output file in an authenticated image format to determine whether the device that generated the output file is trustworthy.

[0077] In some embodiments, the output may include a digital certificate containing the public key of a device's public-private key pair. In some embodiments, the output may further include a device identifier, an expiration date, and a digital signature generated by a CA or sub-CA. The image rendering computer can then use the public key to verify the digital certificate in the output file in the authenticated image format to determine whether the device that generated the output file is trustworthy.

[0078] Figure 5 is a flowchart of an exemplary process for reconstructing raw image data from input data in a certified image format. For convenience, the process is described as being performed by a system for image rendering, for example, system 300 in Figure 3. The system may include an image rendering computer or device that can be configured to analyze input data in a certified image format and reconstruct raw image data from the input data in a certified image format.

[0079] The system receives input data in an authenticated image format (502). The input data includes processed image data and a delta representation of the decoded frame of the processed image data and the raw image data. The processed image data may include the processed image or other forms of processed data generated from the raw image data. For example, the input data may include a post-processed image in JPEG format and the delta of the decoded frame of the post-processed image and the raw image captured by the camera sensor.

[0080] The system receives a request (504) to generate raw image data from the input data. For example, the system may receive a request 306 from a user of the image rendering system to display the raw image 310 and the processed image 308, so that the user can verify the authenticity of the input data 302.

[0081] The system reconstructs the raw image data from the delta representation of the decoded frame of the processed image data and the raw image data (506). In some embodiments, reconstructing the raw image data from the delta representation of the decoded frame of the processed image data and the raw image data may include generating the raw image data by combining the processed image data with the delta representation of the decoded frame of the processed image data and the raw image data.

[0082] In some embodiments, the input data for the authenticated image format may include a digital signature generated based on both the complete content, i.e., the processed image data and the delta representation of the decoded frame of the processed image data and the raw image data. The digital signature generated on the complete content can guarantee the authenticity of the input data. That is, the digital signature can guarantee that the processed image data and the delta representation cannot be altered after the digital signature is computed.

[0083] In some embodiments, the digital signature contained in the input data can be generated by (i) generating a hash value based on the processed image data and the delta representation of the decoded frame of the processed image data and the raw image data, and (ii) encrypting the hash value using the private key of a public-private-key pair associated with a device that generates the input data in an authenticated image format. In some embodiments, the public- The private key pair can be a hardware key associated with the device that generated the input data, for example, the hardware key 228 of computing device 200. The digital signature can indicate whether the input data was generated by a trusted device authorized to access the private key. In some embodiments, the system can obtain the public key of a public-private key pair associated with the device that generates the input in an authenticated image format, and the system can use the public key to verify the digital signature of the input data.

[0084] In some embodiments, input data in a certified image format may include a digital certificate that can be used to verify whether the input data was generated by a trusted device. The system may verify the digital certificate of the input data before verifying the digital signature of the input data in a certified image format, for example, to verify that the input data was generated by a trusted device. Any device can provide a certified image with a valid signature, but only a legitimate device can prove its public key, for example, by including the device's public key in a valid digital certificate. An invalid device manufacturer cannot obtain proof of its public key from, for example, a CA or secure server.

[0085] Depending on the success of the digital certificate verification, the system can extract the device's public key from the digital certificate. The system can then proceed to verify the validity of the input data in the authenticated image format by verifying the digital signature using the device's public key. If the input data is valid, the system can reconstruct the raw image data from the input data in the authenticated image format and display the reconstructed raw image to the user so that the user can determine whether the processed image accurately represents the reconstructed raw image. In some embodiments, the system can provide the reconstructed raw image and the processed image to a computer-implemented algorithm, such as a machine learning algorithm, which can generate a matching score between the reconstructed raw image and the processed image, the matching score representing the likelihood that the processed image accurately represents the reconstructed raw image.

[0086] In some embodiments, the device's public key can be provided by a digital certificate. The digital certificate can be certified by a Certificate Authority (CA) and can be included in the input data in a certified image format. The digital certificate can be generated outside the device and provisioned within the device during manufacturing. The device's public key, associated with the device's private key, can be certified by a CA. To ensure the authenticity of a device that generated input data in a certified image format, the system can verify the digital certificate included in the input data using the CA root's public key. For example, the system can obtain the digital certificate by accessing metadata included in the input data, and the system can verify the digital certificate using the CA root's public key.

[0087] In some embodiments, the device's public key can be stored in a trusted server or database accessible by the system, and the system can retrieve the device's public key stored on the trusted server and use the device's public key to verify the digital signature of the input data. For example, the system can retrieve the device's public key stored on the trusted server based on the identification (ID) provided in the metadata of the input data. The system can then use the device's public key retrieved from the trusted server to verify the digital signature.

[0088] In some embodiments, the system processes the image data and the reconstructed raw image data. These can be presented side-by-side on the system's display device. In some embodiments, the system can present a user interface element that toggles between processed image data and reconstructed raw image data. For example, a user of the system can use a user interface element to toggle between processed images and reconstructed raw images.

[0089] In some embodiments, the system or another system can take processed image data and reconstructed raw image data as input and generate a prediction result indicating whether the processed image data is an authentic representation of the reconstructed raw image data. In some embodiments, the system or another system can use a trained machine learning model to generate a prediction result indicating the likelihood that the processed image data is an authentic representation of the reconstructed raw image data.

[0090] Figures 6A and 6B illustrate an exemplary system 600 for generating output in a certified image format and reconstructing a previous version of the image from that output.

[0091] Figure 6A is a diagram of an exemplary system 600 for generating output in a certified image format. System 600 can be the system 100 described above with respect to Figure 1. System 600 may include the computing device 200 described above with respect to Figure 2, and may include an image rendering computer 304 with the display 314 described above with respect to Figure 3.

[0092] Computing device 200 can be a server, desktop or laptop computer, smartphone, tablet computing device, PDA, or other computing device.

[0093] The image rendering computer 304 can be a computing device such as a server, desktop or laptop computer, smartphone, tablet computing device, or PDA. The display 314 can be an integrated display such as an LED, OLED, LCD, or other display type screen on a laptop computer, smartphone, tablet computing device, or PDA. Alternatively, the display 314 can be an external display such as an LED, OLED, LCD, or other display type monitor electronically connected to the computing device.

[0094] The computing device 200 can communicate with the image rendering computer 304 via a network such as a wireless network. For example, the computing device 200 can communicate with the image rendering computer 304 via a cellular network or the internet.

[0095] The computing device 200 can generate raw image data 104. For example, the computing device 200 can generate raw image data 104 using the camera sensor 202 as described above with respect to Figure 2. Alternatively, the computing device 200 can receive raw image data 104 from different devices via a network.

[0096] The computing device 200 can generate processed image data 108 from raw image data 104 as described above with respect to Figure 1. For example, the computing device 200 can process the raw image data 104 using the image processing pipeline described above with respect to Figure 1 to create an image in a compressed image format (e.g., JPEG) with a smaller file size. It can be converted to G-image, PNG image, etc.

[0097] Alternatively, the computing device 200 can receive processed image data 108 from different devices via a network. For example, the computing device 200 can generate raw image data 104 using a camera sensor and acquire both raw image data 104 and processed image data 108 from a smartphone that generates processed image data 108 from the raw image data 104.

[0098] The raw image data 104 can be in Bayer format. The raw image data 104 can be a Bayer RGGB image, where a group of four pixels includes the top-left pixel which is a red (R) color pixel, the top-right pixel which is a green (G) color pixel, the bottom-left pixel which is also a green (G) color pixel, and the bottom-right pixel which is a blue (B) color pixel. However, the raw image data 104 can be in a different format, such as Bayer BGGR, Bayer RGBG, Bayer GRBG, or other formats with different colors and / or arrangements depending on the camera sensor used to generate the raw image data.

[0099] As an example, raw image data 104 may include a group of four pixels. The group may include a red pixel (raw pixel 1) with a color intensity value of 180 (e.g., an 8-bit intensity value), a first green pixel (raw pixel 2) with a color intensity value of 89, a second green pixel (raw pixel 3) with a color intensity of 78, and a blue pixel (raw pixel 4) with a color intensity of 40. While a simplified example of four pixels is shown and explained, please understand that the raw image data 104 or any part of the raw image data 104 used to generate the output image file 130 may contain any number of pixels.

[0100] In the example shown in Figure 6A, both the raw image data 104 and the decoded frame 122 use 8-bit color (e.g., 8-bit RGB image size or depth), but various other scenarios are possible. For example, the raw image data 104 may have a 10-bit, 12-bit, 14-bit, or 16-bit color depth. Typically, the decoded frame 122 has an 8-bit RGB image size because the processed image data 108 is a JPEG or PNG image that typically has an 8-bit color depth. However, the processed image data 108 and the decoded frame may have different color depths, such as 10-bit or 12-bit. As will be described in more detail below, when the color depth of the raw image data 104 differs from that of the decoded frame 122, the differential encoder 124 may convert the lower-order bit pixel color intensity values ​​of the decoded frame 122 to match the bit size of the raw image data 104 in order to generate the delta 126.

[0101] The raw image data 104 can represent all image data acquired from the camera sensor, or a portion of the image data acquired from the camera sensor. For example, the raw image data 104 can be selected frames from a video captured by the camera sensor.

[0102] The processed image data 108 can also be generated from all image data acquired from the camera sensor, or from a portion of the image data acquired from the camera sensor. For example, if the raw image data 104 is a single frame from a video data object, the processed image data 108 can be generated by the computing device 200 using the same frame from the video data object. As another example, if the raw image data 104 contains the raw image data of the entire video data object so that it contains multiple image frames, the processed image data 108 can similarly contain multiple image frames generated by the computing device 200 using multiple frames of the raw image data 104.

[0103] The decoded frame 122 generated by the decoder 120 can represent all or part of the processed image data 108. For example, if the processed image data 108 contains multiple image frames, the decoder 120 may select specific frames from the processed image data 108 and use them to generate the decoded frame 122. The decoder 120 may continuously generate decoded frames from the processed image data 108, for example, when generating decoded frames, when the differential encoder 124 retrieves previously generated decoded frames, or when processing resources become available. The decoder 120 may continue generating decoded frames until all frames of the processed image data 108 have been converted into decoded frames. Next, the computing device 200 may provide (i) the decoded frame 122 representing part of the processed image data 108, and (ii) the raw image data 104 or the corresponding portion of the raw image data 104 as input to the differential encoder 124.

[0104] The decoded frame 122 may represent all of the processed image data 108. For example, if the processed image data 108 contains a single image frame, the decoded frame 122 can represent all of the processed image data 108. Alternatively, the decoded frame 122 may contain multiple image frames generated by the decoder 120 from multiple frames of the processed image data 108. Next, the computing device 200 can provide (i) the decoded frame 122 representing all of the processed image data 108, and (ii) the raw image data 104 or a corresponding portion of the raw image data 104 as input to the differential encoder 124.

[0105] The decoded frame 122 can be a red-green-blue (RGB) reconstructed image. For example, the decoder 120 can convert the processed image data 108 into an RGB image, and each pixel in the resulting decoded frame 122 has a red channel, a green channel, and a blue channel. Each channel of each pixel in the decoded frame 122 can have its own color or light intensity value. For example, pixel 1 of the decoded frame 122 may include a red channel with a color intensity value of 57 out of 255 (e.g., 8-bit color depth), a green channel with a color intensity value of 205 out of 255, and a blue channel with a color intensity value of 242 out of 255.

[0106] The computing device 200 can provide the decoded frame 122 and the raw image data 104 or a portion of the raw image data 104 corresponding to the decoded frame 122 as input to the differential encoder 124. The differential encoder may include a differential module 602 configured to generate a delta 126 from the decoded frame 122 and the raw image data 104 by calculating the difference between selected channel intensity values ​​604 of the decoded frame 122 and the corresponding intensity values ​​of the raw image data 104. The differential module 602 may be a software module containing one or more models for generating the delta 126. These one or more models may include static or machine learning models.

[0107] The difference module 602 can select the color channels of pixels in the decoded frame 122. For example, the difference module 602 can select a specific color channel for each pixel in the decoded frame 122.

[0108] The difference module 602 can use the raw image data 104 to determine which color channel to select for different pixels in the decoded frame 122. For example, for each pixel in the decoded frame 122, the difference module 602 can identify the corresponding pixel in the raw image data 104 and determine the color of that pixel in the raw image data 104 (e.g., Bayer color fill). The difference module 602 can then determine the color of the corresponding pixel in the raw image data 104. For example, based on pixel 1 in the decoded frame 122 that corresponds to raw pixel 1 in the raw image data 104 which has the color red, the difference module 602 can select the red channel for pixel 1. Similarly, based on the color of the corresponding pixel in the raw image data 104, the difference module 602 can select the green channel for pixel 2 in the decoded frame 122, the green channel for pixel 3 in the decoded frame 122, and the blue channel for pixel 4 in the decoded frame 122.

[0109] Using the technique described above, the difference module 602 can obtain selected channel intensity values ​​604 from the decoded frame 122. For example, the selected channel values ​​may include a red intensity value of 57 out of 255 for pixel 1 of the decoded frame 122, a green intensity value of 255 out of 255 for pixel 2 of the decoded frame 122, a green intensity value of 255 out of 255 for pixel 3 of the decoded frame 122, and a blue intensity value of 40 out of 255 for pixel 4 of the decoded frame 122.

[0110] After obtaining the selected channel intensity value 604, the difference module 602 can calculate the delta 126 by taking the difference between the color intensity value of the raw image data 104 (or a portion of the raw image data 104) and the corresponding value of the selected channel intensity value 604. For example, the difference module 602 can calculate the color intensity value 123 of the delta 126 by subtracting the red intensity value 57 of pixel 1 from the red intensity value 180 of raw pixel 1 in the raw image data 104. The difference module 602 can similarly calculate other intensity values ​​of the delta 126.

[0111] In some embodiments, when the color depth of the raw image data 104 does not match the color depth of the decoded frame 122, the difference module 602 modifies the color depth of the decoded frame 122 or the selected channel intensity value 604 to match the depth of the raw image data 104. For example, if the raw image data 104 uses a 12-bit color depth, the difference module 602 may convert the selected channel intensity value 604 from 8 bits to 12 bits by scaling the decoded frame 122 to a 12-bit size or by multiplying the channel intensity value 604 by 16. The difference module 602 can then obtain the difference between the converted channel value and the corresponding 12-bit value of the raw image data 104 to generate a delta 126.

[0112] Delta 126 can be an image such as an RGB image or a grayscale image. For example, the first pixel of Delta 126 may be a pixel having a red channel with an intensity value of 123, a green channel with an intensity value of 0, and a blue channel with an intensity value of 0. Alternatively, the first pixel of Delta 126 may be a grayscale pixel with an intensity value of 123.

[0113] The difference module 602 can generate metadata for the delta 126. For example, the difference module 602 can generate metadata indicating whether the light intensity or color intensity values ​​of the pixels in the delta 126 should be applied as positive and / or negative values ​​during raw image reconstruction, which will be described in more detail below with respect to Figure 6B. For example, the metadata can identify all pixels for which the intensity value should be applied as a negative value.

[0114] The difference module 602 may instead indicate that the pixels in the delta 126 contain positive and / or negative values ​​by adding a bit to each pixel value. For example, the difference module 602 may generate the delta 126 to have a 9-bit size or depth instead of an 8-bit size or depth, where the added bit indicates the sign of the pixel intensity value. For example, a ninth bit with a value of "0" can indicate that the pixel contains a positive value, and a ninth bit with a value of "1" can indicate that the pixel contains a negative value.

[0115] After generating the delta 126, the difference module 602 can provide the delta 126 to an encoding and compression engine 610 configured to encode and compress the delta 126. For example, the encoding and compression engine 610 can divide the delta 126 into macroblocks of a specific size (e.g., 8x8 pixels, 16x16 pixels, 32x32 pixels, etc.). In some embodiments, the engine 610 can use 1 / 2 pixel and / or 1 / 4 pixel matching during the division.

[0116] After splitting the delta 126, in some embodiments, the engine 610 can apply a discrete cosine transform (DCT), an inverse DCT (IDCT), or another type of DCT.

[0117] After splitting delta 126, in some embodiments, engine 610 can apply entropy coding. For example, after splitting delta 126 and applying DCT, engine 610 can apply Huffman entropy coding to generate the coded and / or compressed delta 612.

[0118] After splitting delta 126, in some embodiments, engine 610 can apply run-length coding. For example, after splitting delta 126, applying DCT, and applying Huffman entropy coding, engine 610 can apply run-length coding to generate the coded and / or compressed delta 612.

[0119] After splitting delta 126, engine 610 can apply one or more compression steps. For example, after splitting delta 126, applying DCT, applying Huffman entropy coding, and applying run-length coding, engine 610 can apply one or more compression steps to generate an encoded and / or compressed delta 612.

[0120] The engine 610 of the differential encoder 124 provides the encoded and / or compressed delta 612 as input to the image format generator 128. As will be described in more detail below with respect to Figure 7, upon receiving the encoded and / or compressed delta 612, the image format generator 128 uses the encoded and / or compressed delta 612 and the processed image data 108 to generate an output image file 130. The output image file 130 may contain the encoded and / or compressed delta 612 and the processed image data 108, such as an image in a compressed image format (e.g., JPEG format).

[0121] For example, by encoding and / or compressing the delta 126 instead of the raw image data 104, more efficient compression can be achieved, and the size of the output image file 130 can be made relatively small. For example, it is desirable that the output image file includes processed image data 108 that can be viewed by a general image viewer, as will be explained in more detail below with respect to Figure 7. However, an output image file containing both processed image data 108 (e.g., JPEG) and raw image data 104 would be very large. Instead, by calculating the delta 126 and including it in the output image file 130 after encoding and / or compression instead of the raw image data 104, the size of the output image file 130 can be significantly reduced while still retaining the ability to reconstruct the raw image data 104, as will be explained in more detail below with respect to Figure 6B.

[0122] This is because the resulting encoded and / or compressed delta 612 is typically smaller in size than the raw image data 104 or the encoded and / or compressed raw image data 104. This is because, at least in part, the difference between pixel intensity values ​​between the raw image data 104 and the selected channels of the decoded frame 1222 (e.g., the reconstructed RGB image) is typically small, allowing for more efficient compression of delta 126 compared to the raw image data 104.

[0123] Figure 6B is a diagram of a portion of the system 600 for reconstructing a previous version of an image from output received in a certified image format. As described above, the system 600 includes an image rendering computer 304 configured to reconstruct a previous version of an image from output in a certified image format, and a display 314 used to present the reconstructed image data next to the processed image data 108.

[0124] The image rendering computer 304 may use an image rendering module 620 configured to reconstruct a previous version of the image, such as the raw image data 104, from the output image file 130. The image rendering module 620 may be a software module including one or more models configured to perform the techniques described below and elsewhere, as described above with respect to Figure 3.

[0125] Upon receiving the output image file 130, the image rendering computer 304 can use the image rendering module 620 to decode and / or decompress the encoded and / or compressed delta 612 of the output image file 130. For example, by decode and decompressing the encoded and / or compressed delta 612 data, the image rendering module 620 can obtain the delta 126.

[0126] Upon receiving the output image file 130, the image rendering computer 304 can use the image rendering module 620 to generate a decoded frame 622 from the processed image data 108 of the output image file 130. For example, the computer 304 may also include a decoder that provides the functionality of the decoder 120, and use that decoder to generate a reconstructed RGB image from the compressed image of the output image file 130. The decoded frame 622 may be the same as the decoded frame 122, as shown in the figure.

[0127] In some embodiments, the decoded frame 622 differs from the decoded frame 122. For example, the decoded frame 622 may differ from the decoded frame 122 based on differences in how the computer 304 generates the decoded frame (e.g., the RGB reconstruction methodology used). However, ideally, the computing device 200 and the computer 304 use the same method to generate, for example, the reconstructed RGB image from the processed image data 108.

[0128] After generating the decoded frame 622, the image rendering module 620 can select color channels for the pixels of the decoded frame 622 and obtain the selected channel intensity values ​​624. The process of selecting color channels for the pixels of the decoded frame 622 to match the color of the corresponding pixel in delta 126 is described in more detail above with respect to the encoded frame 122 in Figure 6A.

[0129] After obtaining the selected channel intensity value 624, the image rendering module 620 can reconstruct the raw image data 104 or a portion of the raw image data 104 by adding the selected channel intensity value 624 to the delta 126. For example, module 620 can take the red channel intensity value 57 of pixel 1 in the decoded frame 622 and the corresponding pixel in the delta 126. In addition to the intensity value of 123, raw pixel 1 of the raw image data 104 having a color intensity of 180 can be reconstructed. Module 620 can repeat this process for the other pixels of the decoded frame 622 and delta 126 to reconstruct raw pixels 2, 3, and 4 of the raw image data 104.

[0130] After reconstructing the raw image data 104 or a portion of the raw image data 104, the image rendering module 620 can demosaice the raw image data 104 or a portion of the raw image data 104 to generate a full-color image 626. The full-color image 626 is not mosaicked in the sense that it does not contain a Bayer pattern image. Instead, each pixel of the full-color image 626 can be a full-color pixel such that the pixel has red channel values, green channel values, and blue channel values. That is, unlike the raw image data 104, for example, the pixels of the full-color image 626 are not limited to a single color channel or filter.

[0131] When generating a full-color image 626, module 620 can apply a demosaiking algorithm. For example, module 620 can generate a full-color image 626 by interpolating the missing color channel values ​​in all pixels of the raw image data 104. For example, through interpolation, module 620 can generate green and blue channel values ​​for raw pixel 1 of the raw image data 104, which contains only red intensity values. The demosaiking algorithm can be, for example, linear interpolation, bilinear interpolation, bicubic interpolation, a uniformly directed demosaiking algorithm, or higher-order interpolation.

[0132] After generating the full-color image 626, the computer 304 can display the full-color image 626 on the interface 630 of the display 314. For example, the computer 304 can display the full-color image 626 next to the processed image data 108. By displaying the full-color image 626 next to the processed image data 108, the user of the computer 304 can easily identify the difference between the processed image data 108 and the raw image data 104 represented by the full-color image 626. The full-color image 626 can function as a ground truth frame that is independent of the multiple potential finishing steps performed on the image to generate the processed image data 108 and the type of finishing steps. In this way, the user of the computer 304 can quickly identify edits made to the image by another user or edits made to the image automatically by the device, for example, when applying filters or enhancing the image (e.g., low-light correction).

[0133] As an example, the image rendering computer 304 can first present processed image data 108 from the output image file 130 on the interface 630. After presenting the processed image data 108, the computer 304 can present the full-color image 626 in response to a request from the user to view the raw image data 104.

[0134] For example, the vehicle owner can use the camera of computing device 200 to capture images of the damaged vehicle. Since the images are captured at night, computing device 200 may perform low-light correction and JPEG encoding, and can generate processed encoded images of the raw images of the damaged vehicle. Computing device 200 can generate an image file in a certified image format that includes the low-light corrected processed encoded image of the damaged vehicle and a delta representation of the processed image data and the raw image captured by the camera of computing device 200. The vehicle owner can upload the image file in the certified image format to the insurance company's computer during the process of claiming compensation for damage to the vehicle. For example, the image rendering computer 304 can receive the uploaded image file in an authenticated image format. A user of computer 304 (for example, insurance agent 316 as described above with respect to Figure 3) can send a request to computer 304 to view the raw image contained in the uploaded image file in an authenticated image format. For example, insurance agent 316 may want to visually compare the raw image with a processed image to determine whether the post-processed image is an authentic representation of the damaged vehicle. In some embodiments, insurance agent 316 can send a request to computer 304 instructing computer 304 to verify the digital signature contained in the uploaded image file.

[0135] In some embodiments, the computer 304 may, or may not, display delta 126 or a representation of delta 126 on the interface 630. For example, the computer 304 may perform demosaicing on delta 126 to generate a difference full-color image, and then display the difference full-color image on the interface 630.

[0136] Figure 7 shows an exemplary authenticated image format for the output image file 130. The authenticated image format can specify that the output image file 130 includes a standard header 702 with a new flag, processed image data 108, a differential image header 704 of the encoded and / or compressed delta 612, the encoded and / or compressed delta 612, and a digital signature 706 applied to the standard header 702, processed image data 108, header 704, and the encoded and / or compressed delta 612.

[0137] Standard header 702 may be the standard image header of processed image data 108, which has a new flag added to the standard image header to indicate the presence of additional image data (e.g., encoded and / or compressed delta 612). The flag may indicate that additional image data appears after the processed image data 108 in the output image file 130.

[0138] A certified image format can specify that differential image data is appended to the output image file after the compressed image data. For example, a certified image format can specify that encoded and / or compressed delta 612 is appended to the output image file 130 after the processed image data 108.

[0139] In some embodiments, the header 704 may indicate the encoding used to generate the differential image data. For example, the header 704 may indicate that Huffman entropy coding was used to generate the encoded and / or compressed delta 612. The computer 304 can identify the type of encoding performed on the differential image data from the header 704 and use this information to decode the encoded and / or compressed delta 612.

[0140] In some embodiments, the header 704 may indicate the compression used to generate the differential image data. For example, the header 704 may indicate the type of compression used to generate the encoded and / or compressed delta 612. The computer 304 can identify the type of compression performed on the differential image data from the header 704 and use this information to decompress the encoded and / or compressed delta 612.

[0141] A certified image format may be backward compatible so that legacy image readers can read and / or present processed image data in an output image file that is in the certified image format. For example, the standard header 702 and processed image data 108 can be processed by standard or legacy compressed image viewers. More specifically, if the processed image data 108 is a compressed image such as JPEG, then the certified image format As a result, a standard JPEG viewer can ignore the new flags, read the standard header 702, and present the processed image data 108 to the display interface. However, this standard JPEG viewer cannot process, for example, the header 704 and the encoded and / or compressed delta 612. Therefore, the standard JPEG viewer in this example cannot reconstruct the raw image data 104 from the processed image data 108 and the encoded and / or compressed delta 612.

[0142] A secure or dedicated image viewer may be configured to process the entire output image file 130 in an authenticated image format. For example, the secure image viewer may be installed on the computer 304 described above, or may be accessed by the computer 304 via a web browser, and may be used by the computer 304 to extract the encoded and / or compressed delta 612 from the output image file 130. The secure image viewer may also be used for additional functions, such as the functions of module 620 described above.

[0143] For example, when opening the output image file 130, the secure image viewer can recognize a new flag in the standard header 702 and use it to determine that the output image file 130 contains differential image data (e.g., encoded and / or compressed delta 612). The secure image viewer can read the header 704 and use the information in the header 704 to obtain the differential image data, present the differential image data, reconstruct the raw image data 104 from the differential image data and processed image data 108, and / or generate a full-color image from the reconstructed raw image data.

[0144] A secure image viewer can also perform the functions of a legacy or standard image viewer, as described above. For example, a secure image viewer can read a standard header 702 and use the information in the standard header 702 to present the processed image data 108.

[0145] The authenticated image format may feature joint authentication. For example, hashing and signing can be performed on the entire data bundle, including the standard header 702, the processed image data 108, the header 704, and the encoded and / or compressed delta 612. The resulting digital signature 706 is included in the output image file 130 according to the authenticated image format and can be used by the computer 304 to authenticate the output image file 130.

[0146] Figure 8 is a flowchart of an exemplary process 800 for generating output in a certified image format. Process 800 can be executed by the system 100 described above. Process 800 can be executed by the system 600 described above with respect to Figures 6A and 6B. For example, process 800 can be executed by the computing device 200 described above with respect to Figures 2 and 6A and 6B.

[0147] Process 800 includes acquiring raw image data of an image (802). For example, the computing device 200 may acquire raw image data 104 from an onboard camera sensor or from an external device via a wireless network. The raw image data 104 may be in a Bayer image format such as Bayer RGGB.

[0148] Process 800 includes acquiring processed image data of the image (804). For example, the computing device 200 can generate processed image data 108 from raw image data 104. Alternatively, the computing device 200 can acquire processed image data from an external device via a wireless network. Processed image data This could be, for example, a JPEG file, a PNG file, or another image file in a compressed and / or encoded image format.

[0149] In some embodiments, the processed image data is encoded image data. For example, the processed image data 108 can be generated by encoding the raw image data 104, as shown in Figure 6A.

[0150] In some embodiments, the processed image data is compressed image data. For example, the processed image data 108 can be generated by compressing the raw image data 104, as shown in Figure 6A.

[0151] In some embodiments, the processed image data is encoded and compressed image data. For example, the processed image data 108 can be generated by encoding and compressing the raw image data 104, as shown in Figure 6A.

[0152] Process 800 includes generating a decoded frame from the processed image data (806). For example, with respect to Figures 1 and 6A, the computing device 200 can use the decoder 120 to generate a decoded frame 122 from a processed image such as the processed image data 108. The decoded frame may be a red-green-blue (RGB) reconstructed image, and each pixel of the decoded frame may have three color channels: a red channel with a red intensity value, a green channel with a green intensity value, and a blue channel with a blue intensity value. The decoded frame may be generated from the entire processed image data such that the decoded frame and the processed image data share the same number of pixels. Alternatively, the decoded frame may be generated from a portion of the processed image data.

[0153] Process 800 includes calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame (808). For example, with respect to Figures 1 and 6A, the computing device 200 can use the difference module 602 of the difference encoder 124 to calculate the difference between the decoded frame 122 and at least a portion of the raw image data 104. Since each pixel of the decoded frame may have a corresponding pixel in at least a portion of the raw image data, the number of pixels in the decoded frame can be equal to the number of pixels in at least a portion of the raw image data.

[0154] In some embodiments, at least a portion of the raw image data includes all of the raw image data. This may be the case, for example, when the decoded frame is generated using the entirety of the processed image data.

[0155] In some embodiments, at least a portion of the raw image data includes a portion of the raw image data. This may be the case, for example, when the decoded frame is generated using a portion of the processed image data.

[0156] In some embodiments, calculating the difference in pixel values ​​involves, for each pixel in the decoded frame, identifying the color channels of pixels that match the color of the corresponding pixel in at least a portion of the raw image data, and for each pixel in the decoded frame, discarding other color channels that do not match the color of the corresponding pixel in at least a portion of the raw image data, so that the pixel contains a single color channel. For example, with respect to Figure 6A, the computing device 200 can use the difference module 602 to select a specific color channel for each pixel in the decoded frame 122 based on the color of the corresponding pixel in the raw image data 104, and obtain a selected channel intensity value 604.

[0157] In some embodiments, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame includes, for at least a portion of the pixels in the raw image data, identifying the value of the pixel and a second value for the corresponding pixel in the decoded frame, and for at least a portion of the pixels in the raw image data, taking the difference between the value and the second value. For example, with respect to Figure 6A, the computing device 200 can generate a difference image (e.g., delta 126) by using a difference module 602 to subtract the value of a selected channel intensity value 604 for a pixel in the decoded frame 122 from the corresponding color intensity value of at least a portion of the raw image data 104.

[0158] In some embodiments, calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame includes determining the scale of at least a portion of the raw image data and scaling the decoded frame to the scale of at least a portion of the raw image data. Identifying, for at least a portion of the pixels in the raw image data, the pixel value and the second value of the corresponding pixel in the decoded frame may include identifying, for at least a portion of the pixels in the raw image data, the pixel value and the second value of the corresponding pixel in the decoded frame after the decoded frame has been scaled.

[0159] For example, with respect to Figure 6A, the difference module 602 of the computing device 200 can scale the values ​​of the decoded frame 122 or the selected channel intensity values ​​604 to the scale of the raw image data 104. More specifically, if the raw image data 104 is within a 12-bit scale or color depth, the difference module 602 can convert the values ​​of the selected channel intensity values ​​604 from an 8-bit scale or color depth to a 12-bit scale or color depth by multiplying them by 16.

[0160] In some embodiments, the pixel value and the corresponding second value of the pixel are values ​​representing color intensity. For example, in Figure 6A, the red channel value of pixel 1 in the decoded frame, 57, can represent a red intensity of 57 out of 255 on an 8-bit scale.

[0161] Process 800 includes generating an image file containing both the processed image data and a representation of the difference in pixel values ​​(810). For example, with respect to Figure 6A, the computing device 200 can encode and / or compress the difference in pixel values ​​(e.g., delta 126) using the encoding and compression engine 610 of the difference encoder 124. The computing device 200 can then generate an output image file 130 from the resulting encoded and / or compressed delta 612 and the processed image data 108 using the image format generator 128.

[0162] In some embodiments, generating an image file containing both processed image data and a representation of the difference in pixel values ​​involves encoding and compressing the difference in pixel values. For example, with respect to Figure 6A, the computing device 200 may use the encoding and compression engine 610 of the difference encoder 124 to encode the difference in pixel values ​​(e.g., delta 126), and then apply one or more compression steps to the encoded difference in pixel values. Applying encoding may include the engine 610 applying entropy encoding and / or run-length encoding.

[0163] In some embodiments, the representation of the difference in pixel values ​​is the difference between encoded and compressed pixel values. For example, as shown in Figure 7, the output image file 130 may include encoded and / or compressed delta 612.

[0164] In some embodiments, the difference in pixel values ​​is a difference image, and the difference in pixel values ​​is encoded and Compression includes dividing the difference image into one or more macroblocks, encoding the divided images to generate an encoded image, and compressing the encoded image to generate an encoded and compressed image. For example, with respect to Figure 6A, the engine 610 of the computing device 200 can divide the delta 126 (e.g., difference image) into a plurality of 8x8 pixel macroblocks, apply Huffman entropy coding to the divided difference images, and apply one or more compression steps to the encoded image.

[0165] In some embodiments, generating an image file involves generating a digital signature using a representation of the difference between the processed image data and pixel values, and the image file includes the digital signature. For example, with respect to Figure 7, the output image file 130 may include the processed image data 108, the difference image data (e.g., encoded and / or compressed delta 612), a standard header 702, and a digital signature 706 applied to the header 704.

[0166] Embodiments of the subject matter, as well as actions and operations, described herein may be implemented in digital electronic circuits, tangibly embodied computer software or firmware, or computer hardware, including structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded in a tangible, non-temporary storage medium, which are executed by or control the operation of a data processing device. Alternatively or additionally, program instructions may be encoded in artificially generated propagating signals, such as machine-generated electrical signals, optical signals, or electromagnetic signals, which are generated to encode information to be sent to a suitable receiving device for execution by the data processing device. The computer storage medium may be a machine-readable memory device, a machine-readable memory board, a random-access memory device, or a serial-access memory device, or a combination of one or more thereof, or a part thereof. The computer storage medium is not a propagating signal.

[0167] A computer program (which may also be called, or be expressed as, a program, software, software application, app, module, software module, engine, script, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages. A computer program may also be deployed in any form, such as a standalone program, or as a module, component, engine, subroutine, or other unit suitable for execution in a computing environment, which may include one or more computers interconnected by a data communication network in one or more locations.

[0168] Computer programs may, but do not necessarily, correspond to files in a file system. They can be stored in a part of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program itself, or in multiple collaborative files (e.g., files containing one or more modules, subprograms, or parts of code).

[0169] To interact with a user, embodiments of the subject matter described herein may be run on or configured to communicate with a computer having a display device for displaying information to the user, such as an LCD (liquid crystal display) monitor, and an input device for the user to input into the computer, such as a keyboard, and a pointing device, such as a mouse, trackball, or touchpad. Other types of devices may also be used to provide user interaction. Example For example, the feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or haptic feedback, and user input can be received in any form, including acoustic, spoken language, or haptic input. Furthermore, a computer can interact with the user by sending documents to and receiving documents from a device used by the user. For example, it can interact with the user by sending a web page to a web browser on the user's device in response to a request received from a web browser, or by interacting with an application running on the user's device (e.g., a smartphone or electronic tablet). The computer may also interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and then receiving a response message from the user.

[0170] Embodiments of the subject matter described herein may be implemented as a computing system including backend components, such as a data server, or a computing system including middleware components, such as an application server, or a computing system including frontend components, such as a client device having a graphical user interface, a web browser, or an application that a user can interact with the embodiments of the subject matter described herein, or as any combination of one or more such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and, for example, the Internet.

[0171] A computing system can include a client and a server. The client and server are generally geographically separated and typically interconnected via a communication network. The client-server relationship arises from computer programs running on each computer and having a client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to a user device for the purpose of displaying data to a user interacting with a device acting as a client and receiving user input from that user. Data generated on the user device (e.g., the results of user interactions) can be received by the server from the device.

[0172] In addition to the embodiments described above, the following embodiments are also original. Embodiment 1 is a method performed on a computer, Obtaining raw image data of an image, To obtain processed image data of the aforementioned image, The above processed image data is used to generate a decoded frame, The process involves calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame, This includes generating an image file that includes both the processed image data and the representation of the difference between the pixel values.

[0173] Embodiment 2 is the method of Embodiment 1, and generating the decoded frame from the processed image data is: Identifying the frame of the processed image data, This includes reconstructing red, green, and blue (RGB) frames from the above frames, where the above RGB frames are the decoded frames.

[0174] Embodiment 3 is the method of Embodiment 2, wherein the RGB frame includes a set of color channels for each pixel of the RGB frame. Calculating the difference in the above pixel values ​​means For each pixel in the above RGB frame, identify the color channel of the pixel that matches the color of at least some of the corresponding pixels in the above raw image data. This includes discarding other color channels in the raw image data that do not match the color of the corresponding pixel in at least some of the RGB frames, so that each pixel contains a single color channel.

[0175] Embodiment 4 is a method according to any one of Embodiments 1 to 3, wherein the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is calculated as follows: For each of the pixels in the raw image data described above, the value of the pixel and the second value of the corresponding pixel in the decoded frame are identified. This includes taking the difference between the above value and the second value for each pixel of the above raw image data.

[0176] Embodiment 5 is the method of Embodiment 4, and the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is calculated as follows: To identify the scale of at least a portion of the above raw image data, This includes scaling the above-described decoded frame to the above-described scale of at least a portion of the above-described raw image data, Identifying, for each of the above-mentioned pixels in the raw image data, the above-mentioned value of the pixel and the above-mentioned second value of the corresponding pixel in the decoded frame, includes, for each of the above-mentioned pixels in the raw image data, after the decoded frame has been scaled, identifying the above-mentioned value of the pixel and the above-mentioned second value of the corresponding pixel in the decoded frame.

[0177] Embodiment 6 is a method according to any one of Embodiments 4 to 5, wherein the above value of the pixel and the above second value of the corresponding pixel are values ​​representing color intensity.

[0178] Embodiment 7 is a method according to any one of Embodiments 1 to 6, wherein generating an image file that includes both the processed image data and the representation of the difference in pixel values ​​includes encoding and compressing the difference in pixel values.

[0179] Embodiment 8 is the method of Embodiment 7, wherein the above representation of the difference in pixel values ​​is the difference in encoded and compressed pixel values.

[0180] Embodiment 9 is a method according to any one of Embodiments 8 to 9, wherein the difference in the above pixel values ​​is a difference image. Encoding and compressing the difference in the above pixel values ​​is The above difference image is divided into one or more macroblocks, The above divided images are encoded to generate an encoded image, This includes compressing the above encoded image to generate an encoded and compressed image.

[0181] Embodiment 10 is a method according to any one of Embodiments 1 to 9, wherein generating the image file includes generating a digital signature using the above-mentioned representation of the difference between the processed image data and the pixel values, and the image file includes the above-mentioned digital signature.

[0182] Embodiment 11 includes one or more computers and one or more computer-enabled devices. The system includes a reading medium, the computer-readable medium storing instructions that, when executed, cause one or more computers to perform the method described in any one of embodiments 1 to 10.

[0183] Embodiment 12 is one or more non-temporary computer-readable media, which, when executed by one or more computers, stores instructions that cause one or more computers to perform the method described in any one of Embodiments 1 to 10.

[0184] This specification includes details of many specific embodiments, which should not be construed as limitations on the scope of any invention or the scope of what is claimed or potentially claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described as a single embodiment may also be implemented in multiple embodiments, individually or in any preferred secondary combination. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from a claimed combination may be removed from the combination, and the claims may cover secondary combinations or variations of secondary combinations.

[0185] Similarly, while operations are shown in the drawings and described in a specific order in the claims, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all shown operations be performed, in order to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0186] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order, and this may still yield desirable results. As an example, the process shown in the accompanying figures does not necessarily require that the actions be performed in the specific order or sequence shown to obtain the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method that is performed on a computer, Obtaining raw image data of an image, To obtain processed image data of the aforementioned image, The process involves generating a decoded frame from the processed image data, The process involves calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame, A method performed on a computer, comprising generating an image file that includes both the processed image data and a representation of the difference between the pixel values.

2. Generating the decoded frame from the processed image data is Identifying the frame of the processed image data, The method according to claim 1, comprising reconstructing red, green, and blue (RGB) frames from the frame, wherein the RGB frame is the decoded frame.

3. The RGB frame includes a set of color channels for each pixel of the RGB frame. Calculating the difference in the aforementioned pixel values ​​is, For each pixel of the RGB frame, identify the color channel of the pixel that matches the color of at least some of the corresponding pixels in the raw image data. The method according to claim 2, further comprising discarding other color channels in the raw image data that do not match the color of the corresponding pixel, such that for each pixel of the RGB frame, the pixel contains a single color channel.

4. Calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is: For each of the pixels in the raw image data, the value of the pixel is identified with the second value of the corresponding pixel in the decoded frame. The method according to any one of claims 1 to 3, comprising taking the difference between the value and the second value for each of the pixels of the raw image data.

5. Calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is: Identifying the scale of at least a portion of the raw image data, This includes scaling the decoded frame to the scale of at least a portion of the raw image data, The method according to claim 4, wherein for each of the at least portion of pixels in the raw image data, identifying the value of the pixel and the second value of the corresponding pixel in the decoded frame comprises identifying the value of the pixel and the second value of the corresponding pixel in the decoded frame for each of the at least portion of pixels in the raw image data after the decoded frame has been scaled.

6. The method according to claim 4 or 5, wherein the value of the aforementioned pixel and the second value of the corresponding pixel are values ​​representing color intensity.

7. The method according to any one of claims 1 to 6, wherein generating an image file that includes both the processed image data and the representation of the difference in pixel values ​​includes encoding and compressing the difference in pixel values.

8. The method according to claim 7, wherein the representation of the difference in pixel values ​​is the difference in the encoded and compressed pixel values.

9. The difference in the aforementioned pixel values ​​is a difference image. Encoding and compressing the difference in the aforementioned pixel values ​​is The aforementioned difference image is divided into one or more macroblocks, The divided images are encoded to generate an encoded image, The method according to claim 7 or 8, comprising compressing the encoded image to generate an encoded and compressed image.

10. The generation of the image file includes generating a digital signature using the processed image data and the representation of the difference between the pixel values, The method according to any one of claims 1 to 9, wherein the image file includes the digital signature.

11. It is a system, One or more computers, It comprises one or more computer-readable media storing instructions, and when an instruction is executed, it causes the one or more computers to perform an action. The aforementioned operation is, Obtaining raw image data of an image, To obtain processed image data of the aforementioned image, The process involves generating a decoded frame from the processed image data, The process involves calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame, A system that includes generating an image file containing both the processed image data and a representation of the difference between the pixel values.

12. Generating the decoded frame from the processed image data is Identifying the frame of the processed image data, The system according to claim 11, comprising reconstructing red, green, and blue (RGB) frames from the frame, wherein the RGB frame is the decoded frame.

13. The RGB frame includes a set of color channels for each pixel of the RGB frame. Calculating the difference in the aforementioned pixel values ​​is, For each pixel of the RGB frame, identify the color channel of the pixel that matches the color of at least some of the corresponding pixels in the raw image data. The system according to claim 12, comprising discarding, for each pixel of the RGB frame, other color channels in the raw image data that do not match the color of the corresponding pixel, such that the pixel contains a single color channel.

14. Calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is: For each of the pixels in the raw image data, the value of the pixel is identified with the second value of the corresponding pixel in the decoded frame. The system according to any one of claims 11 to 13, comprising taking the difference between the value and the second value for each of the pixels of the raw image data.

15. The difference in pixel values ​​between at least a portion of the raw image data and the decoded frame is To calculate, Identifying the scale of at least a portion of the raw image data, This includes scaling the decoded frame to the scale of at least a portion of the raw image data, The system according to claim 14, wherein for each of the at least portion of pixels in the raw image data, identifying the value of the pixel and the second value of the corresponding pixel in the decoded frame comprises identifying the value of the pixel and the second value of the corresponding pixel in the decoded frame for each of the at least portion of pixels in the raw image data after the decoded frame has been scaled.

16. The system according to claim 14 or 15, wherein the value of the aforementioned pixel and the second value of the corresponding pixel are values ​​representing color intensity.

17. The system according to any one of claims 11 to 16, wherein generating an image file that includes both the processed image data and a representation of the difference in pixel values ​​includes encoding and compressing the difference in pixel values.

18. The system according to claim 17, wherein the representation of the difference in pixel values ​​is the difference in the encoded and compressed pixel values.

19. The difference in the aforementioned pixel values ​​is a difference image. Encoding and compressing the difference in the aforementioned pixel values ​​is The aforementioned difference image is divided into one or more macroblocks, The divided images are encoded to generate an encoded image, The system according to claim 17 or 18, comprising compressing the encoded image to generate an encoded and compressed image.

20. One or more non-temporary computer-readable media storing instructions, wherein when an instruction is executed by one or more computers, the one or more computers are instructed to perform an action, and the action is: Obtaining raw image data of an image, To obtain processed image data of the aforementioned image, The process involves generating a decoded frame from the processed image data, The process involves calculating the difference in pixel values ​​between at least a portion of the raw image data and the decoded frame, One or more non-temporary computer-readable media, comprising generating an image file that includes both the processed image data and a representation of the difference between the pixel values.

Images