Authentication of images captured by an image capture device

Abstract

Methods, systems, and apparatus for implementing source and content authentication of image frames in a camera pipeline. A computing system may receive a key from a first processor by an image processor, and the image processor may receive image data from an imaging subsystem, the image data including (i) data representing an image frame and (ii) data hashed in the first hash. The computing system may then generate second hashed data by the image processor using the key and sensor information for the image sensor, and the image processor may determine whether the first hashed data matches the second hashed data. Based at least on whether the first hashed data matches the second hashed data, the computing system may perform a first action regarding the image frame by the image processor.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Application No. 19 / 251,157, filed June 26, 2025. The disclosure of the prior application is considered part of the disclosure of this application and is incorporated herein by reference. Technical Field

[0003] This specification relates to the authentication of image frames, and more specifically to systems and methods for authenticating the source and content of image frames. Background Technology

[0004] Typically, in an imaging system, a sensor (an example of an image capture device) acts as a front-end component that captures light from a real-world scene and converts it into electrical signals for processing by the imaging system. The sensor can then send the raw sensor signals or image data to an image processor for processing. The image processor can perform operations (e.g., color correction, noise reduction, tone mapping, etc.) to transform and enhance the image data into an image that can be displayed, stored (e.g., for later viewing), or transmitted (e.g., via a network) to other devices.

[0005] With artificial intelligence (AI) and machine learning (ML) becoming increasingly embedded in modern computing systems and devices, including cameras, media governance has taken on greater importance in ensuring the authenticity and integrity of images and image data. AI and ML technologies can be used to generate and manipulate image frames and data in real time during the transmission of image data from sensors to image processors, raising concerns about unauthorized modifications to captured image data. Therefore, verifying the source and content of image data can help ensure that the data is original and unmodified when received by the image processor that processes the captured images, which are then subsequently displayed, stored, or transmitted to other systems / devices. Summary of the Invention

[0006] This specification describes techniques for accessing and protecting image data in a computing system by verifying the authenticity of image frames. More specifically, a system can determine whether an image frame has been modified during the transmission of the image data from an imaging subsystem that captured the image data to one or more image processing components.

[0007] Typically, in this disclosure, the authenticity of an image frame refers to whether the image data of that frame is not forged and has not been altered. Specifically, verifying the authenticity of an image frame may involve verifying the source of the image frame, for example, whether the image was generated by a local sensor (e.g., a local sensor of an imaging system / device) rather than by another source (e.g., AI / ML technology), and / or whether the content of the image frame (e.g., image data) has been altered and / or manipulated based on the original captured image data.

[0008] In this disclosure, image data may refer to any data included in and describing an image frame, and may include two components: (i) sensor data, which refers to information about the sensor used to capture the image; and (ii) statistical data, which characterizes the content of the image frame, such as pixel data.

[0009] More specifically, upon receiving an authentication request, the system may (i) use a key and sensor information to verify the source of the image frame, (ii) use pixel data information to verify the content of the image frame, or (iii) both. After determining whether the image frame is authentic in terms of its source and / or content, the system may perform one or more actions based on that determination.

[0010] Specific implementations of the subject matter described in this specification may be implemented to achieve one or more of the following advantages.

[0011] The techniques described in this specification provide a robust, low-cost, and granular method for authenticating image data. While other authentication methods may require significant overhead, the techniques described in this disclosure provide a way to determine the authenticity of image frames early in the image processing pipeline by introducing lightweight components and leveraging existing components and processes within the computing system.

[0012] First, the techniques described in this specification utilize on-device data protection mechanisms, which include generating keys for protecting and authenticating image content to be sent to one or more imaging components of the device. These keys may be generated on a conventional basis or, in some cases, at random intervals to ensure and verify the authenticity of the image content processed by the device. This on-device data protection mechanism can implement data protection (e.g., authentication of the source information) while utilizing lightweight memory components (e.g., one or more registers) to store this information, thereby capturing erroneous data early in the image pipeline and preventing the device from further processing erroneous and / or potentially harmful data.

[0013] Furthermore, the techniques described in this specification can be used by an image signal processor (ISP) to utilize existing pixel data collection procedures during image content processing for content authentication. Pixel data can be collected by the image subsystem before the image data is transmitted to the ISP for processing, and then collected by the ISP after the image data is received to determine whether the image content has been modified during transmission. These techniques can implement a lightweight pixel data collection process in the imaging subsystem (e.g., using one or more counters), and the ISP can use existing pixel data collection procedures to compare the data collected through these two processes to perform content authentication.

[0014] Furthermore, by providing two authentication methods (e.g., source authentication and content authentication) that can be performed jointly or separately, the system enables high-granularity analysis of image data. That is, the system can use (i) source authentication, (ii) content authentication, or (iii) both to verify the authenticity of the source and / or content of the image data. Further, the system can dynamically determine the image data to be authenticated (e.g., image frames), thereby saving computation by selectively performing the authentication process. The system can further save computational resources by performing the authentication process early in the camera pipeline. Specifically, in some cases, the authentication system can determine that an image frame is authentic before storing the data in memory, thus saving computational and storage resources by not maintaining authentic image frames.

[0015] At a higher level, the techniques described in this specification introduce lightweight components and operations that, given the use of registers, counters, keys, and existing processes, facilitate the transmission of source and content information of image data between device components (e.g., to facilitate source and content verification). In other words, the techniques described in this specification achieve proper and robust authentication without introducing any significant latency into the system.

[0016] Details of one or more embodiments of this subject matter will become apparent from the description, accompanying drawings, and claims.

[0017] Other features, aspects and advantages of this subject matter will become apparent from the description, drawings and claims. Attached Figure Description

[0018] Figure 1 This is a block diagram of an example computing system.

[0019] Figure 2 It demonstrates some details of how the computing system operates and its various components.

[0020] Figure 3 This is a diagram illustrating an example flow between components of the computational system during the verification of the authenticity of an image frame.

[0021] Figure 4 This is a flowchart illustrating an example process for verifying the authenticity of an image frame.

[0022] Figure 5 yes Figure 4 A flowchart of a sub-step within a process.

[0023] Figure 6 yes Figure 4 A flowchart of a sub-step within a process.

[0024] The same reference numerals and names in the various figures indicate the same elements. Detailed Implementation

[0025] This disclosure relates to methods and systems for verifying the authenticity of image frames. More specifically, this disclosure describes a computing device (such as an edge device, e.g., a mobile device or tablet computer) having an imaging subsystem and an image signal processor, which can implement access authorization and data protection using sensors and image data to enable verification of the source and content of image frames captured by the imaging subsystem.

[0026] The system can determine whether raw sensor signals or image data transmitted from the imaging subsystem (which includes the image sensor) to the image signal processor (referred to as the ISP or simply the image processor) have been altered after initial capture by the image sensor. In other words, the system can determine whether an unauthorized source manipulated the image data after it was captured by the sensor, or even whether synthetic data generated by that unauthorized source bypassed the data captured in real time.

[0027] Specifically, (for example, as implemented at a computing device) an authentication system can receive a request to authenticate an image frame and can (i) verify the source of the frame, (ii) verify the content of the frame, or (iii) both.

[0028] To verify the source of a frame, the authentication system can use a first processor (e.g., the CPU of an edge device) to generate a key. The key can be sent from the first processor to an imaging subsystem (including a sensor) and an image processor, and received by both.

[0029] The imaging subsystem can receive image frames captured by an image sensor and associated metadata (e.g., sensor static information and sensor configuration information) and hash the metadata of the image frame using a key to generate hashed data representing the source information of the frame. The image processor can use the key to copy the hashed data generated by the imaging subsystem. In some implementations, the image processor can use the key, sensor information and sensor configuration information specific to the local sensor, and the key to generate the hashed data. The image processor can compare the hashed data received from the imaging subsystem with the hashed data generated by itself and determine if they match. If the hashed values ​​match, the source of the image frame is confirmed to be the image sensor included in the imaging subsystem. Conversely, if the hashed values ​​do not match, it indicates that the image sensor is not the source of the received image frame.

[0030] Alternatively, to verify the content of a frame, both the imaging subsystem and the image processor can collect statistics representing the content of the image. For example, the imaging subsystem can collect pixel data representing the image frame when it was first captured by the image sensor, and the image processor can collect pixel data representing the signature of the current frame (e.g., the current content of the image frame). The image processor can compare the pixel data collected by the imaging subsystem with the pixel data collected by the image processor after receiving the image data to determine whether the pixel data matches. If the pixel data matches, it is confirmed that the content of the image frame has not been modified during the transfer from the imaging subsystem to the image processor. On the other hand, if the pixel data does not match, it indicates that the content of the image frame has been modified during the transfer from the imaging subsystem to the image processor.

[0031] In this way, the authentication system can verify (i) that the source of the image frame is authorized, (ii) that the content of the image frame is original and unmanipulated, or (iii) both. By verifying the source and content of the image frame, the authentication system can protect and monitor authorized sensor access and image data.

[0032] In this disclosure, the authorized source of an image frame refers to the local sensor (e.g., a sensor local to the computing device) in which the image data was initially captured. In contrast, an unauthorized source could be, for example, an artificial intelligence (AI) or machine learning (ML) model or other AI / ML technology that has generated different images or otherwise modified the image data captured by the imaging subsystem (and its image sensor).

[0033] In this disclosure, the original, unmanipulated content of an image frame, compared to modified image data / frames, refers to the pixel data of the image frame that has been converted from the original sensor signal generated by the sensor. For example, pixel data in an image frame that has been generated and / or modified by an AI / ML model or technique is not original, unmanipulated content.

[0034] Figure 1 This is a block diagram of an example computing system 100. The example computing system 100 (which may be an edge device) may include a system-on-a-chip (“SoC 102”) and an imaging subsystem 152, the SoC further including a first processor 104, shared memory 108, and IP / circuit blocks 110. The SoC 102 may be implemented in an integrated circuit of an example user / client device 130, a consumer device, or a mobile device, each of which may include items such as: a smartphone 130a, a tablet computer 130b, a laptop computer 130c, and a smartwatch or wearable device 130d (or another edge device). The client device 130 may also include other items such as: an e-netbook, a netbook, a smart speaker, or a mobile computer. In some examples, system 100 and SoC 102 are integrated circuits of a desktop computer, a web server, or an associated cloud-based asset. The SoC 102 may also be coupled to an imaging subsystem 152, which may include an image sensor 153. Figure 1 A single image sensor 153 is shown, but more than one image sensor may be included. Specifically, the image subsystem 152, which includes image sensor 153, may be part of the client device 130.

[0035] The first processor 104 can be any processor configured to perform the operations described in this disclosure. As an example, the first processor 104 can be a central processing unit (CPU). For the purposes of this disclosure, the terms first processor and CPU can be used interchangeably.

[0036] CPU 104 can be a general-purpose CPU (e.g., a single-core or multi-core CPU). An operating system can run on CPU 104. Examples of operating systems include Android, iOS, Windows, etc. The operating system can run one or more image processing applications. For example, image processing applications can include photography applications, video calling applications, authentication applications, facial recognition applications, etc. The operating system can include one or more hardware drivers that allow applications running on that operating system to control the hardware.

[0037] Memory 108 is system memory, shared memory, or both. Figure 1 In the example, memory 108 is described as being external to circuit block 110. However, memory 108 may include the following portions of memory: i) a portion specific to circuit block 110, ii) a portion external to circuit block 110, or iii) both. Memory 108 may be the random access memory of SoC 102, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), or double data rate (DDR) SDRAM. In some implementations, aspects of memory 108 are configured as shared scratchpad memory, which supports parallel access to its memory resources by two or more processors of circuit block 110. Memory 108 may also include various other types of memory, such as high-bandwidth memory (HBM), narrow memory (e.g., for storing 8-bit values), wide memory (e.g., for storing 16-bit or 32-bit values), etc.

[0038] Circuit block 110 may include processing units such as an image signal processor (ISP) 112, a tensor processing unit (TPU) 114, a digital signal processor (DSP) 116, and a graphics processing unit (GPU) 118. Circuit block 110 may also be referred to as IP block 110, wherein the IP block may include one or more dedicated or proprietary hardware elements. For example, each of ISP 112, TPU 114, DSP 116, and GPU 118 may be a corresponding proprietary IP block (or IP device) of a particular entity or device manufacturer.

[0039] In this disclosure, the image processor may be an image signal processor (ISP) 112. The terms image processor and ISP may be used interchangeably.

[0040] The ISP 112 can process image frames or image data (e.g., raw sensor signals) received from the image sensor 153. The ISP 112 can perform one or more processing tasks on the image frames, such as demosaicing, noise reduction, filtering, color correction, etc. For example, the ISP 112 can perform focus, exposure, and white balance correction on the image data provided by the image sensor 153. In some other examples, the ISP 112 can be configured to provide data only to an external driver (such as a driver included in a first image processing stack), which can use algorithms to determine parameters for focus, exposure, and white balance correction and provide the values ​​of these parameters to the ISP 112.

[0041] Before the ISP 112 can perform one or more processing tasks on an image frame, the ISP 112 can determine the authenticity of the image frame. To do this, the ISP 112 can receive a key from a first processor (e.g., CPU 104) and hash that key using sensor information received during the establishment of communication between the ISP 112 and the sensor 153. The ISP 112 can also provide data characterizing the image frame, such as histograms (which are the distribution of pixel intensity on the image), noise statistics, sharpness indices, color statistics, edge detection indices, etc. The data provided by the ISP 112 can be used to verify the content of the image frame against image data collected by the sensor 153 to determine whether the image data has been altered during transmission between the sensor 153 and the ISP 112, and thus whether the image frame has been modified by an unauthorized source.

[0042] Imaging subsystem 152 may include image sensor 153 and (reference) Figure 2 Other components (described in more detail).

[0043] Image sensor 153 may include a digital charge-coupled device (CCD) type or a complementary metal-oxide-semiconductor (CMOS) type image sensor. Image sensor 153 may be... m × n Sensor matrix, which generates and m × n Image data associated with a sensor. For example, image sensor 153 may include hundreds of thousands to hundreds of millions of such sensors. The output of image sensor 153 may be image frames or image data, which can be converted into image frames (e.g., by an image signal processor). In some examples, image sensor 153 may include a front-end module, a mechanical and / or electronic shutter, and an aperture control, the front-end module including a lens, the aperture control controlling the amount of light incident on the sensor. In some examples, the focal length of the lens, the shutter speed, and / or the aperture size (e.g., f-stop) may be adjustable. Image subsystem 152 may also provide adjustments to the sensor (such as sensor gain). Adjusting the sensor gain can change, for example, the intensity of the image generated by image sensor 153. In some instances, these parameter adjustments may be performed by components external to image sensor 153. For example, an image sensor driver may provide a value for each of these parameters of image sensor 153. In some instances, the image sensor 153 can access a register file to extract parameter values, where values ​​can be written to the register file by an external component (e.g., a CPU).

[0044] The example computing system 100 can run one or more image processing applications. For example, the image processing application could be a photo application, a video call application, a video conferencing application, etc. The image processing application can be run by the CPU 104. Once a user (e.g., via a front-end interface set on the device's display) requests to launch the image processing application, the example computing system 100 can perform a series of actions to launch the image processing application.

[0045] When CPU 104 executes one or more image processing applications, it can authenticate image data (e.g., pixel data) generated by image sensor 153 for one or more image processing applications to ensure that the image data has not been altered and / or modified during transmission from sensor 153 to image signal processor (ISP) 112. That is, system 100 can verify that the source of an image frame is an authorized source (e.g., sensor 153) and that the content of the frame is original and unmodified.

[0046] To verify the source of an image frame, CPU 104 can generate a key and send it to sensor 153 and ISP 112 for hashing sensor information corresponding to the source of the image data. ISP 112 can compare the hashed information generated by sensor 153 with the hashed information generated by ISP 112 (itself) to determine if the hashed information matches, and thus determine whether the source of the image frame is genuine.

[0047] To verify the content of an image frame, the imaging subsystem 152 and the ISP 112 can collect pixel data of the image frame. The ISP 112 can compare the pixel data collected by the imaging subsystem 152 with the pixel data collected by the ISP 112 itself to determine whether the pixel data matches, and thus determine whether the content of the image frame is real.

[0048] After determining the authenticity of an image frame, the CPU 104 can be notified, and one or more actions can be performed in response to the determination of authenticity, depending on whether the image frame is authentic. As described above, when an image frame is determined to be authentic, the CPU 104 can continue to run one or more image processing applications using that image frame.

[0049] Figure 2 Showing Figure 1 Some details of the operation of the computing system 100 and its various components.

[0050] The computing system 100 may include a CPU 204 (such as, for example, Figure 1 The first processor / CPU 104 in the imaging subsystem 252 (such as, for example, Figure 1The imaging subsystem 152 and image signal processor 212 (such as, for example, Figure 1 ISP 112), and system memory 208 (such as, for example, Figure 1 (System memory 108 in the system).

[0051] As described above, computing system 100 can perform an authentication process to verify the authenticity of the source and / or content of image data (e.g., image frames) sent to ISP 212 via sensor 253.

[0052] In some implementations, computing system 100 may receive a request from CPU 204 to authenticate an image frame (e.g., frame 203). In some implementations, CPU 204 may generate a request to authenticate image frame 203 in response to additional information received by CPU 204. CPU 204 may receive this additional information from one or more components of the computing device. For example, CPU 204 may receive the additional information from ISP 212, imaging subsystem 252, or itself. Examples of the additional information may include the initiation of an image processing application received from CPU 204 (itself), anomalous pixel data from a sensor received from ISP 212 (e.g., significant changes in pixel data), etc.

[0053] In some implementations, CPU 204 may generate a request to authenticate a frame in response to (i) a notification received by CPU 204 or (ii) an interrupt received by CPU 204. For example, an operating system running on the CPU may send a notification when the camera application has been launched. As another example, ISP 212 may send an interrupt to CPU 204 when a drastic scene change occurs from one image frame to another (e.g., the pixel data differs significantly from one frame to another). As a specific example, if an object appears in the middle of an image frame that did not appear in the previous frame. As another example, if the background of an image frame changes from a dark forest to a sunny beach.

[0054] CPU 204 can generate key 211, which is sent to imaging subsystem 252 and ISP 212 to generate hashed data to verify the authenticity of the source of image frame 203.

[0055] Key 211 can be an access key that can be used to determine authorized access to system 100. That is, authorized components of system 100 can receive the access key to authenticate data generated, transmitted, and / or received by these authorized components. In some implementations, key 211 can be a dynamic access key (e.g., a temporary key that can be regenerated after a certain period of time) to ensure that the key remains confidential from unauthorized sources.

[0056] In some implementations, CPU 204 may generate a key in response to: (i) a request to authenticate an image frame; (ii) additional information received by CPU 204; (iii) a notification received by CPU 204; or (iv) an interrupt received by CPU 204.

[0057] In some implementations, CPU 204 can generate keys without any prompting or notification. For example, CPU 204 can generate keys periodically.

[0058] CPU 204 can generate key 211 in any way. For example, key 211 can be random. As another example, key 211 can be pseudo-random.

[0059] Generally, randomness refers to non-reproducible values ​​generated by unpredictable physical processes, such as the random number when rolling dice. On the other hand, pseudo-randomness refers to values ​​generated by an algorithm that only appear random but are actually deterministic and repeatable. That is, given the same starting input (e.g., a seed), a pseudo-random generator will always produce the same sequence of values ​​because the algorithm follows a fixed sequence of operations.

[0060] CPU 204 can regenerate key 211 at any frequency. In other words, to further enhance security, CPU 204 can generate a new key 211 after any time period.

[0061] For example, CPU 204 can generate key 211 at periodic intervals. This periodic interval can be a random interval, a predetermined interval, a combination of both, or any other interval. For example, CPU 204 can generate key 211 every 4 milliseconds. As another example, CPU 204 can generate key 211 after 3 milliseconds for a first interval, after 12 milliseconds for a second interval, and after 50 milliseconds for a third interval.

[0062] As another example, CPU 204 can generate key 211 for each image frame received by imaging subsystem 252.

[0063] The imaging subsystem 252 may include a sensor 253, a key register 255, and one or more statistical counters 257.

[0064] During the establishment of communication between the imaging subsystem 252 (specifically sensor 253) and the ISP 212, the ISP 212 can obtain sensor information 209 from the sensor 253.

[0065] Sensor information 209 can be any sensor information characterizing sensor 253. For example, sensor information 209 acquired by the ISP can be sensor static information (e.g., sensor serial number). Examples of sensor static information may include sensor serial number, sensor identifier, calibration data, frame rate, or any other static information characterizing the sensor.

[0066] The ISP 212 can further acquire sensor configuration information (e.g., sensor mode). The ISP 212 can acquire sensor configuration information for the sensor during authentication of each image frame, as this sensor configuration information can depend on the specific image processing task and / or each individual image frame. Examples of sensor configuration information may include resolution, binning, remosaic, exposure time, focal length, or any other dynamic settings and / or information characterizing the sensor.

[0067] In some implementations, the ISP 212 may store the sensor information 209 in memory (e.g., in one or more registers in the imaging subsystem 252).

[0068] As a specific example, ISP 212 can acquire the sensor serial number of sensor 153 and the mode of sensor 153 for a specific image frame (e.g., merging, resolution, rearrangement mosaic) during setup.

[0069] As described above, the imaging subsystem 252 can receive key 211 from the CPU 204. The imaging subsystem 252 can then store key 211 in key register 255 until a new key is sent that replaces (e.g., overwrites) the key currently stored in key register 255.

[0070] Sensor 253 can generate image frame 203 and send image frame 203 to ISP 212 for further analysis.

[0071] Before sending it to ISP 212, imaging subsystem 252 can use key 211 received by CPU 204 to generate data with a first hash of the following: (i) sensor static information associated with frame 203 (e.g., sensor serial number) and (ii) sensor configuration information associated with frame 203. That is, image frame 203 may include metadata representing information related to the source of the image frame in frame 203.

[0072] The imaging subsystem 252 may include a microcontroller that includes one or more registers and can implement logic of the type described above (e.g., hash processing, receiving keys, etc.).

[0073] For example, if sensor 253 (e.g., a sensor local to the computing device) generates image frame 203, imaging subsystem 252 can hash the sensor information for sensor 253 using a key. As a specific example, imaging subsystem 152 can hash the sensor serial number associated with image frame 203 and the sensor pattern associated with image frame 203 (e.g., merging, resolution, rearranged mosaic) using key 211.

[0074] Imaging subsystem 152 can use a hash algorithm to hash sensor information with a key. More specifically, imaging subsystem 152 can use a hash algorithm that uses a key to generate hashed information about the sensor data. For example, the hash algorithm could be a hash function using a hash-based message authentication code. As a specific example, the hash algorithm could be a secure hash algorithm (e.g., SHA-1, SHA-224, SHA-256).

[0075] The data after the first hash can be embedded as metadata in frame 203. For example, the data after the first hash can be embedded in the frame header.

[0076] The imaging subsystem 252 can also use the statistical counter 257 to collect statistical image data characterizing the image frame and embed the statistical image data into the frame 203.

[0077] Statistical image data can be any data representing the content of image frame 203. Examples of statistical image data can be pixel data, such as histograms, edge maps, texture data, color statistics, or any other image data. In some implementations, pixel data can be raw pixel data represented as a 3D array with individual values ​​for the red, green, and blue channels of the image. In some implementations, statistical image data can be a histogram. For example, a histogram can be a color histogram or a luminance histogram. A color histogram can be a grayscale histogram or an RGB histogram representing the distribution of pixel colors in image frame 203 (e.g., individual histograms for the red, green, and blue channels). A luminance histogram can represent the distribution of pixel luminance in image frame 203.

[0078] Statistical image data can be stored in various ways. In some implementations, the statistical image data can be stored in one or more registers of the imaging subsystem 252. In some implementations, the statistical image data can be stored in digital memory. For example, the statistical image data can be stored in system memory (e.g., ...). Figure 1 (SRAM 108). As another example, statistical image data can be stored in pre-existing microcontroller memory.

[0079] In some implementations, the collection of statistical image data by the imaging subsystem (e.g., statistical counter 257) can be configured via a SoC (e.g., SoC 102). In some implementations, the collection of statistical image data by the imaging subsystem (e.g., statistical counter 257) can be configured via an additional programmable microcontroller. In some implementations, the statistical image data can be stored in an additional microcontroller memory.

[0080] Statistical image data can be embedded as metadata in frame 203. In some implementations, statistical image data can be embedded in the frame as hashed data. In some implementations, statistical image data can be embedded in the frame as separate segments for different data types (e.g., the histogram is stored in the first segment, and the edge map is stored in the second segment). In some implementations, statistical image data can be embedded in the frame footer. In some implementations, statistical image data can be embedded in the frame header. For example, both the statistical image data and the first hashed data can be embedded in the frame header.

[0081] At a higher level, as an example, an image frame may include: (i) a first portion (e.g., a head) that includes a hash value of sensor information data; (ii) a second portion that includes the content of the image frame (e.g., payload); and (iii) a third portion (e.g., a tail) that includes statistical image data.

[0082] ISP 212 may include: a front-end 215 for verifying the authenticity of the source of image frame 203; and a statistical verification (“statistical verification”) 217 ​​for verifying the authenticity of the content of image frame 203. ISP 212 may perform one or more of the following: (i) source verification using ISP front-end 215; and (ii) content verification using statistical verification 217. In the following description, both source verification and content verification (e.g., statistical verification 217) are described as being performed, but in some implementations, only one or the other may be performed.

[0083] The ISP 212 can receive an embedded frame 203 and a key 211 from the CPU 204 as input. The embedded frame includes (i) image data characterizing the image frame and (ii) data after a first hash representing the source of the image frame.

[0084] The front end 215 can receive the embedded frame 203 and compare the data after the second hash with the data after the first hash embedded in the header of the frame.

[0085] ISP 212 can generate data after a second hash by copying the hash data generated after the first hash. That is, ISP 212 can generate data after a second hash of (i) local sensor (e.g., sensor 253) static information and (ii) local sensor configuration information using key 211.

[0086] The ISP front-end 215 can verify the first hashed data against the second hashed data to determine whether the first hashed data matches the second hashed data. A match between the hashed data indicates that the image frame was captured by an image sensor (e.g., local sensor 253) included in the imaging subsystem 252. A mismatch between the hashed data may indicate that the image frame was captured by an image sensor outside the imaging subsystem 252, and / or generated by another technology (e.g., AI / ML technology). In other words, a mismatch between the hashed data indicates that the image frame has been captured and / or generated by an unauthorized source.

[0087] Based at least on whether the data after the first hash matches the data after the second hash, the ISP 212 can perform a first action regarding the image frame. For example, when there is no match between the data after the first hash and the data after the second hash, the ISP front-end 215 can notify the CPU 204 of the frame authenticity 221, specifically regarding the source of the image frame. That is, the ISP front-end 215 can notify the CPU 204 that frame 203 is not authentic. As another example, when there is no match between the data after the first hash and the data after the second hash, the ISP 212 can notify the CPU 204 of the frame authenticity 221, and the CPU 204 can inform the user of the user device of the authentication failure detection and source. For example, the CPU 204 can send a notification to the user of the user device (e.g., to a camera application being run by the CPU 204) that image frame 203 is not authentic, or that an unauthorized source has generated image frame 203.

[0088] As another example, when the data after the first hash matches the data after the second hash, the ISP 212 can send the frame data 205 of the image frame to the system memory 208 for storage. As another example, when the data after the first hash does not match the data after the second hash, the ISP 212 can discard the image frame 203 without storage (e.g., without allocating memory to store the frame). In other words, the ISP 212 can discard frame 203.

[0089] In some implementations, ISP 212 may perform one or more first actions (e.g., one or more of the first actions described above). For example, ISP 212 may notify CPU 204 that a frame is not real and discard the frame before it can be stored in memory.

[0090] Statistical check 217 can receive frame 203. In some implementations, statistical check 217 can receive the embedded frame 203 from front-end 215 after verifying the source (e.g., ...). Figure 2 (As depicted in the text). Although Figure 2 While not described in the text, in some implementations, statistical verification can be performed by directly receiving the embedded frame 203 from the imaging subsystem 252 to perform content verification.

[0091] The statistical check 217 can receive frame 203, including image data representing the image frame embedded within frame 203. To verify the content of image frame 203, the statistical check 217 can compare the image data embedded within frame 203 with statistical image data (“statistics”) 213, which the ISP 212 has collected for image frame 203 after receiving image frame 203 from sensor 253.

[0092] Specifically, the authentication process can compare the statistical image data 213 already collected by the ISP 212 with the statistical data collected by the imaging subsystem 252. That is, the ISP 212 typically collects the statistical image data 213 in order to process and enhance the image data in the camera pipeline.

[0093] The statistical image data 213 collected by ISP 112 may include any data representing the content of image frame 203. Examples of statistical image data 213 collected by ISP 212 may be pixel data (such as histograms), brightness statistics, color statistics, exposure statistics, noise statistics, edge maps, signal quality statistics, or any other image data.

[0094] In order to compare with the statistical data embedded in the image frame 203 collected by the imaging subsystem 252, the ISP 212 must at least collect and utilize the same statistical data collected by the imaging subsystem 252. For example, the ISP 212 may at least collect and utilize histograms (e.g., brightness and / or color) for comparison with the histograms collected by the imaging subsystem 252.

[0095] Statistical verification 217 can determine whether the pixel data (e.g., histogram) calculated by ISP 212 matches the pixel data of the image embedded in image frame 203. A match between the pixel data and the calculated pixel data indicates that the content of image frame 203 has not been altered and / or modified during transmission between sensor 253 and ISP 212. A mismatch between the pixel data and the calculated pixel data indicates that the content of image frame 203 has been altered and / or modified during transmission between sensor 253 and ISP 212, for example, the content of image frame 203 has been altered by an unauthorized source.

[0096] Based at least on whether the calculated pixel data matches the pixel data, the ISP 212 can perform a second action with respect to the image frame 203.

[0097] For example, ISP 212 can notify CPU 204 of frame authenticity 221, specifically regarding the content of image frame 203, where there is no match between pixel data and calculated pixel data. As another example, when there is no match between pixel data and calculated pixel data, ISP 212 can notify CPU 204 of frame authenticity 221, and CPU 204 can inform the user of the user device of the detection and source of the authentication failure. For example, CPU 204 can send a notification to the user of the user device (e.g., to a camera application running on CPU 204 on the device) that the content of image frame 203 has been altered and / or modified after capture.

[0098] As another example, ISP 212 can discard image frames where there is no match between the pixel data calculated therein and the pixel data included in the image data. As an example of discarding, ISP 212 can delete image frames from system memory 208.

[0099] As another example, when there is a match between the pixel data and the calculated pixel data, the ISP 212 can further process the image frame 203 (as referenced above). Figure 1 (As described). That is to say, the ISP 212 can continue to run one or more image applications and / or continue to process image frame 203.

[0100] In some implementations, ISP 212 may perform one or more second actions (e.g., one or more of the second actions described above). For example, ISP 212 may notify CPU 204 of an illegitimate frame and delete the frame from memory.

[0101] Figure 3 This is a diagram illustrating an example flow between components of the computational system during the verification of the authenticity of an image frame.

[0102] The following example flow details the authentication process, in which the computing system (i) authenticates the source of the image frame and (ii) authenticates the content of the image frame.

[0103] The computing system may include a CPU 304 (such as, for example, Figure 1 The first processor / CPU 104 in the imaging subsystem 352 (such as, for example, Figure 1 Sensor 153 in the middle), ISP front end 315 (such as, for example, Figure 2 ISP front-end 215), and statistical verification 317 (such as, for example, Figure 2 Statistical verification in 217).

[0104] During the establishment of communication between the imaging subsystem 352 (specifically the sensor) and the ISP 312, the ISP front end 315 can acquire sensor information 309 from the sensor (as referenced above). Figure 2 (As described).

[0105] In some implementations, sensor information 309 may include sensor static information (e.g., sensor serial number) and sensor configuration information (e.g., sensor mode).

[0106] CPU 304 can generate key 311 and send the key to imaging subsystem 352 and ISP front end 315.

[0107] CPU 304 can generate key 311 in any way and at any frequency (as referenced above). Figure 2 (As described).

[0108] Imaging subsystem 352 can hash the key using (i) sensor static information and (ii) sensor mode configuration for the current image frame. That is, imaging subsystem 352 can hash the key using source information for the image frame. Imaging subsystem 352 can embed the first hashed data into the header of the image frame to generate an embedded frame 303. Imaging subsystem 352 can further collect statistical data representing the image frame using one or more statistical counters (as referenced above). Figure 2 (As described). Statistical data representing an image frame generates an original frame signature for that image frame (e.g., the statistical data captures the content of the frame before transmitting the image frame to the ISP 112). Imaging subsystem 352 can embed the statistical data into the image frame to generate an embedded frame 303.

[0109] The imaging subsystem 352 can send the embedded frame 303 to the ISP front end 315.

[0110] The ISP front-end 315 can receive the frame 303 with the hashed data in the header and compare the first hashed data from the sensor 353 with the second hashed data. The ISP (itself) can generate the second hashed data from hashing the sensor information 309 (e.g., sensor information received during the establishment of communication between the sensor 353 and the ISP) using a key (previously received by the ISP from the CPU).

[0111] More specifically, the ISP front end 315 can determine whether the data after the first hash matches the data after the second hash in order to determine whether the source of the image frame is the local sensor 353.

[0112] As referenced above Figure 2 As described, after determining authenticity, the ISP can also perform a first action related to the image frame. For example, after determining the authenticity of a frame, the ISP front-end 315 can update the CPU 304 with the authenticity for non-authentic frames (e.g., frames in which the data after the first hash does not match the data after the second hash). For example, for non-authentic frames, the ISP front-end 315 can discard frame 303 without storing it in memory, and for authentic frames (e.g., authentic frames relating to the source of image frame 303), store frame 303 in memory.

[0113] In some implementations, the ISP may perform one or more first actions (e.g., one or more of the first actions described above).

[0114] The ISP front-end 315 can send the embedded frame 303 to the ISP statistics check 317 to compare the statistics representing the image frame embedded in frame 303 with the statistics collected by the ISP (e.g., Figure 2 The statistical check 317 is compared with the statistical data in the ISP (213). That is, the ISP statistical check 317 can determine whether the image data has been modified and / or altered since frame 303 was transmitted from imaging subsystem 352 to ISP.

[0115] The ISP statistical check 317 can compare the statistical image data calculated by the ISP with the image data included in the embedded frame 303, and determine whether the calculated image data matches the image data included in the image frame 303. For example, if the statistical image data is represented as statistical values, the ISP statistical check 317 can perform directional comparisons of the same statistical type, such as comparing histograms with histograms and edge maps with edge maps.

[0116] Then, the ISP statistical check 317 can perform one or more actions regarding the embedded frame 303. For example, if the calculated statistical image data does not match the image data embedded in frame 303, the ISP statistical check 317 can update / notify the CPU 304 with the frame authenticity regarding the content of the image frame.

[0117] Figure 4 This is a flowchart of an example process 400 for verifying the authenticity of an image frame.

[0118] For convenience, process 400 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figure 1 The system 100 depicted herein can execute process 400. Those skilled in the art will understand that the operation of process 400 can be performed by any computing device.

[0119] Process 400 may include receiving a request (402) from a first processor to authenticate an image frame.

[0120] In some implementations, the first processor may generate a request to authenticate the image frame after periodic intervals.

[0121] In some implementations, a first processor (e.g., a CPU) may generate a request to authenticate the image frame in response to various system events. For example, the first processor may generate the authentication request in response to the following (as referenced). Figure 2(To be described in further detail): (i) additional information received by the CPU, (ii) notifications received by the CPU, and (iii) interrupts received by the CPU.

[0122] Alternatively, in some implementations, the system can perform the authentication process without receiving an authentication request from the first processor.

[0123] Upon receiving a request to authenticate an image frame, the system may perform: (i) an authentication process to verify the source of the image frame, (ii) an authentication process to verify the content of the image frame, or (iii) both. That is, the two authentication steps (408 and 410) described below can be performed jointly or separately by the system to authenticate the image frame, as referred to in detail below. Figure 5 and Figure 6 These two certification steps are described further.

[0124] In some implementations, after the authentication process is performed to verify the source of an image frame, the system will not perform the authentication process for the content of the image frame because the image frame has been determined to be authentic.

[0125] Process 400 may include using a key and image data characterizing the image frame to verify the source of the image frame (408).

[0126] More specifically, image data can refer to information about the sensor used to capture the image (e.g., sensor data).

[0127] Image processor (e.g., Figure 1 The ISP 112 can use the key received by the first processor and image data characterizing the image frame to verify the source of the image frame. The image processor can use the key to hash the image data associated with the local sensor and compare the hashed data with the hashed data embedded in the image frame associated with the source of the image frame (see reference below). Figure 5 (As explained below). Based on whether the hashed data from the image processor matches the hashed data from the sensor, the system can perform one or more actions on the image frame (see reference below). Figure 5 (As explained).

[0128] Process 400 may include using image data characterizing an image frame to verify the content of the image frame (410).

[0129] Image processor (e.g., Figure 1The ISP (112) can use the key received by the first processor and image data characterizing the image frame to verify the content of the image frame. Both the ISP and the sensor can collect pixel data of the image frame and compare the pixel data to determine if they match (see reference below). Figure 6 (As explained below). Based on whether pixel data from the image processor matches pixel data from the sensor, the system can perform one or more actions regarding the image frame (see reference below). Figure 6 (As explained).

[0130] Figure 5 and Figure 6 They represent Figure 4 The flowchart of steps 408 and 410 of process 400.

[0131] As described above, the system can execute (i) Figure 5 The sub-steps described in (ii) Figure 6 The sub-steps described in the text, or (iii) both, are used to verify the authenticity of the image frame.

[0132] Figure 5 yes Figure 4 The flowchart of step 408 (i.e., verifying the source of the image frame) in process 400 is a flowchart.

[0133] Process 400 may include using a first processor to generate a key (502). The first processor (e.g., a CPU) may generate the key in any manner and at any frequency (as per reference). Figure 2 and Figure 3 (As explained). Examples include generating random keys, generating pseudo-random keys, generating keys at periodic random intervals, generating keys at set intervals, generating keys for each image frame received by the system, etc.

[0134] Process 400 may include receiving a key (504) from a first processor by the image processor and the imaging subsystem. After the CPU generates the key, it may send the key to the imaging processor and the imaging subsystem for use in generating hash data representing the source of the image frame (as referenced above). Figure 2 and Figure 3 (As explained).

[0135] Process 400 may include receiving data (506) after a first hash from the imaging subsystem by the image processor. (See reference...) Figure 2 and Figure 3As explained, the imaging subsystem can use a key to generate a first hash of (i) sensor static information of an image frame and (ii) sensor configuration information. The imaging subsystem can then send the first hashed data to the image processor to verify that the source of the image frame is a local sensor.

[0136] Process 400 may further include the image processor using a key and sensor information specific to the image sensor to generate second hashed data (508). For comparison with the first hashed data, the image processor may use the key and (i) local sensor static information and (ii) local sensor configuration information to generate the second hashed data (as per reference). Figure 2 and Figure 3 (As explained).

[0137] Process 400 may further include the image processor determining whether the data after a first hash matches the data after a second hash (510). A match between the data after the first hash and the data after the second hash indicates that the source of the image frame is a local sensor included in the imaging subsystem. The image processor may compare the data after the second hash generated by the image processor (itself) with the data after the first hash embedded in the image frame to determine whether the image frame has been modified or injected by an unauthorized source (as per reference). Figure 2 and Figure 3 (As explained).

[0138] Process 400 may further include the image processor performing a first action (512) on the image frame based at least on whether the data after the first hash matches the data after the second hash. Depending on whether the data after the first hash matches the data after the second hash, the ISP may perform the first action. Examples of the first action may include: notifying the CPU of the authenticity of the frame when the data after the first hash does not match the data after the second hash; discarding the frame when the data after the first hash does not match the data after the second hash; and storing the frame in system memory (as per reference) when the data after the first hash matches the data after the second hash. Figure 2 and Figure 3 (As explained). If the data is stored in memory, the back-end components of the ISP can access the data for further processing (e.g., further image processing operations) before providing the data to system components (e.g., for storage in flash memory or for display to the user of the device via the device's display).

[0139] Figure 6 yes Figure 4 The flowchart of step 410 (i.e., verifying the content of the image frame) in process 400 is a sub-step.

[0140] Process 400 may include receiving image data (602) representing an image frame from an imaging subsystem by an image processor. The imaging subsystem may collect pixel data from the image frame and embed the collected pixel data into the image frame (as shown in reference). Figure 2 and Figure 3 (As explained). To determine whether pixel data has changed during the transmission of an image frame to the ISP, the original pixel data is embedded in the image frame.

[0141] Process 400 may further include the image processor calculating pixel data information for the received image frame (602). The image processor may calculate the pixel data information for the image frame based on the image frame received by the ISP (as shown in the reference). Figure 2 and Figure 3 (As explained).

[0142] Process 400 may further include the image processor comparing the computed pixel data with the pixel data included in the image data (604). In some implementations, the pixel data and the computed pixel data may be represented as histograms. The image processor may compare the histogram of the computed pixel data with the histogram of the pixel data to determine whether the histograms match (as shown in the reference). Figure 2 and Figure 3 (As explained).

[0143] Process 400 may further include performing a second action (606) on the image frame by the image processor based at least on whether the calculated pixel data matches the pixel data included in the image frame. The ISP may perform the second action depending on whether the calculated pixel data matches the pixel data included in the image data embedded in the image frame. Examples of the second action may include: notifying the CPU of the frame's authenticity when the calculated pixel data does not match the pixel data; deleting the image frame from memory or discarding the frame if it is not already stored in memory when the calculated pixel data does not match the pixel data; and further processing the image frame (as per reference) when the calculated pixel data matches the pixel data. Figure 2 and Figure 3 (As explained).

[0144] The term "configured" is used in this specification in conjunction with the terms system and computer program components. For a system of one or more computers configured to perform specific operations or actions, this means that the system has software, firmware, hardware, or a combination thereof installed thereon that causes the system to perform those operations or actions in operation. For one or more computer programs configured to perform specific operations or actions, this means that one or more programs include instructions that, when executed by a data processing device, cause that device to perform those operations or actions. Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of the data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, program instructions can be encoded on artificially generated propagated signals (e.g., machine-generated electrical, optical, or electromagnetic signals), which are generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0145] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, apparatuses, and machines used for processing data, including programmable processors, computers, or multiple processors or computers. The device may also be or further include a dedicated logic circuit system, such as a FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In addition to hardware, the device may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0146] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages); and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple coordinating files (e.g., a file storing one or more modules, subroutines, or code sections). A computer program can be deployed to execute on a single computer or on multiple computers located at a site or distributed across multiple sites and interconnected via a data communication network.

[0147] In this specification, the term "database" is used broadly to refer to any collection of data: data that does not need to be structured in any particular way, or does not need to be structured at all, and can be stored on storage devices in one or more locations. Thus, for example, an indexed database may include multiple collections of data, each of which can be organized and accessed differently.

[0148] Similarly, in this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same one or more computers.

[0149] The processes and logic flows described in this specification can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by a dedicated logic circuit system (e.g., an FPGA or ASIC), or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0150] A computer suitable for executing computer programs can be based on a general-purpose microprocessor or a special-purpose microprocessor or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory or both. The basic components of a computer are the central processing unit for making or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into a special-purpose logic circuit system. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to one or more mass storage devices or both. However, a computer does not necessarily need to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name just a few.

[0151] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0152] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser in response to a request received from a web browser on the user's device. Furthermore, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving responsive messages from the user in response.

[0153] Data processing devices used to implement machine learning models may also include, for example, dedicated hardware accelerator units for handling the general and computationally intensive portions of machine learning training or production (i.e., inference, workloads).

[0154] Machine learning models can be implemented and deployed using machine learning frameworks such as TensorFlow, Microsoft Cognitive Toolkit, Apache Singa, or Apache MXNet.

[0155] Embodiments of the subject matter described in this specification can be implemented in computing systems that include back-end components (e.g., as data servers), or middleware components (e.g., application servers), or front-end components (e.g., client computers having a graphical user interface, web browser, or application through which users can interact with implementations of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. Components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0156] A computing system may include clients and servers. Clients and servers are typically geographically separated and interact via a communication network. The client-server relationship is established by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., HTML pages) to a user device, for example, for the purpose of displaying data to a user interacting with the device acting as a client and receiving user input from that user. Data generated at the user device, such as the result of user interaction, may be received at the server from the device.

[0157] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be characteristic of particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may involve sub-combinations or variations thereof.

[0158] Similarly, although the operations in the accompanying drawings correspond to a specific order and are described in the claims in a specific order, this should not be construed as requiring such operations to be performed in the specific order shown or in sequential order, or requiring all the operations shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0159] Specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions stated in the claims can be performed in a different order and still achieve the desired result. As an example, the processes corresponding to the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method implemented at a computing device, the computing device including a first processor, an image processor, and an imaging subsystem, the method comprising: The image processor receives the key from the first processor; The image processor receives image data from the imaging subsystem, the image data including (i) data representing an image frame and (ii) data after a first hash, wherein the imaging subsystem includes an image sensor; The image processor uses the key and sensor information specific to the image sensor to generate the second hashed data; The image processor determines whether the data after the first hash matches the data after the second hash, wherein the match between the data after the first hash and the data after the second hash indicates that the image frame was captured by an image sensor included in the imaging subsystem; as well as The image processor performs a first action on the image frame based at least on whether the data after the first hash matches the data after the second hash.

2. The method of claim 1, wherein the key is generated by the first processor at periodic intervals.

3. The method of claim 1, wherein the key is generated for each image frame received by the imaging subsystem.

4. The method of claim 1, wherein the key is pseudo-random or random, and is generated in response to at least one of (i) a notification received by the first processor and (ii) an interrupt received by the first processor.

5. The method of claim 1, wherein the first action comprises: When the data after the first hash matches the data after the second hash, the image frame is stored in the memory, or When the data after the first hash does not match the data after the second hash, the image frame is discarded without being stored.

6. The method of claim 1, wherein the first hashed data is generated by the imaging subsystem using the key and sensor information corresponding to the image sensor.

7. The method of claim 5, wherein the sensor information includes sensor static information and sensor configuration information, wherein the sensor static information represents a sensor identifier.

8. The method of claim 6, wherein the sensor information is generated by the computing device.

9. The method of claim 1, wherein the data after the first hash is stored in the header of the image frame.

10. The method of claim 1, wherein the image data received from the imaging subsystem includes pixel data, the method further comprising: The image processor calculates pixel data for the received image frame; The image processor compares the calculated pixel data with the pixel data included in the image data; as well as The image processor performs a second action on the image frame based at least on whether the calculated pixel data matches the pixel data included in the image data, wherein the match between the pixel data included in the image data and the calculated pixel data indicates that the content of the image frame has not been changed.

11. The method of claim 10, wherein the second action comprises at least one of the following: A message indicating that the content of the image frame has been changed is sent to the first processor; and If there is no match between the calculated pixel data and the pixel data included in the image data, the image frame is discarded.

12. The method of claim 10, wherein the second action comprises: When the calculated pixel data matches the pixel data included in the image data, the image frame is further processed.

13. An edge device, comprising: First processor; An imaging subsystem, comprising an image sensor; Image processor; One or more storage devices storing a first programming instruction set, wherein the image processor, when executing the first programming instruction set, performs operations including: Receive the key from the first processor; Image data is received from the imaging subsystem, the image data including (i) data representing image frames captured by the image sensor, and (ii) data after a first hash; The second hashed data is generated using sensor information specific to the image sensor. Based on comparing the data after the first hash with the data after the second hash, it is determined that the source of the image frame is the image sensor; as well as Based on determining that the image sensor is the source of the image frame, a first action is performed regarding the image frame.

14. The edge device of claim 13, wherein the one or more storage devices store a second programming instruction set, wherein the first processor, when executing the second programming instruction set, performs operations including: Generate the key; and The key is sent to the image processor and the imaging subsystem.

15. The edge device of claim 14, wherein the first processor, when executing the second programming instruction set, performs operations including: Generate a new key for each of one or more different time intervals; and Each new key is sent to the image processor and the imaging subsystem at one or more different time intervals.

16. The edge device of claim 15, wherein the image processor, when executing the first programming instruction set, performs the following operations: Calculate the pixel data for the image frame; The calculated pixel data is compared with the pixel data included in the image data; as well as A second action is performed on the image frame based at least on whether the calculated pixel data matches the pixel data included in the image data, wherein the match between the pixel data included in the image data and the calculated pixel data indicates that the content of the image frame has not been changed.

17. The edge device of claim 13, wherein the first action comprises at least one of the following: When the data after the first hash matches the data after the second hash, the image frame is stored in the memory. When the data after the first hash does not match the data after the second hash, the image frame is discarded without being stored.

18. The edge device of claim 16, wherein the second action comprises at least one of the following: A message indicating that the content of the image frame has been changed is sent to the first processor; and If there is no match between the calculated pixel data and the pixel data included in the image data, the image frame is discarded.

19. A system comprising: First processor; Image processor; Imaging subsystem; as well as One or more storage devices, the one or more storage devices storing a first programming instruction set and a second programming instruction set, When the first programming instruction set is executed by the first processor, it causes the following operations to be performed: Generate a key; as well as The key is sent to the image processor and the imaging subsystem; The imaging subsystem includes an image sensor and multiple registers, and is configured as follows: Receive the key from the first processor; Generate data representing the image frame, the data including pixel data for the image; Use the key to generate the data after the first hash; as well as The data after the first hash and the data representing the image frame are sent to the image processor; and When the second programming instruction set is executed by the image processor, it causes the following operations to be performed: The key is received from the first processor and image data is received from the imaging subsystem, wherein the image data includes (i) the data representing an image frame and (ii) the data after the first hash; The second hashed data is generated using sensor information specific to the image sensor. The data obtained from the first hash and the data obtained from the second hash are used to determine that the source of the image frame is the image sensor; as well as Based on determining that the image sensor is the source of the image frame, a first action is performed regarding the image frame.

20. The system of claim 19, wherein: The imaging subsystem is further configured as follows: The data representing the image frame is used to generate the data after the third hash; and The data after the third hash is stored at the end of the image frame; and When the second set of programming instructions is executed by the image processor, it causes the following operations to be performed: Calculate the pixel data for the image frame; The calculated pixel data is compared with the pixel data included in the image data; as well as A second action is performed on the image frame based at least on whether the calculated pixel data matches the pixel data included in the image data, wherein the match between the pixel data included in the image data and the calculated pixel data indicates that the content of the image frame has not been changed.

Images