Optical image processing

Abstract

An image processing unit is used to determine whether an optical image has been maintained in a state that has not been modified by artificial intelligence. In exemplary embodiments, the image processing unit determines whether the optical image has been maintained in a state that has not been modified by artificial intelligence by comparing pixels of the optical image with a signature. A first image processing unit receives an optical image from a sensor and applies an algorithm to the pixels. The result is stored as a signature in a secure portion of memory. A flag indicating that the optical image has not been modified by artificial intelligence is also stored in a secure portion of memory. A second image processing unit receives the optical image and checks the flag. If the flag indicates that the optical image has not been modified, the second image processing unit determines whether the optical image has been maintained in a state that has not been modified by artificial intelligence by comparing pixels with the signature stored in the secure portion of memory.

Description

Background Technology

[0001] It is estimated that approximately 80% of the world's population owns a mobile phone, a type of mobile device. Mobile devices typically include a system-on-a-chip (SoC), which serves as the brain of the device. The SoC includes various components such as a central processor, memory, a graphics processor, an audio processor, high-speed I / O interfaces, and a cryptographic engine. Mobile devices also generally include one or more sensors configured to capture optical images. The captured optical images are then processed by one or more image processing units, which are often located on the SoC.

[0002] Optical images can be modified by artificial intelligence or the like, allowing various semantic objects to be added to or removed from the images. Some examples of semantic objects include people, animals, plants, buildings, terrain, and cars. For instance, artificial intelligence can insert a person into an optical image in a way that makes it visually impossible to determine whether the person was originally present in the image or was added after it was captured by a sensor. Not knowing whether an optical image is original or has been modified by artificial intelligence can be problematic.

[0003] This specification describes using hardware to determine whether an optical image has been modified by artificial intelligence, an unauthenticated image processing unit, etc. This specification describes in detail various exemplary embodiments regarding the nexus between an optical image and artificial intelligence. For example, this specification describes in detail a method in which an authenticated image processing unit compares pixels of an optical image with a signature to determine whether the optical image has been maintained in a state not modified by artificial intelligence. A first image processing unit receives an optical image from a sensor. The first image processing unit applies an algorithm, such as a hash algorithm or a checksum algorithm, to pixels of the received optical image, and the result of applying such an algorithm becomes the signature of the optical image. The first image processing unit stores the signature in a secure portion of memory. The first image processing unit may also set a flag to true in the secure portion of memory, and such true flag indicates that the optical image has not been modified by artificial intelligence.

[0004] This specification describes in detail a method in which a secure portion of memory may be accessed only by an authenticated image processing unit, wherein such authenticated image processing unit is an image processing unit associated with an indication that the image processing unit does not use artificial intelligence in modifying an optical image. The authenticated image processing unit is configured not to add semantic objects to the optical image or delete semantic objects from the optical image during optical processing. An unauthenticated image processing unit may be an image processing unit that modifies the optical image using artificial intelligence. The image processing unit is located on the system-on-chip (SoC) of the device and is used to determine whether the optical image remains unmodified by artificial intelligence, etc.

[0005] This specification describes the sending of an optical image to a second image processing unit. The second image processing unit first reads a flag stored in a secure portion of memory. If the flag indicates that the optical image has been maintained in a state not modified by artificial intelligence (i.e., the flag is set to true), the second image processing unit compares the optical image with a signature stored in the secure portion of memory. The second image processing unit may compare the optical image with a signature by applying an algorithm to the pixels of the optical image to generate a calculated signature and comparing the calculated signature with the signature stored by the previous image processing unit.

[0006] This specification describes in detail that if the result of applying the algorithm (i.e., the calculated signature) matches the stored signature, the optical image remains unmodified by artificial intelligence. Next, the second imaging processing unit applies processing to the optical image that potentially modifies the image. Since the second optical imaging unit is an authenticated image processing unit, it does not apply artificial intelligence to modify the optical image. Instead, the second image processing unit may process the optical image to change characteristics of the optical image such as hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. Next, the second optical imaging unit updates the signature of the optical image by reapplying the algorithm to the pixels of the modified optical image. The signature is updated because individual processing of the optical image by the second imaging unit may have changed the pixels of the optical image. Next, the updated signature is stored in a secure portion of memory.

[0007] The present specification describes that the next authenticated image processing unit may first check a flag stored in a secure portion of memory, and if the flag indicates that the optical image has been maintained in a state not modified by artificial intelligence, repeat the processing of comparing the optical image with a signature stored in the secure portion of memory. If the result of the comparison does not match the signature, the next image processing unit updates the flag stored in the secure portion of memory to indicate that the optical image is no longer authenticated as not modified by artificial intelligence (i.e., sets the flag to false).

[0008] This specification describes in detail that an image processing unit will apply processing to an optical image when the result of applying an algorithm matches a signature stored in a secure portion of memory. As previously mentioned, the image processing unit is an authenticated image processing unit that does not apply artificial intelligence to modify the optical image. Instead, the image processing unit may process the optical image to change characteristics such as hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. After processing the optical image, the image processing unit reapplies the algorithm to the pixels of the processed optical image to update the signature and then stores the updated signature in a secure portion of memory.

[0009] In exemplary embodiments, the device comprises a sensor configured to capture an optical image; and an SoC connected to the sensor, wherein the SoC includes memory on the SoC—the memory includes a security portion—; a first image processing unit on the SoC—the first image processing unit is connected to the sensor, and the first image processing unit is configured to generate a signature for an optical image captured by the sensor and store the signature in the security portion of the memory—; and a second image processing unit on the SoC—the second image processing unit is configured to receive the optical image from the first image processing unit and compare the optical image with the signature to determine whether the optical image has not been modified by artificial intelligence.

[0010] In exemplary embodiments, the method comprises: receiving an optical image from a sensor in a first image processing unit; generating a signature for the optical image; storing the signature in a secure portion of memory; sending the optical image to a second image processing unit; and determining, by the second image processing unit, whether the optical image has not been modified by artificial intelligence by comparing the signature with the optical image.

[0011] In exemplary embodiments, the system comprises: a sensor configured to capture an optical image; a system-on-chip (SoC) connected to the sensor; memory on the SoC—the memory includes a security portion—; a front-end image processing unit on the SoC—the front-end image processing unit is connected to the sensor, and the front-end image processing unit is configured to generate a signature for an optical image captured by the sensor, generate a flag indicating that the optical image has not been modified by artificial intelligence, and store the signature and flag in the security portion of the memory—; and an image processing pipeline on the SoC—the image processing pipeline is connected to the front-end image processing unit and includes a plurality of image processing units, each image processing unit of the image processing pipeline is configured to receive an optical image, read a flag, and compare the optical image with the signature to determine whether the optical image has not been modified by artificial intelligence. Brief explanation of the drawing

[0012] Devices and techniques for optical image processing are described with reference to the following drawings. Throughout the drawings, the same numbers are used to refer to similar features and components. FIG. 1 illustrates an exemplary device having an SoC including a processor or other circuitry capable of implementing optical image processing and at least one sensor. Figure 2 illustrates an exemplary flow path for optical image processing and optical image streaming after processing. Figure 3 illustrates an exemplary connection between an image processing unit and a security part of memory. Figure 4 illustrates an exemplary connection between an image processing unit and a security part of memory. FIG. 5 illustrates an exemplary device for processing optical images. FIGS. 5-1 (Fig. 5a) to FIG. 5-4 (Fig. 5d) illustrate an exemplary apparatus for processing an optical image, along with exemplary processing of the optical image. FIG. 6 illustrates an exemplary device for processing optical images. FIG. 7 is a flowchart illustrating an exemplary process for implementing optical image processing. FIG. 8 is a flowchart illustrating an exemplary process for implementing optical image processing. FIG. 9 is a flowchart illustrating an exemplary process for implementing optical image processing. Specific details for implementing the invention

[0013] outline

[0014] Optical images can be captured by mobile devices such as mobile phones. Due to the sheer number of mobile devices in use, the number of images available on the Internet is virtually unlimited. However, it can be difficult, if not impossible, for a user to visually determine whether an optical image is original or has been preserved without modification by artificial intelligence, machine learning, etc. For example, a user might view an optical image that appears to depict a family vacation in Yellowstone. However, the user may not be able to distinguish whether the family in the photo is the actual people who visited Yellowstone and were present in the original captured optical image, or if artificial intelligence added that family to an optical image of Yellowstone that was originally captured without any people. Artificial intelligence can modify optical images by adding or removing semantic objects, and these changes are not visually discernible. Users may not trust optical images if they cannot determine whether they are original and have been preserved without modification by artificial intelligence.

[0015] This specification describes hardware that may be used to determine whether an optical image has been maintained in a state that has not been modified by artificial intelligence, an unauthenticated image processing unit, etc. This specification describes in detail a method in which an authenticated image processing unit on an SoC of a mobile device compares pixels of an optical image with a previously generated signature to determine whether the optical image has been maintained in a state that has not been modified by artificial intelligence and / or an unauthenticated image processing unit. An optical image captured by a sensor of a mobile device is sent to a first image processing unit on an SoC. The first image processing unit may be a front-end image processing unit. As the first image processing unit receives the optical image directly from the sensor, the first image processing unit sets a flag to true to indicate that the optical image has not been modified by artificial intelligence. The flag is set within a secure portion of memory that can be accessed by the authenticated image processing unit of the mobile device. The first image processing unit also generates or creates a signature corresponding to the optical image based on the pixels of the optical image. For example, the first image processing unit can apply an algorithm to the pixels of an optical image, and the result of applying this algorithm can be a signature. Afterwards, this signature is stored in a secure part of memory.

[0016] This specification describes in detail how an image processing unit of an SoC can access both a flag and a signature stored in a secure portion of memory to determine whether an optical image remains unmodified by artificial intelligence or the like. The image processing unit is an authenticated image processing unit, which means that the image processing unit does not use artificial intelligence to modify the optical image. The authenticated image processing unit is configured not to add semantic objects to the optical image or delete semantic objects from the optical image during the optical processing of the optical image.

[0017] This specification describes the transmission of an optical image to a subsequent image processing unit on the SoC. When each image processing unit receives an optical image, the image processing unit first reads a flag stored in a secure portion of memory. If the flag indicates that the optical image has been maintained without being modified by artificial intelligence (i.e., the flag is set to true), the image processing unit then compares the optical image with a signature also stored in the secure portion of memory. For example, the image processing unit may compare the optical image with a signature by applying an algorithm to the pixels of the optical image and comparing the result with the signature.

[0018] This specification describes in detail that if the result matches the signature, the optical image remains unmodified by artificial intelligence. After comparing the pixels with the signature, the image processing unit then applies individual processing to the optical image, which may modify the image. Each optical imaging unit on the SoC is an authenticated image processing unit and therefore does not apply artificial intelligence to modify the optical image. Instead, the image processing unit may process the optical image to change characteristics such as hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. Processing of the optical image by the authenticated image processing unit may change the pixels of the optical image by comparing them with the pixels of the optical image initially received by the image processing unit. Accordingly, the optical imaging unit then generates an updated signature corresponding to the current version of the optical image, and the updated signature is stored in a secure portion of memory. The updated signature may be generated or created by applying an algorithm to the pixels of the optical image after the image processing unit has applied processing to the optical image.

[0019] This specification describes that this processing is repeated by each authenticated image processing unit located within the mobile device. If the comparison result does not match the signature, the image processing unit will update a flag stored in a secure part of memory to indicate that the optical image is no longer maintained in a state unmodified by artificial intelligence (i.e., the flag is set to false). A flag indicating that the optical image is maintained in a state unmodified by artificial intelligence may be included in the metadata for the optical image. Similarly, a flag indicating that the optical image is no longer maintained in a state unmodified by artificial intelligence may be included in the metadata for the optical image. In this way, the user can determine whether the optical image has been modified by artificial intelligence using the review metadata of the optical image. These and other embodiments are described in this specification.

[0020] Exemplary Environment and Electronic Device

[0021] FIG. 1 illustrates an exemplary environment (100) comprising a device (102), also referred to herein as a mobile device. The device (102) comprises an SoC (104) comprising at least a processor (106), memory (108), a front-end image processing unit (IPU) (110), an image processing pipeline (120), and an encoder / decoder (CODEC) (130). The device (102) comprises one or more sensors (140), comprising a front sensor and a rear sensor (140-2), configured to capture an optical image. The device (102) may implement instructions for comparing an optical image captured by one or more sensors (140) with a signature of an optical image using components of the SoC (104) to determine whether the optical image has not been modified by artificial intelligence. In embodiments, instructions for determining whether the optical image has not been modified by artificial intelligence are as described herein.

[0022] A front-end IPU (110) located on the SoC (104) is configured to receive an optical image from an optical sensor (140) and generate a signature corresponding to the optical image. The generated signature is then stored in a secure portion of memory (108). The front-end IPU (110) is also configured to set a flag in the secure portion of memory (108) indicating that the optical image has been kept unmodified by artificial intelligence. An image processing pipeline (120) located on the SoC (104) includes one or more IPUs configured to receive the optical image, check the flag in the secure portion of memory (108), compare the optical image with the signature stored in the secure portion of memory (108), and process the optical image.

[0023] In this example, the device (102) is depicted as a smartphone. However, the device (102) may be implemented as any suitable computing or other electronic device. Examples of the device (102) include a mobile electronic device or mobile device, a mobile communication device, a modem, a cellular or mobile phone, a mobile station, a gaming device, a navigation device, a media or entertainment device (e.g., a media streamer or a gaming controller), a laptop computer, a desktop computer, a tablet computer, a smart home appliance, a vehicle-based electronic system, a wearable computing device (e.g., clothing, a watch, or reality-altering glasses), an Internet of Things (IoT) device, a sensor, an inventory management device, an electronic part of a machine or equipment (e.g., a vehicle or a robot), a memory storage device (e.g., a solid-state drive (SSD)), a server computer or part thereof (e.g., a server blade or rack or other part of a data center), etc. Examples of the device (102) include a tablet device (102-1), a smart television (102-2), a desktop computer (102-3), a server computer (102-4), a smartwatch (102-5), a smartphone (or document reader) (102-6), and intelligent glasses (102-7).

[0024] In exemplary embodiments, the device (102) includes an SoC (104) which is the brain of the device (102). The SoC (104) is generally a complex chip that includes a number of different components, such as a processor, memory, a cryptographic engine, and high-speed I / O. The SoC (104) generally includes a single integrated circuit. The integrated circuit may be part of a chip, package, module, assembly, or at least one printed circuit board (PCB) (not shown), or may be realized as such a PCB. Examples of PCBs include flexible PCBs, rigid PCBs, single-layer or multi-layer PCBs, surface-mount or through-hole PCBs, combinations thereof, etc. One or more integrated circuit (IC) chips may be mounted on the PCB. Each IC chip may be realized as a general-purpose processor, a microcontroller, an application-specific IC (ASIC), etc. Other examples of IC chips include security-oriented IC chips, memory chips, communication IC chips (e.g., modems or radio frequency ICs), graphics processors, artificial intelligence (AI) accelerators, sensor chips, combinations thereof, etc. The sensor chip may include, for example, an accelerometer, a camera or other optical sensor, a satellite positioning system (e.g., a Global Positioning System (GPS)) chip, etc. The IC chip may be packaged alone or together with other IC chips. Although part of the disclosure refers to using techniques with an SoC, the invention is not limited thereto. For example, the memory (108), front-end IPU (110), and image processing pipeline (120) as described herein may be included in other circuits, such as any IC or chip, that access an optical image from the sensor (140) of the device (102).

[0025] Exemplary device, system, and mode of operation

[0026] FIG. 2 illustrates a system (200) that can be used to determine whether an optical image is maintained in a state that has not been modified by artificial intelligence. An optical image is captured by one or more sensors (140). The captured optical image is sent to a front-end IPU (110) on an SoC (104). The front-end IPU (110) is connected to one or more sensors (140) via a serial interface (250). The optical image is processed by the front-end IPU (110) and one or more IPUs within an image processing pipeline (120). The front-end IPU (110) and the IPUs within the image processing pipeline (120) are authorized IPUs, which means that each IPU is authorized to process the optical image but does not modify the optical image through artificial intelligence. The authorized IPU can process the optical image to refine the optical image, such as by changing the hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. of the optical image. A certified IPU is associated with an indication that image processing is performed without using artificial intelligence. A certified IPU neither adds semantic objects to optical images nor removes semantic objects from optical images. Examples of semantic objects include people, mountains, landscapes, backgrounds, cars, and animals.

[0027] The front-end IPU (110) and the image processing pipeline (120) determine whether the optical image remains unmodified by artificial intelligence as described in detail in this specification. Then, the optical image is sent to an image codec (130) to be encoded or decoded and compressed or decompressed. Then, the optical image may be included in a data stream (260).

[0028] The data stream (260) may be a video stream or an image stream. The data stream (260) of the optical image may include metadata indicating that the optical image has been maintained in a state not modified by artificial intelligence. For example, a user may right-click the optical image in a browser to determine whether the optical image has been maintained in a state not modified by artificial intelligence. Alternatively, the metadata may indicate that the optical image has no longer been maintained in a state not modified by artificial intelligence if determined by at least one of the front-end IPU (110) and the IPUs within the image processing pipeline (120).

[0029] Before including the optical image in the data stream (260), the codec (130) may use symmetric encryption to ensure that the user viewing the optical image can be assured that the optical image has been kept unmodified by artificial intelligence. The metadata of the optical image may include a private encryption key. The user of the data stream (260) may also use a public encryption key to determine whether a signature, such as a hash value, still matches the content of the optical image.

[0030] FIG. 3 illustrates an embodiment of a device (300) that can be used to determine whether an optical image has been maintained in a state not modified by artificial intelligence. The device (300) includes an image processing unit (110, 310). The image processing unit may be a front-end IPU (110) or an IPU (310) within an image processing pipeline (120). When an optical image is received from a sensor (140), the front-end IPU (110) sets a flag (374) in a secure portion (370) of memory. The flag (374) is set to true ("T") or false ("F"), where "T" indicates that the optical image has been maintained in a state not modified by artificial intelligence and "F" indicates that the optical image has no longer been maintained in a state not modified by artificial intelligence.

[0031] When an optical image is received from a sensor, the front-end IPU (110) also generates a signature (372) corresponding to the optical image. In one embodiment, the front-end IPU (110) applies an algorithm to the pixels of the optical image to generate the signature (372). For example, the front-end IPU (110) may apply a hash algorithm to the optical image, and the resulting hash value is the signature (372). Similarly, the front-end IPU (110) may apply a checksum algorithm to the pixels of the optical image, and the resulting checksum value is the signature (372). Then, the signature (372) is stored in a secure portion (370) of memory. The secure portion (370) of memory may be accessible only from the authenticated IPUs (110, 310) of the SoC (104) of the mobile device (102).

[0032] After the front-end IPU (110) receives the optical image, sets the flag (374), and generates the signature (372), the optical image is received by the IPU (310) within the image processing pipeline (120). Upon receiving the optical image, the IPU (310) will check the flag (374) stored in the secure portion (370) of memory. If the flag (374) is set to "T", indicating that the optical image has been kept unmodified by artificial intelligence, the IPU (310) will compare the signature (372) stored in the secure portion (370) of memory with the received optical image. For example, the IPU (310) may apply an algorithm to the pixels of the received optical image and compare the results with the signature (372). If the results match, the IPU (310) will know that the optical image has been kept unmodified by artificial intelligence. Since the signature (372) is generated by applying an algorithm to the pixels of the optical image, the pixel count of the optical image will remain the same in these instances, so the signature (372) can still be applied even if the optical image is rotated, locked, scaled, etc.

[0033] Then, the IPU (310) will apply individual processing to the optical image that modifies the image but does not use artificial intelligence. For example, the IPU (310) may refine the optical image by modifying hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. The IPU (310) does not add semantic objects to the optical image. Likewise, the IPU (310) does not remove semantic objects from the optical image. Then, the IPU (310) updates the signature (372) for the optical image as discussed herein and stores the updated signature (372) in the secure portion (370) of memory. For example, the IPU (310) may apply an algorithm to the pixels of the optical image and store the result as the signature (372) in the secure portion (370) of memory.

[0034] If the results of comparing the optical image with the signature (372) do not match, the IPU (310) will set the flag (374) stored in the secure part (370) of memory to “F”, which indicates that the optical image is no longer maintained in a state that has not been modified by artificial intelligence. Then, the IPU (310) will apply individual processing to the optical image to modify or refining the optical image without the use of artificial intelligence. For example, the IPU (310) may modify the hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. of the optical image. The IPU (310) does not add semantic objects to the optical image. Likewise, the IPU (310) does not delete semantic objects from the optical image. However, the IPU (310) does not update the signature (372) for the optical image because it has already determined that the optical image is no longer maintained in a state that has not been modified by artificial intelligence and has set the flag (374) to “F”. Any subsequent IPU (310) in the image processing pipeline (120) realizes that the optical image is no longer maintained in a state that has not been modified by artificial intelligence. Therefore, any subsequent IPU (310) in the image processing pipeline (120) will no longer compare the optical image with the signature (372) stored in the secure portion (370) of memory. Likewise, no subsequent IPU (310) in the image processing pipeline (120) will update the signature (372) after processing the optical image individually.

[0035] FIG. 4 illustrates an embodiment of a device (400) that can be used to determine whether an optical image is maintained in a state that has not been modified by artificial intelligence. The device (400) includes an image processing unit (110, 310). The image processing unit may be a front-end IPU (110) or an IPU (310) within an image processing pipeline (120). When an optical image is received from a sensor (140), the front-end IPU (110) sets a flag (374) and stores the flag (374) in a secure portion (370) of memory. When an optical image is received from a sensor, the front-end IPU (110) sets the flag (374) to T.

[0036] Then, the front-end IPU (110) generates or creates a signature (372) corresponding to the optical image. In one embodiment, the front-end IPU (110) applies an algorithm to the pixels of the optical image to generate the signature (372). Then, the signature (372) is stored in a secure portion (370) of memory. The secure portion (370) of memory may be accessible only from authenticated IPUs (110, 310) located on the SoC (104) of the mobile device (102).

[0037] The front-end IPU (110) also stores a key chain (476) in a secure portion (370) of memory. The key chain (476) indicates each IPU (110, 310) that processed the optical image. In this case, the optical image was processed only by the front-end IPU (110). The front-end IPU (110) also stores a pointer (478) in the secure portion (370) of memory. The pointer (478) indicates the location where the key chain (476) will be attached to include the identity of the subsequent IPU (310) that processed the optical image.

[0038] As discussed herein, upon receiving an optical image by the IPU (310) within the image processing pipeline (120), the IPU (310) will check a flag (374) stored in the secure portion (370) of memory. If the flag (374) is set to "T," indicating that the optical image has been kept unmodified by artificial intelligence, the IPU (310) will compare the received optical image with a signature (372) stored in the secure portion (370) of memory. For example, the IPU (310) may apply an algorithm to the pixels of the received optical image and compare the results with the signature (372). If the results match, the IPU (310) will know that the optical image has been kept unmodified by artificial intelligence. Then, as discussed herein, the IPU (310) will apply individual processing to the optical image that refines the optical image but does not use artificial intelligence.

[0039] Then, the IPU (310) updates the signature (372) for the optical image as discussed herein and stores the updated signature (372) in the secure portion (370) of memory. For example, the IPU (310) may apply an algorithm to the pixels of the optical image and store the result as the signature (372) in the secure portion (370) of memory. After processing the optical image, the IPU (310) attaches a key chain (476) to include the identity of the IPU (310) at a location specified by a pointer (478). Then, the updated key chain (476) will be stored in the secure portion (370) of memory. The IPU (310) will also update the pointer (478) to include a new location that a subsequent IPU (310) that processes the optical image must attach to the key chain (476). Then, the updated pointer (478) will be stored in the secure portion (370) of memory. A key chain (476) containing a list of each IPU (110, 25) that processed the optical image may be included in the metadata of the optical image.

[0040] FIG. 5 illustrates an exemplary system (500). The system (500) includes a front-end IPU (110), a secure portion of memory (370), and an image processing pipeline (120). The image processing pipeline (120) includes a first IPU (310-1), a second IPU (310-2), and a third IPU (310-3). The number of IPUs (310) in the image processing pipeline (120) may be more or less than the three IPUs (310-1, 310-2, 310-3) illustrated in FIG. 5 for exemplary purposes. The front-end IPU (110) receives optical images (502) from one or more sensors (140). The front-end IPU (110) is configured to receive optical images (502) from one or more sensors (140). For example, the front-end IPU (110) can be connected to the sensor (140) via a serial interface (250) as discussed in this specification.

[0041] FIGS. 5-1 through 5-4 illustrate exemplary operation scenarios using the hardware illustrated in FIG. 5. As illustrated in FIG. 5-1, the front-end IPU (110) generates a signature (372) upon receiving an optical image (502). As discussed herein, the front-end IPU (110) may apply an algorithm, such as a hash algorithm or a checksum algorithm, to the pixels of the optical image (502). The signature "S" (372) corresponds to the pixels of the optical image (502) as discussed herein. The value of the signature (372) is the result of the algorithm applied to the optical image (502).

[0042] When generating signature "S" (372), the front-end IPU (110) stores signature "S" (372) in a secure portion (370) of memory. Signature "S" (372) in the secure portion (370) of memory can only be accessed by the authenticated IPUs (110, 310) of the SoC (104). The front-end IPU (110) also sets a flag (374) in the secure portion (370) of memory to be true "T". A flag (374) set to "T" indicates that the optical image (502) has been maintained in a state that has not been modified by artificial intelligence, machine learning, etc. Because the front-end IPU (110) received the optical image (502) directly from the sensor (140), the front-end IPU (110) can set the flag (374) to "T".

[0043] The optical image (502) is sent to the first IPU (310-1) in the image processing pipeline (120) for processing the optical image (502). Upon receiving the optical image (502), the first IPU (310-1) checks the flag (374) in the secure portion (370) of the memory. If the flag (374) is set to "T", the first IPU (310-1) will compare the optical image (502) with the signature "S" (372) stored in the secure portion (370) of the memory. This is done by applying an algorithm, such as a hash algorithm or a checksum algorithm, to the pixels of the optical image (502) to generate a result (i.e., a calculated signature), and comparing the result with the signature "S" (372) stored in the secure portion (370) of the memory. If the result matches the signature "S" (372), the optical image (502) remains unmodified by artificial intelligence, and the first IPU (310-1) may proceed to apply processing to the optical image (502). As discussed herein, the first IPU (310-1) will apply processing that modifies or refines the optical image (502) but does not use artificial intelligence. For example, the first IPU (310-1) will not add semantic objects to the optical image (502) and will not delete semantic objects from the optical image (502). Instead, the processing applied to the optical image (502) by the first IPU (310-1) may refine the optical image (502) by modifying, for example, the hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. of the optical image (502).

[0044] Referring to FIG. 5-2 (Fig. 5b), the first IPU (310-1) in the image processing pipeline (120) will generate a signature "S1" (372-1) for the optical image (502). The signature "S1" (372-1) will be generated or created by applying an algorithm to the pixels of the optical image (502), and the result (i.e., the calculated signature) is the signature "S1" (372-1). The signature "S1" (372-1) of the optical image (502) may not be the same as the previous signature "S" (372) of the optical image (502) due to individual processing applied to the optical image (502) by the first IPU (310-1).

[0045] The first IPU (310-1) in the image processing pipeline (120) now stores the signature "S1" (372-1) in the current instance of the secure portion of memory (370-1). The past instance of the secure portion of memory (370) is shown with dashed lines for illustrative purposes. Since the result of comparing the optical image (502) with the signature "S" (372) indicates that the optical image (502) has been maintained in a state not modified by artificial intelligence, the first IPU (310-1) does not change the flag (374) in the secure portion of memory (370-1).

[0046] The optical image (502) is sent to a second IPU (310-2) within the image processing pipeline (120) for processing the optical image (502). Upon receiving the optical image (502), the second IPU (310-2) checks a flag (374) in the secure portion (370-1) of memory. If the flag (374) is set to "T", the second IPU (310-2) will compare the optical image (502) with a signature "S1" (372-1) also stored in the secure portion (370-1) of memory. This is done by applying an algorithm, such as a hash algorithm or a checksum algorithm, to the pixels of the optical image (502) and comparing the result (i.e., the calculated signature) with the signature "S1" (372-1) stored in the secure portion (370-1) of memory. If the result matches the signature "S1" (372-1), the optical image (502) remains unmodified by artificial intelligence, and the second IPU (310-2) may proceed to apply processing to the optical image (502). Likewise, if the result matches the signature "S1" (372-1), the optical image (502) remains unmodified by an intermediate non-authenticated IPU. As discussed herein, the second IPU (310-2) will apply processing that modifies the optical image (502) but does not use artificial intelligence. For example, the second IPU (310-2) will not add semantic objects to the optical image (502) and will not delete semantic objects from the optical image (502). Instead, the processing applied to the optical image (502) by the second IPU (310-2) can modify the characteristics of the optical image (502), such as modifying the hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. of the optical image (502).

[0047] Referring to FIG. 5-3 (Fig. 5c), the second IPU (310-2) in the image processing pipeline (120) will generate a signature "S2" (372-2) for the optical image (502). The signature "S2" (372-2) will be generated or created by applying an algorithm to the pixels of the optical image (502), and the result is the signature "S2" (372-2). The signature "S2" (372-2) of the optical image (502) may not be identical to the previous signature "S1" (372-1) of the optical image (502) due to individual processing of the optical image (502) by the second IPU (310-2).

[0048] The second IPU (310-2) in the image processing pipeline (120) now stores the signature "S2" (372-2) in the current instance of the secure portion of memory (370-2). Past instances of the secure portion of memory (370, 370-1) are shown with dashed lines for illustrative purposes. Since the result of comparing the optical image (502) with the signature "S1" (372-1) indicates that the optical image (502) has been maintained in a state not modified by artificial intelligence, the second IPU (310-2) does not change the flag (374) in the secure portion of memory (370-2).

[0049] The optical image (502) is sent to the third IPU (310-3) in the image processing pipeline (120) for processing the optical image (502). Upon receiving the optical image (502), the third IPU (310-3) checks the flag (374) in the secure portion (370-2) of memory. If the flag (374) is set to "T", the third IPU (310-3) will compare the optical image (502) with the signature "S2" (372-2) stored in the secure portion (370-2) of memory. This is done by applying an algorithm to the pixels of the optical image (502) and comparing the result (i.e., the calculated signature) with the signature "S2" (372-2) stored in the secure portion (370-2) of memory. If the result matches the signature "S2" (372-2), the optical image (502) remains unmodified, and the third IPU (310-3) may proceed to apply processing to the optical image (502). As discussed herein, the third IPU (310-3) will apply processing that modifies or refines the optical image (502) but does not use artificial intelligence. For example, the third IPU (310-3) will not add semantic objects to the optical image (502) nor will it delete semantic objects from the optical image (502). Instead, the processing applied to the optical image (502) by the third IPU (310-3) may modify the characteristics of the optical image (502), such as modifying the hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. of the optical image (502).

[0050] Referring to FIG. 5-4, the third IPU (310-3) in the image processing pipeline (120) will generate a signature "S3" (372-3) for the optical image (502). The signature "S3" (372-3) will be generated or created by applying an algorithm to the pixels of the optical image (502), and the result is the signature "S3" (372-3). The signature "S3" (372-3) of the optical image (502) may not be identical to the previous signature "S2" (372-2) of the optical image (502) due to individual processing of the optical image (502) by the third IPU (310-3).

[0051] The third IPU (310-3) in the image processing pipeline (120) now stores the signature "S3" (372-3) in the current instance of the secure portion of memory (370-3). Past instances of the secure portion of memory (370, 370-1, 370-2) are shown with dashed lines for illustrative purposes. Because the result of comparing the optical image (502) with the signature "S2" (372-2) indicates that the optical image (502) has not been modified by artificial intelligence, the third IPU (310-3) does not change the flag (374) in the secure portion of memory (370-3). The signature "S3" (372-3) can be used by a subsequent IPU (typically 310) in the image processing pipeline (120), if present, to confirm that the optical image (502) has been maintained in a state not modified by artificial intelligence.

[0052] FIG. 6 illustrates an exemplary system (600). The system (600) includes a front-end IPU (110), a secure portion of memory (370, 370-1, 370-2, 370-3), and an image processing pipeline (120). The image processing pipeline (120) includes a first IPU (310-1), a second IPU (310-2), and a third IPU (310-3). The number of IPUs (typically 310) within the image processing pipeline (120) may be more or less than the three IPUs (310-1, 310-2, 310-3) shown in FIG. 5 for exemplary purposes. The front-end IPU (110) receives an optical image (502) from one or more sensors (140). The system (600) illustrated in FIG. 6 represents the processing of an image modified by artificial intelligence at some point during image processing.

[0053] As discussed in this specification, the front-end IPU (110) generated a signature (372) by applying an algorithm to the pixels of the optical image (502) upon receiving the optical image (502). Upon generating the signature "S" (372), the front-end IPU (110) stored the signature "S" (372) in the secure portion (370) of memory and set the flag (374) to true "T", which is shown as a dashed line because it is not the current instance of the secure portion (370-3) of memory. The optical image (502) was sent to the first IPU (310-1) in the image processing pipeline (120) for processing. Upon receiving the optical image (502), the first IPU (310-1) checked the flag (374) in the secure portion (370) of memory. Since the flag (374) was set to "T", the first IPU (310-1) compared the optical image (502) with the signature "S" (372) stored in the secure portion (370) of the memory. As a result of the comparison, the pixels of the optical image (502) matched the signature "S" (372), and the first IPU (310-1) proceeded to apply processing to the optical image (502) to refining or modify the optical image (502) without using artificial intelligence.

[0054] After processing the optical image (502), the first IPU (310-1) in the image processing pipeline (120) generates a signature "S1" (372-1) for the optical image (502). As discussed herein, the signature "S1" (372-1) of the optical image (502) may not be identical to the previous signature "S" (372) of the optical image (502) due to individual processing applied to the optical image (502) by the first IPU (310-1). After generating the signature "S1" (372-1), the first IPU (310-1) stores the signature "S1" (372-1) in the secure portion (370-1) of memory, which is shown as a dotted line because it is not the current instance of the secure portion (370-3) of memory. Because the result of comparing the optical image (502) with the signature "S" (372) indicates that the optical image (502) remains unmodified by artificial intelligence, the first IPU (310-1) does not change the flag (374) in the security portion (370-1) of the memory.

[0055] After the signature "S1" (372-1) is stored in the secure portion (370-1) of memory, the optical image (502) is sent to the second IPU (310-2) in the image processing pipeline (120) for processing the optical image (502). Upon receiving the optical image (502), the second IPU (310-2) checks the flag (374) in the secure portion (370-1) of memory. Since the flag (374) in the secure portion (370-1) of memory is set to "T", the second IPU (310-2) compares the optical image (502) with the signature "S1" (372-1) also stored in the secure portion (370-1) of memory. This is done by applying an algorithm to the pixels of the received optical image (502) and comparing the result (i.e., the calculated signature) with the signature "S1" (372-1). In this case, the result of applying the algorithm to the pixels of the optical image (502) did not match the signature "S1"372-1), which indicates that the optical image (502) was not kept in a state where it was no longer modified by artificial intelligence.

[0056] In one embodiment, the image processing pipeline (120) may include an unauthenticated IPU. A user may request to modify an optical image using the unauthenticated IPU within the image processing pipeline (120). For example, the unauthenticated IPU may be used after processing of the optical image by the first IPU (310-1) and before processing of the optical image by the second IPU (310-2). The unauthenticated IPU may be used to remove semantic objects from the optical image or to add semantic objects to the optical image. In this case, the result of applying an algorithm to pixels of the optical image (502) that do not match the signature "S1" (372-1) may indicate that the optical image (502) has been modified by an intermediate unauthenticated IPU between the first IPU (310-1) and the second IPU (310-2). In one embodiment, the image processing pipeline (120) may include an unauthenticated IPU. The user may request to modify an optical image using an unauthenticated IPU.

[0057] Then, the second IPU (310-2) sets the flag (374) in the secure memory (370-2) to false “F”. The secure memory (370-2) is shown with a dotted line because it is not the current instance of the secure memory (370-3). Then, the second IPU (310-2) proceeds to apply individual processing to the optical image (502) to refining or modifying the optical image (502). As discussed herein, the second IPU (310-2) applied processing to modify the optical image (502) but without using artificial intelligence. After processing the optical image (502), the second IPU (310-2) did not generate an updated signature (372) because the previous comparison result indicated that the optical image (502) was no longer maintained in a state not modified by artificial intelligence.

[0058] The optical image (502) was sent to the third IPU (310-3) in the image processing pipeline (120) for processing the optical image (502). Upon receiving the optical image (502), the third IPU (310-3) checked the flag (374) in the security portion (370-2) of the memory. Because the flag (374) was set to "F", the third IPU (310-3) did not compare the optical image (502) with the updated signature (372). The flag (374) set to "F" in the security portion (370-2) of the memory indicated to the third IPU (310-3) that the optical image (502) was no longer kept in a state unmodified by artificial intelligence. Then, the third IPU (310-3) proceeded to apply processing to the optical image (502). As discussed in this specification, the third IPU (310-3) applies processing that modifies or refines the optical image (502) but does not use artificial intelligence.

[0059] After processing the optical image (502), the third IPU (310-3) in the image processing pipeline (120) does not generate an updated signature "S" (372) for the optical image (502). This is because the flag (374) has already indicated that the optical image (502) is no longer maintained in a state where it has not been modified by artificial intelligence. The flag (374) is maintained in a state set to "F" in the current instance of the secure portion of memory (370-3). Past instances of the secure portion of memory (370, 370-1, 370-2) are shown with dashed lines for illustrative purposes. The flag (374) set to "F" may be used by subsequent IPUs in the image processing pipeline (120) to indicate that the optical image (502) has been modified by artificial intelligence. This indication may also be used in the metadata of the optical image (502).

[0060] Exemplary methods

[0061] Exemplary methods are described below with reference to the flowcharts of FIGS. 7 through 9. Although exemplary method modalities will be described individually below, they may be implemented together in any combination or permutation.

[0062] FIG. 7 is a flowchart (700) illustrating an exemplary method for determining whether an optical image has been modified by artificial intelligence. The flowchart (700) includes five blocks (702-710). The operations of the exemplary processes may be performed by the electronic circuit components described herein. For example, the operations may be performed by at least one of a first image processing unit and a second image processing unit. The first image processing unit may be a front-end IPU (110), and the second image processing unit may be an IPU (310-1) within an image processing pipeline (120).

[0063] In 702, the first image processing unit receives an optical image (502) from the sensor (140). For example, the front-end IPU (110) may receive the optical image (502) from the sensor (140) via a serial interface (250). In 704, the first image processing unit generates a signature (372) for the optical image (502). In one embodiment, the first image processing unit may generate the signature (372) by applying an algorithm to the pixels of the optical image (502). In 706, the first image processing unit stores the signature (372) in a secure portion (370) of memory. In one example, the front-end IPU (110) may store the generated signature (372) in the secure portion (370) of memory.

[0064] In 708, the optical image (502) is sent to a second image processing unit. The optical image (502) may be received directly or indirectly from the first image processing unit at the second image processing unit. In one embodiment, the second image processing unit is an IPU (310) within an image processing pipeline (120). In 710, the second image processing unit (310) compares the signature (372) with the optical image (502) to determine whether the optical image (502) has not been modified by artificial intelligence. In one embodiment, the IPU (310-1) within the image processing pipeline (120) may compare the optical image (502) with the signature (372) generated by the front-end IPU (110).

[0065] FIG. 8 is a flowchart (800) illustrating an exemplary method for determining whether an optical image has been modified by artificial intelligence. The flowchart (800) includes eight blocks (802-816). The operations of the exemplary processes may be performed by the electronic circuit components described herein. For example, the operations may be performed by at least one of a first image processing unit and a second image processing unit. The first image processing unit may be a front-end IPU (110), and the second image processing unit may be an IPU (310-1) within an image processing pipeline (120).

[0066] In 802, the first image processing unit receives an optical image (502) from the sensor (140). For example, the front-end IPU (110) may receive the optical image (502) from the sensor (140) via a serial interface (250). In 804, the first image processing unit applies an algorithm to the pixels of the optical image (502). In one embodiment, the front-end IPU (110) applies a hash or checksum algorithm to the pixels of the optical image (502). In 806, the first image processing unit generates a signature (372) for the optical image (502). In one embodiment, the signature (372) is the value or result of the hash or checksum algorithm applied to the pixels by the front-end IPU (110). In 808, the first image processing unit stores the signature (372) in a secure portion (370) of memory. In one example, the front-end IPU (110) can store the generated signature (372) in a secure portion of memory (370).

[0067] In 810, the optical image (502) is sent to a second image processing unit. In one embodiment, the second image processing unit is an IPU (310-1) within an image processing pipeline (120). In 812, the second image processing unit determines whether the optical image (502) has not been modified by artificial intelligence by applying an algorithm to the pixels of the optical image (502) and comparing the signature (372) with the optical image (502). In one embodiment, the IPU (310-1) within the image processing pipeline (120) applies a checksum or hash algorithm to the pixels of the optical image (502) and compares the result with the signature (372) generated by the front-end IPU (110).

[0068] In 814, the second image processing unit processes the optical image (502) to modify the optical image (502) without adding semantic objects to the optical image (502) and without deleting semantic objects from the optical image (502). In one embodiment, the IPU (310-1) of the image processing pipeline (120) applies processing to the optical image (502) to modify hue, brightness, contrast, saturation, scale, tone, resolution, tint, etc. In 816, the second image processing unit updates the signature (372) to include the processing of the optical image (502). In one embodiment, the processing of the optical image (502) by the IPU (310-1) modified pixels of the optical image (502). The IPU (310-1) updates the signature (372) by applying an algorithm to the modified pixels of the optical image (502).

[0069] FIG. 9 is a flowchart (900) illustrating an exemplary method for determining whether an optical image has been modified by artificial intelligence. The flowchart (900) includes four blocks (902-908). The operations of the exemplary processes may be performed by the electronic circuit components described herein. For example, the operations may be performed by at least one of a first image processing unit and a second image processing unit. The first image processing unit may be a front-end IPU (110), and the second image processing unit may be an IPU (310-1) within an image processing pipeline (120).

[0070] In 902, the first image processing unit generates a flag (374) for the optical image (502) and sets the flag (374) to true “T”, which indicates that the optical image (502) has not been modified by artificial intelligence. For example, the front-end IPU (110) may generate a flag (374) for the optical image (502) from the sensor (140). In 904, the first image processing unit stores the flag (374) in a secure portion (370) of memory. In one embodiment, the front-end IPU (110) stores the flag (374) in a secure portion (370) of memory.

[0071] In 906, the second image processing unit examines the flag (374) before comparing the signature (372) with the optical image (502). In one embodiment, the IPU (310-1) in the image processing pipeline (120) examines the flag (374) in the secure portion (370) of memory before comparing the signature (372) with the optical image (502). In 908, the second image processing unit updates the flag (374) to false “F” if the result of comparing the signature (372) with the optical image (502) determines that the optical image (502) is no longer modified by artificial intelligence. In one example, the IPU (310-1) in the image processing pipeline (120) applies an algorithm to the pixels of the optical image (502) and compares the result with the signature (372). If the result and the signature (372) do not match, the IPU (310-1) updates the flag (374) to "F" to indicate that the optical image (502) is no longer modified by artificial intelligence.

[0072] With respect to the methods and associated flowchart(s) and flow diagram(s) described herein, the order in which operations are illustrated and / or described is not intended to be interpreted as a limitation. Instead, any number or combination of the described method operations may be combined in any order to implement a given method or alternative method, including combining the operations of the flowchart or flow diagram with the previously described modes and techniques in one or more ways. Operations may also be omitted or added from the described methods. Additionally, the described operations may be implemented in a manner that is completely or partially overlapping.

[0073] Exemplary aspects and embodiments of optical image processing

[0074] In the following, some exemplary modes and embodiments are described:

[0075] Exemplary embodiment 1. The device comprises a sensor configured to capture an optical image; and a system-on-chip (SoC) connected to the sensor, wherein the SoC includes a memory comprising a security portion; a first image processing unit connected to the sensor—the first image processing unit is configured to generate a signature for an optical image captured by the sensor and store the signature in the security portion of the memory—; and a second image processing unit—the image processing unit is configured to receive an optical image from the first image processing unit and compare the optical image with the signature to determine whether the optical image has not been modified by artificial intelligence.

[0076] Exemplary embodiment 2. As an apparatus of Exemplary embodiment 1, the optical image comprises pixels, and the first image processing unit is configured to generate a signature by applying an algorithm to the pixels of the optical image.

[0077] Exemplary modality 3. As an apparatus of Exemplary modality 1 or Exemplary modality 2, the algorithm comprises a hash algorithm, and the signature comprises a hash value.

[0078] Exemplary embodiment 4. As an apparatus of any one of exemplary embodiments 1 to 3, the second image processing unit is configured to apply a hash algorithm to an optical image received from the first image processing unit to determine a calculated hash value, and to compare the calculated hash value with a signature to determine whether the optical image has not been modified by artificial intelligence.

[0079] Exemplary embodiment 5. An apparatus of any one of exemplary embodiments 2 to 4, wherein the algorithm includes a checksum algorithm and the signature includes a checksum value.

[0080] Exemplary embodiment 6. As an apparatus of any one of exemplary embodiments 2 to 5, the second image processing unit is configured to apply a checksum algorithm to an optical image received from the first image processing unit to determine a calculated checksum value, and to compare the calculated checksum value with a signature to determine whether the optical image has not been modified by artificial intelligence.

[0081] Exemplary embodiment 7. An apparatus of any one of exemplary embodiments 2 to 6, wherein the first image processing unit includes a front-end image processing unit, and the second image processing unit includes one of a plurality of image processing units within the image processing pipeline of the SoC.

[0082] Exemplary embodiment 8. As an apparatus of any one of exemplary embodiments 2 to 7, a plurality of image processing units within a front-end image processing unit and an image processing pipeline include an image processing unit associated with an indication that image processing is performed without using artificial intelligence.

[0083] Exemplary embodiment 9. As an apparatus of any one of exemplary embodiments 2 to 8, each image processing unit in the image processing pipeline is configured to receive an optical image from a previous image processing unit, apply an algorithm to pixels of the optical image to generate a calculated signature, and compare the calculated signature with a signature stored by the previous image processing unit to determine whether the optical image has not been modified by artificial intelligence.

[0084] Exemplary embodiment 10. As an apparatus of any one of exemplary embodiments 2 to 9, each image processing unit in the image processing pipeline is configured to apply individual processing to modify an optical image and to update a signature in a secure portion of memory to reflect the modification by the individual processing applied to the optical image.

[0085] Exemplary embodiment 11. As an apparatus of any one of exemplary embodiments 2 to 10, individual processing applied to an optical image modifies the hue, brightness, contrast, saturation, scale, tone, resolution, or tint of the optical image.

[0086] Exemplary embodiment 12. As an apparatus of any one of exemplary embodiments 2 to 11, the individual processing of each image processing unit within the image processing pipeline is configured to modify the optical image without adding semantic objects to the optical image and without deleting semantic objects from the optical image.

[0087] Exemplary embodiment 13. As an apparatus of any one of exemplary embodiments 2 to 12, the front-end image processing unit is configured to generate a flag for an optical image and store the flag in a secure portion of memory.

[0088] Exemplary embodiment 14. As an apparatus of any one of exemplary embodiments 2 to 13, the front-end image processing unit is configured to set a flag to true for an optical image received from a sensor.

[0089] Exemplary embodiment 15. As a device of any one of exemplary embodiments 2 to 14, the front-end image processing unit is connected to the sensor via a serial interface.

[0090] Exemplary embodiment 16. As an apparatus of any one of exemplary embodiments 2 to 15, each image processing unit in the image processing pipeline is configured to examine a flag upon receiving an optical image.

[0091] Exemplary embodiment 17. As an apparatus of any one of exemplary embodiments 2 to 16, when a flag is set to false, each image processing unit in the image processing pipeline is configured to omit applying an algorithm to pixels of an optical image and comparing signatures of optical images.

[0092] Exemplary embodiment 18. As an apparatus of any one of exemplary embodiments 2 to 17, each image processing unit in the image processing pipeline is configured to set a flag to false in response to the result of applying an algorithm to a pixel not matching a signature.

[0093] Exemplary embodiment 19. As an apparatus of any one of exemplary embodiments 2 to 18, the front-end image processing unit is configured to write a key chain to a secure portion of memory and write a pointer to a secure portion of memory, and the pointer indicates a location where a subsequent image processing unit will attach the key chain.

[0094] Exemplary embodiment 20. As an apparatus of any one of exemplary embodiments 2 to 19, the key chain indicates each individual image processing unit that processes an optical image.

[0095] Exemplary embodiment 21. As an apparatus of any one of exemplary embodiments 2 to 20, each image processing unit in the image processing pipeline is configured to attach the identification of each image processing unit to a key chain after individual processing is applied to an optical image by each image processing unit.

[0096] Exemplary embodiment 22. As an apparatus of any one of exemplary embodiments 2 to 21, each image processing unit in the image processing pipeline is configured to update a pointer in a secure portion of memory and attach the identity of the image processing unit to a key chain.

[0097] Exemplary embodiment 23. The method comprises the steps of: receiving an optical image from a sensor in a first image processing unit; generating a signature for the optical image; storing the signature in a secure portion of memory; sending the optical image to a second image processing unit; and determining whether the optical image has not been modified by artificial intelligence by comparing the signature with the optical image by the second image processing unit.

[0098] Exemplary embodiment 24. As a method of exemplary embodiment 23, the step of generating a signature for an optical image further comprises the step of applying an algorithm to pixels of the optical image, and the step of comparing the signature with the optical image further comprises the step of generating a calculated signature by applying an algorithm to pixels of the optical image and the step of comparing the calculated signature with a stored signature.

[0099] Exemplary embodiment 25. As a method of Exemplary embodiment 23 or Exemplary embodiment 24, the method further comprises the step of processing an optical image by a second image processing unit—the processing of the optical image includes modifying the optical image without adding semantic objects to the optical image and without deleting semantic objects from the optical image—and the step of updating a signature stored in a secure portion of memory to take into account the image change resulting from the processing of the optical image by the second image processing unit.

[0100] Exemplary embodiment 26. As a method of any one of exemplary embodiments 23 to 25, the first image processing unit further comprises the steps of generating a flag for an optical image, setting the flag to true to indicate that the optical image has not been modified by artificial intelligence; and storing the flag in a secure portion of memory.

[0101] Exemplary embodiment 27. As a method of any one of exemplary embodiments 23 to 26, the method further comprises the step of reviewing a flag before comparing a signature with an optical image by a second image processing unit; and the step of updating the flag to false by the second image processing unit if, as a result of comparing the signature with the optical image, it cannot be determined that the optical image has not been modified by artificial intelligence.

[0102] Exemplary embodiment 28. As a method of any one of exemplary embodiments 23 to 27, the method further includes the step of sending an optical image to a third image processing unit.

[0103] Exemplary embodiment 29. Any one of exemplary embodiments 23 to 28 further comprises the step of examining a flag by a third image processing unit, and if the flag is false, further comprises the step of processing an optical image and sending the optical image to a fourth image processing unit.

[0104] Exemplary embodiment 30. As a method of any one of exemplary embodiments 23 to 29, the method further comprises the step of examining a flag by a third image processing unit, and if the flag is true, the method further comprises the step of determining whether the optical image has not been modified by artificial intelligence by applying an algorithm to a pixel of the optical image and comparing the result of applying the algorithm with a signature.

[0105] Exemplary embodiment 31. As a method of any one of exemplary embodiments 23 to 30, if it is determined that the optical image is no longer modified by artificial intelligence, the method further comprises the step of updating a flag to false and processing the optical image by a third image processing unit.

[0106] Exemplary embodiment 32. As a method of any one of exemplary embodiments 23 to 31, where it is determined that the optical image has not been modified by artificial intelligence, the method further comprises the steps of: processing the optical image by a third image processing unit; updating a signature to include processing of the optical image by the third image processing unit; and storing the updated signature in a secure portion of memory.

[0107] Exemplary embodiment 33. As a method of any one of exemplary embodiments 23 to 32, the method further comprises the step of writing a key chain to a secure portion of memory—the key chain indicates each image processing unit that processed the optical image—; and the step of writing a pointer to a secure portion of memory—the pointer indicates a location to subsequently attach the key chain.

[0108] Exemplary embodiment 34. As a method of any one of exemplary embodiments 23 to 33, the method further comprises the step of attaching a key chain to include the identification of an image processing unit that processed the optical image after processing is applied to the optical image; and the step of updating a pointer whenever the key chain is attached.

[0109] Exemplary embodiment 35. The system comprises: a sensor configured to capture an optical image; a system-on-chip (SoC) connected to the sensor; memory on the SoC - the memory includes a security portion -; a front-end image processing unit on the SoC - the front-end image processing unit is connected to the sensor, and the front-end image processing unit is configured to generate a signature for an optical image captured by the sensor, generate a flag indicating that the optical image has not been modified by artificial intelligence, and store the signature and flag in the security portion of the memory -; and an image processing pipeline on the SoC - the image processing pipeline is connected to the front-end image processing unit and includes a plurality of image processing units, each image processing unit of the image processing pipeline is configured to receive an optical image, read a flag, and compare the optical image with the signature to determine whether the optical image has not been modified by artificial intelligence -.

[0110] Exemplary embodiment 36. As a system of exemplary embodiment 35, further comprises an image codec connected to an image processing pipeline, wherein the image codec is configured to generate a data stream containing an optical image.

[0111] Exemplary embodiment 37. As a system of exemplary embodiment 35 or exemplary embodiment 36, the data stream further includes metadata, the metadata is associated with an optical image, and the metadata indicates that the optical image has not been modified by artificial intelligence.

[0112] Exemplary embodiment 38. As a system of any one of exemplary embodiments 35 to 37, a front-end image processing unit is configured to generate a signature by applying an algorithm to pixels of an optical image, and each image processing unit of an image processing pipeline is configured to apply an algorithm to pixels of an optical image and compare the result of the application of the algorithm with the signature to determine whether the optical image has not been modified by artificial intelligence.

[0113] Exemplary embodiment 39. As a system of any one of exemplary embodiments 35 to 38, each image processing unit of the image processing pipeline is configured to apply individual processing to an optical image and update a signature to include the individual processing applied to the optical image.

[0114] Exemplary embodiment 40. As a system of any one of exemplary embodiments 35 to 39, each image processing unit of the front-end image processing unit and the image processing pipeline is associated with an indication that image processing is performed without using artificial intelligence.

[0115] Unless otherwise specified in the context, the use of the word “or” in this specification may be construed as the use of a term that includes or applies to one or more items connected by the word “or” (e.g., the phrase “A or B” may be interpreted as allowing only “A,” allowing only “B,” or allowing both “A” and “B)). Additionally, the phrase “at least one” in a list of items used in this specification means any combination of such items that includes a single item. For example, “at least one of a, b or c” may include a, b, c, ab, ac, bc and abc, as well as any combination of multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc and ccc or any other order of a, b, c). Additionally, the items expressed in the accompanying drawings and terms discussed in this specification may represent one or more items or terms, and thus may be referenced interchangeably with single or multiple forms of items and terms in the descriptions provided in this specification.

[0116] Although embodiments for optical image processing have been described in language specific to specific features and / or methods, the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, specific features and methods are disclosed as exemplary embodiments for optical image processing.

Claims

Claim 1 A device comprising: a sensor configured to capture an optical image; a system-on-chip (SoC) connected to the sensor, wherein the SoC is Memory including a security portion; A first image processing unit connected to a sensor - the first image processing unit is, Generate a signature for the optical image captured by the sensor; and Configured to store a signature in the secure part of memory -; and Second image processing unit - The second image processing unit is, Receive an optical image from a first image processing unit; and A device comprising a signature configured to compare an optical image with a signature to determine whether the optical image has not been modified by artificial intelligence. Claim 2 A device according to claim 1, wherein the optical image includes pixels; and the first image processing unit is configured to generate a signature by applying an algorithm to the pixels of the optical image. Claim 3 In paragraph 2, the algorithm comprises a hash algorithm or a checksum algorithm; and the signature comprises a hash value or a checksum value, a device. Claim 4 In paragraph 3, the device is configured such that the second image processing unit applies a hash algorithm to an optical image received from the first image processing unit to determine a calculated hash value and compares the calculated hash value with a signature to determine whether the optical image has not been modified by artificial intelligence; or applies a checksum algorithm to an optical image received from the first image processing unit to determine a calculated checksum value and compares the calculated checksum value with a signature to determine whether the optical image has not been modified by artificial intelligence. Claim 5 A device according to claim 2, 3, or 4, wherein the first image processing unit comprises a front-end image processing unit; and the second image processing unit comprises one of a plurality of image processing units within the image processing pipeline of the SoC. Claim 6 In paragraph 5, the device comprises a plurality of image processing units within the front-end image processing unit and image processing pipeline, the image processing unit being associated with an indication that image processing is performed without using artificial intelligence. Claim 7 A device according to claim 5 or 6, wherein each image processing unit in the image processing pipeline receives an optical image from a previous image processing unit; generates a calculated signature by applying an algorithm to pixels of the optical image; and determines whether the optical image has not been modified by artificial intelligence by comparing the calculated signature with a signature stored by the previous image processing unit. Claim 8 A device according to claim 7, wherein each image processing unit in the image processing pipeline is configured to apply individual processing to modify an optical image; and to update a signature in a secure portion of memory to reflect the modification by the individual processing applied to the optical image, and the individual processing of each image processing unit in the image processing pipeline is configured to modify the optical image without adding semantic objects to the optical image and without deleting semantic objects from the optical image. Claim 9 A device according to any one of claims 5 through 8, wherein the front-end image processing unit is configured to generate a flag for an optical image; store the flag in a secure portion of memory; and set the flag to true for an optical image received from a sensor. Claim 10 A device according to claim 9, wherein each image processing unit in the image processing pipeline examines a flag upon receiving an optical image; and if the flag is set to false, each image processing unit in the image processing pipeline is configured to omit applying an algorithm to a pixel of the optical image and comparing the signature of the optical image. Claim 11 A device according to claim 9 or 10, wherein each image processing unit within the image processing pipeline is configured to set a flag to false in response to the result of applying an algorithm to a pixel not matching a signature. Claim 12 A method comprising: receiving an optical image from a sensor in a first image processing unit; generating a signature for the optical image; storing the signature in a secure portion of memory; sending the optical image to a second image processing unit; and determining, by the second image processing unit, whether the optical image has not been modified by artificial intelligence by comparing the signature with the optical image. Claim 13 In claim 12, the step of generating a signature for the optical image further comprises the step of applying an algorithm to pixels of the optical image, and the step of comparing the signature with the optical image further comprises the step of generating a calculated signature by applying an algorithm to pixels of the optical image and the step of comparing the calculated signature with a stored signature. Claim 14 A method according to claim 12 or 13, further comprising the step of processing an optical image by a second image processing unit—the processing of the optical image includes modifying the optical image without adding semantic objects to the optical image and without deleting semantic objects from the optical image—; and the step of updating a signature stored in a secure portion of memory to take into account the image change resulting from the processing of the optical image by the second image processing unit. Claim 15 A method according to claim 12, 13, or 14, further comprising: generating a flag for an optical image by a first image processing unit; setting the flag to true to indicate that the optical image has not been modified by artificial intelligence; storing the flag in a secure portion of memory; reviewing the flag by a second image processing unit before comparing a signature with the optical image; and updating the flag to false by the second image processing unit if, as a result of comparing the signature with the optical image, it cannot be determined that the optical image has not been modified by artificial intelligence.

Images