Securing the content processing pipeline

Abstract

A system for protecting a content processing pipeline, the system comprising a hardware processor and a memory storing software code. The hardware processor executes the software code to insert a synthetic test image configured to activate one or more neurons of a malicious neural network into a content stream, provide the content stream as an input stream to a first processing node of the pipeline, and receive an output stream including a processed test image. The hardware processor further executes the software code to compare the processed test image at the output with an expected image corresponding to the synthetic test image. And when the processed test image at the output matches the expected image, verifying that at least a portion of the pipeline is secure.

Description

Background Art

[0001] Advances in machine learning have enabled the production of realistic but fabricated representations of a person's image or voice, known as "deepfakes" because they are created using deep artificial neural networks. Deepfakes may be produced without the permission of the person whose image or voice is being used, and may make the person being represented appear to be saying or doing things they are not actually saying or doing. As a result, digital content manipulated by deepfakes can be used maliciously to spread misinformation.

[0002] Due to the widespread popularity of digital content for the distribution of entertainment and news, ensuring data security of the processing pipelines used for digital content is of paramount importance to its creators, owners, and distributors. However, as machine learning models continue to improve, the detection of malware capable of crafting deepfakes and introducing them into one or more processing nodes of the content processing pipeline will continue to be challenging. Therefore, in the absence of a robust and reliable solution for assessing the security of content processing pipelines, subtly manipulated or even substantially fake digital content may be inadvertently broadcast or distributed in violation of contractual agreements or regulatory restrictions, exposing content owners and / or distributors to potential legal risks. Summary of the invention

[0003] Systems and methods are provided for protecting a content processing pipeline, substantially as shown and / or described in conjunction with at least one of the accompanying drawings, and as more fully set forth in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 A schematic diagram illustrating an exemplary system for protecting a content processing pipeline according to one embodiment is shown;

[0005] Figure 2 An exemplary synthetic test image suitable for evaluating the security of a content processing pipeline according to one embodiment is shown;

[0006] Figure 3 According to one embodiment, a Figure 1 Exemplary software code used by the illustrated system; and

[0007] Figure 4 A flow chart representing an exemplary method for protecting a content processing pipeline is shown according to one embodiment. DETAILED DESCRIPTION

[0008] The following description contains specific information related to the embodiments in the present disclosure. Those skilled in the art will recognize that the present disclosure can be implemented in a manner different from that specifically discussed herein. The drawings in this application and the specific embodiments attached thereto are only for exemplary embodiments. Unless otherwise specified, the same or corresponding elements in the drawings will be represented by the same or corresponding reference numerals. In addition, the drawings and illustrations of the present application are usually not drawn to scale, and are not intended to correspond to actual relative sizes.

[0009] The present application discloses a system and method for protecting a content processing pipeline, which overcomes the defects and deficiencies of conventional technologies. By inserting a synthetic test image configured to activate one or more neurons of a malicious neural network into a content stream provided as an input stream to a content processing pipeline, the present security solution advantageously enables detection when a malicious neural network is present in the pipeline. In addition, by comparing a synthetic test image included in an output stream received from one or more processing nodes in the pipeline with an expected image corresponding to the synthetic test image, the present solution enables identification of one or more nodes in the pipeline infected by a malicious neural network.

[0010] Note that in some embodiments, the present security solution may be performed by a substantially automated system as a substantially automated process. It should also be noted that, as used in this application, the terms "automatic," "automated," and "automatic" refer to systems or processes that do not require the involvement of a human user (e.g., a system administrator). Although in some embodiments, a human system operator or administrator may review the security assessments generated by the automated system and according to the automated methods described herein, human involvement is optional. Thus, the methods described in this application may be performed under the control of the hardware processing components of the disclosed automated system.

[0011] It should also be noted that as defined in this application, a neural network (NN), also known as an artificial neural network (ANN), is a machine learning framework in which patterns in observed data, or learned representations of data, are processed through highly connected computational layers that map the relationship between inputs and outputs. In the context of deep learning, a "deep neural network" may refer to a neural network that utilizes multiple hidden layers between the input and output layers, which enables it to learn based on features that are not explicitly defined in the original data. Therefore, various forms of neural networks can be used to make predictions about new data based on past samples or "training data." In various embodiments, neural networks can be used to perform image processing or natural language processing.

[0012] Figure 1 FIG. 1 is a schematic diagram of an exemplary system for protecting a content processing pipeline according to one embodiment. Figure 1As shown, system 100 includes a computing platform 102 having a hardware processor 104, a system memory 106 implemented as a non-transitory storage device, and an optional display 108. According to the exemplary embodiment, system memory 106 stores software code 110 including an optional NN 112, a content library 144 providing content streams 128, and an image database 126 including one or more synthetic test images 124 (hereinafter referred to as "test images 124").

[0013] Note that Figure 3 As shown and described below, some software code 110 includes an optional NN 112 ( Figure 3 In some embodiments, NN 112 may be implemented as a generative adversarial network (GAN) that includes a discriminator configured to model a malicious neural network (hereinafter referred to as a "malicious NN"), such as a deep fake NN. In those embodiments, NN 112 may be used to generate a synthetic test image 124, which may then be stored on and obtained from an image database 126.

[0014] like Figure 1 As further shown in FIG. 1 , the system 100 is implemented in a use environment that includes a training platform 140, a content processing pipeline 150 (including a first processing node 152a, an intermediate processing node 152b, and a final processing node 152c), a communication network 120, and a system administrator or other user 132 (hereinafter referred to as "user 132") using a user system 130 including a display 138. In addition, Figure 1 A network communication link 122 is shown that communicatively couples the training platform 140 , the content processing pipeline 150 , and the user system 130 with the system 100 via the communication network 120 .

[0015] also, Figure 1 An input stream 148 and a processed content stream are shown, wherein the input stream 148 is to a first processing node 152a of a content processing pipeline 150, and the processed content stream is received by the system 100 as output streams 154, 156, 158 from the first processing node 152a, the intermediate processing node 152b, and the final processing node 152c, respectively. Figure 1 Also shown is a security assessment 164 of at least some portions of the content processing pipeline 150 generated by the system 100 .

[0016] although Figure 1Content processing pipeline 150 is depicted as including three processing nodes represented by first processing node 152a, intermediate processing node 152b, and final processing node 152c, but this representation is merely exemplary. In other embodiments, content processing pipeline 150 may include as few as one processing node, or more, or more than three processing nodes. Thus, in some embodiments, any one of first processing node 152a, intermediate processing node 152b, and final processing node 152c may be the first and final processing node of content processing pipeline 150, while in other embodiments, intermediate processing node 152b may include multiple processing nodes linking first processing node 152a and final processing node 152c.

[0017] Note that first processing node 152a, intermediate processing node 152b, and final processing node 152c may be implemented as any of a variety of image processing devices. That is, one or more of first processing node 152a, intermediate processing node 152b, and final processing node 152c may take the form of a digital camera, a video editing workstation, a desktop, a laptop or a tablet computer, a smart phone, or a cloud-based data storage service device, to name a few.

[0018] The software code 110, when executed by the hardware processor 104 of the computing platform 102, is configured to generate a security assessment 164 of at least a portion of the content processing pipeline 150 using the synthetic test image 124, as described in more detail below. Although for conceptual clarity, the present application refers to the software code 110, the image database 126, and the content library 144 as being stored in the system memory 106, more generally, the system memory 106 may take the form of any computer-readable non-transitory storage medium.

[0019] As used in this application, the expression "computer-readable non-transitory storage medium" refers to any medium, excluding carrier waves or other temporary signals that provide instructions to the hardware processor 104 of the computing platform 102. Therefore, computer-readable non-transitory storage media can correspond to various types of media, for example, volatile media and non-volatile media. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM); non-volatile memory may include optical, magnetic, and electrostatic storage devices. Common forms of computer-readable non-transitory storage media include, for example, optical disks, random access memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and flash memory.

[0020] In addition, despite Figure 1The training platform 140 and the user 130 are depicted as computer platforms remote from the system 100, but this representation is merely exemplary. More generally, the system 100 may include one or more computing platforms (e.g., computer servers) that may form an interconnected but distributed system (e.g., a cloud-based system). Thus, the hardware processor 104 and the system memory 106 may correspond to distributed processor and memory resources within the system 100, and the training platform 140 may be a component of the system 100, or may be implemented as a software module stored in the system memory 106. In addition, in some embodiments, the user system 130 may be included as an element of the system 100.

[0021] In one embodiment, the computing platform 102 of the system 100 may correspond to one or more web servers that are accessible via a packet-switched network (e.g., the Internet). Alternatively, the computing platform 102 may correspond to one or more computer servers that support a wide area network (WAN), or a local area network (LAN), or are included in another restricted distributed or dedicated network.

[0022] Note that although the user system 130 is Figure 1 1 is shown as a desktop computer, but this representation is merely exemplary. More generally, user system 130 may be any suitable system that implements data processing capabilities sufficient to provide a user interface, support connection to communication network 120, and implement the functions attributed to user system 130 herein. For example, in other embodiments, user system 130 may take the form of, for example, a laptop computer, a tablet computer, or a smart phone.

[0023] It should also be noted that in various embodiments, the display 108 may be physically integrated with the computing platform 102, or may be communicatively coupled to the computing platform 102 but physically separate. In addition, the display 138 of the user system 130 may be physically integrated with the user system 130, or may be communicatively coupled to the user system 130 but physically separate. For example, where the user system 130 is implemented as a smartphone, a laptop, or a tablet computer, the display 138 is typically integrated with the user system 130. In contrast, where the user system 130 is implemented as a desktop computer, the display 138 may take the form of a display that is separate from the user system 130 in the form of a stand-up computer. The display 108 of the system 100 and the display 138 of the user system 130 may be implemented as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or other suitable display screen that performs a physical conversion of signals to light.

[0024] Figure 2 An exemplary synthetic test image 224 suitable for evaluating the security of a content processing pipeline is shown according to one embodiment. Figure 2 As shown, synthetic test image 224 is a "meaningless" image that is indistinguishable to a human observer. However, synthetic test image 224 is configured to be mistaken for a recognizable object by a malicious NN that is fully trained using popular and widely available training datasets and architectures, or fine-tuned on a neural network that originally used these datasets or architectures, thereby causing the malicious NN to "generate hallucinations" of objects that are not actually depicted in synthetic test image 224. Note that, as defined for the purposes of this application, a malicious NN is characterized as "generating hallucinations" when the scores of one or more neurons within the malicious NN are substantially maximized in response to synthetic test image 224.

[0025] For example, synthetic test image 224a is configured to cause the malicious NN to recognize the pattern depicted in the image as a tree, even though it would not be discernible as any particular object to a human observer. Similarly, for example, synthetic test image 224b is configured to cause the malicious NN to hallucinate humans; synthetic test image 224c is configured to cause the malicious NN to hallucinate bicycles; and synthetic test image 224d is configured to cause the malicious NN to hallucinate cars.

[0026] The synthetic test image 224 substantially corresponds to Figure 1 . Thus, synthetic test image 124 may share any of the features attributed to synthetic test image 224 of the present disclosure, and vice versa. Thus, similar to synthetic test image 224, synthetic test image 124 is a synthetic test image 124 that is indiscernible to a human observer, but is still configured to be recognized by the malicious NN as one or more discernible objects. The latest state of the art in deep learning has demonstrated that it is possible to generate (i.e., synthesize) images that are completely indiscernible to humans with full confidence, but that the most advanced NNs consider to be discernible objects with almost complete credibility.

[0027] Thus, synthetic test images 124 / 224 may be used to detect and prevent so-called “man-in-the-middle” attacks. A man-in-the-middle attack may occur due to the interception of data communications within content processing pipeline 150. Alternatively, a man-in-the-middle attack may occur due to malware being installed during a firmware update of one or more of first processing node 152a, intermediate processing node 152b, or final processing node 152c (e.g., due to a camera firmware update or a security vulnerability on the cloud).

[0028] The type of synthetic test image 124 / 224 that causes a malicious NN configured to generate deepfakes to produce hallucinations in turn depends on the architecture of the malicious NN and the dataset used to train it. The current security solution for content processing pipelines is to use one or more synthetic test images to deceive the most popular NN architectures trained on commonly used training datasets, such as ResNet, you only look once (YOLO), Visual Geometry Group (VGG) architectures, which are trained using ImageNet or "Labeled Faces in the Wild". Therefore, the synthetic test image 124 / 224 is configured to activate one or more neurons of the malicious NN.

[0029] For example, if a malicious NN configured as a deep fake NN interprets a synthetic test image 124 / 224 (e.g., synthetic test image 224b) as including a face, it will typically transform the synthetic test image in an unexpected manner. By comparing the synthetic test image as input to the content processing pipeline 150 and after being processed by at least a portion of the pipeline with the expected image corresponding to the synthetic test image, the present security solution can enable detection of a man-in-the-middle attack using a malicious NN.

[0030] Figure 3 According to one embodiment, a Figure 1 1. Example software code 310 of system 100 is shown. Figure 3 As shown, the software code 310 includes a training module 314, a NN 312, an image insertion module 316, and a safety assessment module 318. In addition, Figure 3 Also shown are training data 342, content stream 328, one or more synthetic test images 324 (hereinafter referred to as "test images 324"), and Figure 1 The input stream 348 of the content processing pipeline 150 in the embodiment of the present invention, and the output streams 354, 356, 358 received from the content processing pipeline 150. Figure 3 Also shown are a synthetic test image 324 (hereinafter referred to as the "processed test image 360") that has been processed by at least a portion of the content processing pipeline 150, one or more expected images 362 (hereinafter referred to as the "expected image 362") corresponding to the synthetic test image 324, and a security assessment 364.

[0031] The software code 310 including the NN 312, the content stream 328, the input stream 348 to the content processing pipeline 150, the output streams 354, 356, 358 received from the content processing pipeline 150, and the security assessment 364 generally correspond to Figure 11, including software code 110, content stream 128, input stream 148, output streams 154, 156, 158, and security assessment 164 of NN 112. That is, software code 110, NN 112, content stream 128, input stream 148, output streams 154, 156, 158, and security assessment 164 may share all features attributed to software code 310, NN 312, content stream 328, input stream 348, output streams 354, 356, 358, and security assessment 364, respectively, and vice versa. Therefore, although Figure 1 Although not explicitly shown, the software code 110 may include features corresponding to each of the training module 314, the image insertion module 316, the safety assessment module 318, and the optional NN 112 / 312.

[0032] In addition, Figure 3 The synthetic test image 324 in corresponds generally to the Figure 1 and Figure 2 In other words, synthetic test image 324 may share all of the features that the present disclosure attributes to synthetic test image 124 / 224, and vice versa. Thus, similar to synthetic test image 124 / 224, synthetic test image 324 is not discernible to a human observer, but may still be configured to be recognized by a malicious NN as a synthetic test image of a discernible object.

[0033] The functionality of the system 100 including the software code 110 / 310 will be achieved by combining Figure 1 , 2 , 3 reference Figure 4 Further description. According to one embodiment, Figure 4 A flowchart 470 is shown representing an exemplary method for protecting a content processing pipeline. Figure 4 Note that certain specific details and features are omitted from flowchart 470 in order not to obscure the discussion of the inventive features of the present application.

[0034] First, for conceptual clarity, two exemplary embodiments of the content processing pipeline 150 are described in more detail herein. However, it should be emphasized that the specific implementation details disclosed in this application are merely exemplary and should not be interpreted as limiting the scope of the present inventive concept in any way.

[0035] For example, in one embodiment, the content processing pipeline 150 may include six nodes, namely a first processing node 152a, four intermediate processing nodes 152b, and a final processing node 152c. In one such embodiment, the first processing node 152a may be a mobile unit including a video camera. The content generated by the first processing node 152a is transmitted to an intermediate processing node 152b in the form of a video transcoder; the intermediate processing node 152b may in turn transmit the transcoded video to another intermediate processing node 152b in the form of a segmenter, which is configured to segment and package the content. The content output by the segmenter may be transmitted to another intermediate processing node 152b in the form of a video production device, where it is processed by another intermediate processing node 152b in the form of a color correction system. Finally, the final processing node 152c may add additional text and / or image overlays to the content and broadcast the content to the consumer audience of the video content.

[0036] Alternatively, in another embodiment, the first processing node 152a may be a camera that captures video of a green screen stage in a television studio. The content generated by the first processing node 152a is transmitted to an intermediate processing node 152b in the form of a video transcoder; the intermediate processing node 152b may then transmit the transcoded video to another intermediate processing node 152b in the form of an online application for previewing the video content. The previewed video content may be transmitted to another intermediate processing node 152b in the form of a color correction system, and then to the final processing node 152c in the form of a playback system or a device that enables the user 132 to preview the video content before distribution to consumers.

[0037] Note that in some embodiments, content processing pipeline 150 may include a processing node, such as first processing node 152a, configured to create raw content before it is processed by other processing nodes in content processing pipeline 150. However, in other embodiments, content processing pipeline 150 may receive content that is created outside of content processing pipeline 150, for example, by third-party content creators (such as by authorized field journalists using mobile camera equipment), or from amateur user-submitted images captured using smartphone video or still image cameras.

[0038] Now combine Figure 1 , 2 , 3 reference Figure 4, flowchart 470 begins by inserting a synthetic test image 124 / 224 / 324 into a content stream 128 / 328, the synthetic test image 124 / 224 / 324 configured to activate one or more neurons of a malicious NN (step 471). The content stream 128 / 328 can be, for example, a sequence of digital photos or a video clip. The synthetic test image 124 / 224 / 324 can be inserted into the content stream 128 / 328 by appending to the beginning or end of the content stream 128 / 328, or by inserting between photo images or video frames of the content stream 128 / 328. In one embodiment, the synthetic test image 124 / 224 / 324 can be inserted into the content stream 128 / 328 when a hash value of the content contained in the content stream 128 / 328 is embedded in or appended to the content stream 128 / 328. The synthetic test image 124 / 224 / 324 may be inserted into the content stream 128 / 328 by the software code 110 / 310 executed by the hardware processor 104 and using the image insertion module 316 .

[0039] Note that in some embodiments, the software code 110 / 310 may obtain a previously generated synthetic test image 124 / 224 / 324 from the image database 126. However, in other embodiments, the software code 110 / 310 may include an optional NN 112 / 312. In those embodiments, the hardware processor 104 may execute the software code 110 / 310 to generate the synthetic test image 124 / 224 / 324 using the NN 112 / 312. For example, in some embodiments, the NN 112 / 312 may be a GAN that includes a generator 313 for generating modifications to random images, and includes a discriminator 315 implemented to model a target malicious NN architecture.

[0040] In embodiments where NN 112 / 312 is used to generate synthetic test images 124 / 224 / 324, NN 112 / 312 must be trained prior to step 471. NN 112 / 312 may be trained using training platform 140, training data 342, training module 314 of software code 110 / 310. The purpose of the training is to generate synthetic test images that are indistinguishable to human observers but are likely to be recognized as recognizable objects by the target malicious NN.

[0041] While training, the discriminator 315 of the NN 112 / 312 may view the synthetic test image output by the generator 313 and decide whether it credibly depicts a discernible object. Verification of the learning process during training may be performed by a user 132, who may utilize the user system 130 to evaluate the synthetic test images 124 / 224 / 324. However, in some embodiments, verification of learning may be performed as an automated process using the discriminator 315. Once training is complete, the software code 110 / 310 including the NN 112 / 312 may be utilized in an automated process to generate synthetic test images 124 / 224 / 324.

[0042] In some embodiments, a single synthetic test image 124 / 224 / 324 may be inserted into the content stream 128 / 328 in step 471. However, in other embodiments, it may be advantageous or desirable to insert multiple synthetic test images 124 / 224 / 324 into the content stream 128 / 328. For example, because any one of the synthetic test images 124 / 224 / 324 may or may not activate the malicious NN and cause it to produce hallucinations, the insertion of multiple synthetic test images 124 / 224 / 324 increases the likelihood that the presence of the malicious NN in the content processing pipeline will be detected. When multiple synthetic test images 124 / 224 / 324 are inserted into the content stream 128 / 328, the synthetic test images 124 / 224 / 324 may be grouped together, for example, at the beginning of the content stream 128 / 328, or distributed throughout the content stream 128 / 328.

[0043] Flowchart 470 continues by providing the content stream 128 / 328 including the synthetic test image 124 / 224 / 324 as the input stream 148 / 348 to the first processing node 152a of the content processing pipeline 150 (step 472). Figure 1 As shown, the input stream 148 / 348 may be provided to the first processing node 152a of the content processing pipeline 150 by being transmitted to the first processing node 152a via the communication network 120 and the network communication link 122. Step 472 may be performed by the software code 110 / 310 executed by the hardware processor 104. In some embodiments, the input stream 148 / 348 including the synthetic test image 124 / 224 / 324 may be provided to the content processing pipeline 150 periodically, such as once a day, or once every few hours.

[0044] Flowchart 470 continues by receiving an output stream including processed test image 360 ​​from one of first processing node 152a or second processing node 152b of content processing pipeline 150 after input stream 148 / 348 including synthetic test image 124 / 224 / 324 is processed by at least a portion of content processing pipeline 150 (step 473). In some embodiments, system 100 may generate security assessment 164 / 364 for first processing node 152a alone. In those embodiments, in step 473, hardware processor 104 may execute software code 110 / 310 to receive output stream 154 / 354 from first processing node 152a via communication network 120 and network communication link 122.

[0045] In other embodiments, system 100 may generate security assessments 164 / 364 for first processing node 152a and one or more intermediate processing nodes 152b. In those embodiments, in step 473, hardware processor 104 may execute software code 110 / 310 to receive output stream 156 / 356 from a second processing node included in one or more intermediate processing nodes 152b via communication network 120 and network communication link 122.

[0046] In other embodiments, the system 100 may generate a security assessment 164 / 364 for the entire content processing pipeline 150. In those more comprehensive embodiments, in step 473, the hardware processor 104 may execute the software code 110 / 310 to receive the output stream 158 / 358 from the last processing node 152c via the communication network 120 and the network communication link 122. Step 473 may be performed by the software code 110 / 310 executed by the hardware processor 104.

[0047] Flowchart 470 continues by comparing the processed test image 360 ​​in the received output stream with an expected image 362 corresponding to the synthetic test image 124 / 224 / 324 (step 474). The expected image 362 may be substantially the same as the synthetic test image 124 / 224 / 324, or may be a synthetic test image 124 / 224 / 324 modified by a predictable transformation applied thereto by processing performed by one or more processing nodes of the content processing pipeline 150. The comparison of the processed test image 360 ​​with the expected image 362 may be performed by software code 110 / 310 executed by hardware processor 104 and using security assessment module 318.

[0048] In some embodiments, Figure 4As shown, flowchart 470 may end with verifying that at least a portion of the content processing pipeline 150 being evaluated for security is secure when the processed test image 360 ​​matches the expected image 362 (step 475a). As described above, in the use case where a malicious NN is present in the portion of the content processing pipeline 150 being evaluated, manipulation of the synthetic test image 124 / 224 / 324 by the malicious NN will cause the processed test image 360 ​​to have unexpected features that are not present in the expected image 362. Therefore, the matching of the processed test image 360 ​​with the expected image 362 advantageously determines that a malicious NN is not present in the portion of the content processing pipeline 150 evaluated in steps 471, 472, 473, 474, 475a, thereby verifying the data security of that portion of the pipeline.

[0049] In embodiments where a single synthetic test image 124 / 224 / 324 is inserted into the content stream 128 / 328 in step 471, a single processed test image 360 ​​is compared to a single expected image 362. However, in embodiments where multiple synthetic test images 124 / 224 / 324 are inserted into the content stream 128 / 328, each of the multiple processed test images 360 will be compared to a corresponding one of the expected images 362. In those latter embodiments, validation of the portion of the content pipeline evaluated in steps 471, 472, 473, 474 (hereinafter referred to as "steps 471-474") may require that each processed test image 360 ​​be matched to its corresponding expected image 362.

[0050] In some embodiments, it may be advantageous or desirable to evaluate the security of the content processing pipeline 150 as a whole. For example, in step 473, when the output stream 158 / 358 is received from the last processing node 152c of the content processing pipeline 150, when the processed test image 360 ​​matches the expected image 362, the entire content processing pipeline 150 can be verified as secure. However, in other embodiments, it may be advantageous or desirable to evaluate the security of the content processing pipeline by evaluating the security of each processing node individually. In those latter embodiments, for example, steps 471-474 may be performed for each of the first processing node 152a, the intermediate processing node 152b, and the last processing node 152c.

[0051] Conversely, in cases where the processed test image 360 ​​does not match the expected image 362, the method outlined by flowchart 470 advantageously detects the possible presence of a malicious NN on at least one processing node of the content processing pipeline 150. In those cases, steps 471, 472, 473, 474 may be repeated for each processing node (or a subset of those processing nodes) of the portion of the content processing pipeline 150 that is provided with the output stream received in step 473 (step 475b). For example, in the event that an input stream 148 / 348 including a synthetic test image 124 / 224 / 324 is provided to a first processing node 152a, and a processed test image 360 ​​included in an output stream 158 / 358 received from a final processing node 152c fails to match an expected image 362, the input stream 148 / 348 may be provided again to the first processing node 152a, but at this time, the processed test image 360 ​​included in an output stream 156 / 356 received from an intermediate processing node 152b is compared with the expected image 362. If the processed test image 360 ​​included in the output stream 156 / 356 matches the expected image 362, but the processed test image 360 ​​included in the output stream 158 / 358 does not match the expected image 362, the first processing node and the intermediate processing node 152b may be verified as safe, while the image manipulation performed by the malicious NN is advantageously identified as being confined to the final processing node 152c.

[0052] Note that in some embodiments, the hardware processor 104 may execute the software code 110 / 310 to perform the steps outlined in the flowchart 470 to generate the security assessment 164 / 364 in an automated process that may eliminate human involvement. That is, in those embodiments, the system 100 is configured to automatically generate the security assessment 164 / 364. Additionally, in some embodiments, the hardware processor 104 may execute the software code 110 / 310 to render the security assessment 164 / 464 on the display 108 of the system 100. Alternatively, in some embodiments, the hardware processor 104 may execute the software code 110 / 310 to transmit the security assessment 164 / 364 to the user system 130 via the communication network 120 and the network communication link 122 for rendering on the display 138 of the user system 130.

[0053] Therefore, the present application discloses a system and method for protecting a content processing pipeline that overcomes the defects and deficiencies of conventional technologies. As described above, by inserting a synthetic test image configured to activate one or more neurons of a malicious NN into a content stream provided as an input stream to a content processing pipeline, the present security solution advantageously enables detection when a malicious NN is present in the pipeline. In addition, by comparing a synthetic test image included in an output stream received from one or more processing nodes in the pipeline with an expected image of its corresponding synthetic test image, the present solution enables identification of one or more nodes in the pipeline that are infected by a malicious NN.

[0054] It is apparent from the above description that various techniques can be used to implement the concepts described in this application without departing from the scope of these concepts. In addition, although these concepts have been described with specific reference to certain embodiments, those skilled in the art will recognize that changes can be made in form and detail without departing from the scope of these concepts. Therefore, the described embodiments should be considered to be illustrative and not restrictive in all respects. It should also be understood that the application is not limited to the specific embodiments described herein, and many rearrangements, modifications and replacements without departing from the scope of the present disclosure are possible.

Claims

1. A system configured to ensure data security of a content processing pipeline comprising a plurality of processing nodes, the system comprising: computing platform, including hardware processor and system memory; software code stored in said system memory; The hardware processor is configured to execute the software code to generate a security assessment of at least a portion of the content processing pipeline by: inserting a synthetic test image into the content stream, the synthetic test image configured to activate one or more neurons of the malicious neural network; providing the content stream including the synthetic test image as an input stream to a first processing node of a plurality of processing nodes of the content processing pipeline; receiving an output stream from one of a first processing node or a second processing node of a plurality of processing nodes of the content processing pipeline, the output stream comprising a processed test image; comparing the processed test image in the received output stream with an expected image corresponding to the synthesized test image; and When the processed test image in the received output stream matches the expected image, at least a portion of the content processing pipeline is verified to be secure.

2. The system of claim 1, wherein: The received output stream is from a second processing node of multiple processing nodes of the content processing pipeline, wherein the second processing node is the last processing node of the content processing pipeline, and wherein, when the processed test image in the received output stream matches the expected image, the entire content processing pipeline is verified to be secure.

3. The system of claim 1, wherein: The software code includes a neural network trained to generate the synthetic test image, and wherein the hardware processor is configured to further execute the software code to generate the synthetic test image using the neural network.

4. The system of claim 3, wherein the neural network comprises a generative adversarial network.

5. The system of claim 4, wherein: The generative adversarial network includes a discriminator configured to model the malicious neural network.

6. The system of claim 4, wherein: The generative adversarial network includes a discriminator configured to model a deep fake neural network.

7. The system of claim 1, wherein: The plurality of processing nodes of the content processing pipeline include an image processing device.

8. The system of claim 1, wherein: The hardware processor is configured to execute the software code to automatically generate the security assessment.

9. A method for use by a system comprising a computing platform having a hardware processor and a system memory storing software code to generate a security assessment of at least a portion of a content processing pipeline comprising a plurality of processing nodes, the method comprising: inserting a synthetic test image into a content stream as a test image by executing the software code by the hardware processor, the synthetic test image being configured to activate one or more neurons of a malicious neural network; By executing the software code by the hardware processor, providing the content stream including the synthetic test image as an input stream to a first processing node of a plurality of processing nodes of the content processing pipeline; receiving, by executing the software code by the hardware processor, an output stream from one of a first processing node or a second processing node of a plurality of processing nodes of the content processing pipeline, the output stream comprising a processed test image; By executing the software code by the hardware processor, comparing the processed test image in the received output stream with an expected image corresponding to the synthetic test image; and By executing the software code by the hardware processor, at least a portion of the content processing pipeline is verified to be secure when the processed test image matches the expected image in the received output stream.

10. The method of claim 9, wherein: The received output stream is from a second processing node of multiple processing nodes of the content processing pipeline, wherein the second processing node is the last processing node of the content processing pipeline, and wherein, when the processed test image in the received output stream matches the expected image, the entire content processing pipeline is verified to be secure.

11. The method of claim 9, wherein: The software code includes a neural network trained to generate the synthetic test image, and wherein the method further includes, by executing the software code by the hardware processor, generating the synthetic test image using the neural network.

12. The method of claim 11, wherein the neural network comprises a generative adversarial network.

13. The method of claim 12, wherein: The generative adversarial network includes a discriminator configured to model the malicious neural network.

14. The method of claim 12, wherein: The generative adversarial network includes a discriminator configured to model a deep fake neural network.

15. The method of claim 9, wherein: The plurality of processing nodes of the content processing pipeline include an image processing device.

16. The method of claim 9, wherein: The hardware processor is configured to execute the software code to automatically generate the security assessment.

17. A computer-readable non-transitory medium storing instructions which, when executed by a hardware processor, instantiate a method comprising: inserting a synthetic test image into the content stream as a test image, wherein the synthetic test image is configured to activate one or more neurons of the malicious neural network; providing the content stream including the synthetic test image as an input stream to a first processing node of a plurality of processing nodes of a content processing pipeline; receiving an output stream from one of a first processing node or a second processing node of a plurality of processing nodes of the content processing pipeline, the output stream comprising a processed test image; comparing the processed test image in the received output stream with an expected image corresponding to the synthesized test image; and When the processed test image matches the expected image in the received output stream, at least a portion of the content processing pipeline is verified to be secure.

18. The computer-readable non-transitory medium of claim 17, wherein: The received output stream is from a second processing node of multiple processing nodes of the content processing pipeline, wherein the second processing node is the last processing node of the content processing pipeline, and wherein, when the processed test image in the received output stream matches the expected image, the entire content processing pipeline is verified to be secure.

19. The computer-readable non-transitory medium of claim 17, wherein: The computer-readable non-transitory medium stores a neural network trained to generate the synthetic test image, and wherein the method further includes generating the synthetic test image using the neural network.

20. The computer-readable non-transitory medium of claim 19, wherein: The neural network includes a generative adversarial network, which includes a discriminator configured to model the malicious neural network.

Images