A sensor hardware Trojan detection method and system based on image processing

Abstract

The application discloses a sensor hardware Trojan detection method and system based on image processing and belongs to the field of sensor anomaly detection. Sensor circuit images and corresponding original design files containing different types, manufacturing processes and covering implanted and unimplanted Trojans are acquired, and part of the images introduces quality defects. The images and the files are registered, the Trojan area is labeled to construct a first training sample set, and different types of regions of interest are cropped to construct a second training sample set. The two sample sets are used to train a target recognition model for recognizing the Trojan area and a detection model for judging whether the region of interest contains image quality defects, respectively. Finally, the two models are combined and an image repair technology is introduced to judge whether there is a Trojan in the circuit image of the sensor sample to be detected. The application can accurately identify malicious circuits of sensors, reduce false positives through image evaluation and feature extraction, automatically detect and improve efficiency, ensure the safety of sensors and be suitable for various circuit Trojans.

Description

Technical Field

[0001] This invention relates to the field of sensor anomaly detection, and more specifically to a sensor hardware Trojan detection method and system based on image processing. Background Technology

[0002] Sensors are crucial in Cyber-Physical Systems (CPS), supporting a wide range of applications from infrastructure management to everyday convenience. Reliable sensor measurements are essential for ensuring the secure operation of CPS-supported applications. A sensor primarily consists of four key components: a transducer, an amplifier, a filter circuit, and an ADC (Analog-to-Digital Converter). Sensor trojans may be present in any one or more of these components. Sensor trojans mainly exploit specific characteristics in transmitted electrical signals as trigger conditions. Potential consequences of sensor trojans include denial-of-service (DoS) attacks on the sensor, manipulation of the sensor's output by the attacker, and damage to the attacked sensor.

[0003] Modern sensors are becoming increasingly sophisticated, integrating complex integrated circuits and intelligent processing capabilities. Similar to integrated circuit manufacturing, sensor production involves a globally distributed supply chain. Designers and manufacturers must collaborate with numerous third-party suppliers, each potentially having access to sensor design, manufacturing processes, or testing procedures. This makes it possible to embed sensor Trojans at multiple stages. The threat model of sensor Trojans aligns with that of hardware Trojans, where attackers can modify sensor hardware components or embedded software through supply chain access. Such modifications might be designed to activate under specific analog input signals, potentially compromising sensor integrity undetected during standard testing. Unlike existing detection methods, sensor Trojans are primarily composed of analog circuits, rendering mainstream digital circuit-based detection methods inapplicable.

[0004] In summary, it is urgent to address the significant threat posed by sensor backdoors and to improve the detection mechanisms for them. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a sensor hardware Trojan detection method and system based on image processing. This method can accurately identify malicious circuits in sensors, reduce false alarms through image evaluation and feature extraction, automate the detection process, improve efficiency, and is applicable to various types of sensors, ensuring their security and functionality. It is also suitable for Trojans on printed circuit boards and integrated circuits.

[0006] The technical solution proposed in this invention is as follows:

[0007] In a first aspect, this invention proposes a sensor hardware Trojan detection method based on image processing, comprising the following steps:

[0008] (1) Obtain sensor circuit images of different types and manufacturing processes and their corresponding original design files; the sensor circuit images include two cases: those with and without Trojans; introduce image quality defects into some sensor circuit images;

[0009] (2) Register the sensor circuit image and the original design file, mark the region where the Trojan is implanted, and construct the first training sample set; and crop the region of interest containing image quality defects, the region of interest containing Trojans but not image quality defects, and the region of interest containing neither image quality defects nor Trojans in the sensor circuit image to construct the second training sample set.

[0010] (3) Use the first training sample set to train a target recognition model for identifying Trojan regions in registered image pairs, and use the second training sample set to train a region of interest detection model for determining whether the region of interest image contains image quality defects.

[0011] (4) Combine the target recognition model and the region of interest detection model, and introduce image restoration technology for image quality defects to determine whether there is a Trojan horse in the sensor circuit image of the sensor sample to be tested.

[0012] Furthermore, step (4) specifically includes:

[0013] The sensor circuit image of the sensor sample under test is acquired and registered with the original design file. The registered image is input into the target recognition model to identify the Trojan region as a suspected abnormal region. If a suspected abnormal region exists, the region of interest image containing the individual suspected abnormal region is extracted. Each region of interest image is used to determine whether it belongs to the image quality defect. If it does, the region of interest of the sensor circuit image is repaired and the suspected abnormal region is re-identified until the suspected abnormal region marked by the target recognition model does not belong to the image quality problem. At this time, the suspected abnormal region is judged to be infected with Trojan. If no suspected abnormal region is marked after repair, it is judged to be a false alarm caused by the image quality defect, and the sensor does not have Trojan.

[0014] Furthermore, the image restoration technique employs a gated convolutional image restoration algorithm.

[0015] Furthermore, the sensor circuit image is a scanning electron microscope image of a sensor with an integrated circuit structure, or a visible light image of a sensor with a printed circuit board structure.

[0016] Furthermore, the image quality defects include dust, dirt, excessive contrast, and blurriness.

[0017] Furthermore, each sample in the first training sample set is an image pair registered with a sensor circuit image and the original design file. If a Trojan horse is implanted in the sensor circuit image of the image pair, the region where the Trojan horse is implanted is marked; otherwise, no marking is required.

[0018] Furthermore, the target recognition model employs a ResNet network.

[0019] Furthermore, each sample in the second training sample set is a local region of interest of the sensor circuit image. If the region of interest contains image quality defects, the label is 1; otherwise, the label is 0.

[0020] Furthermore, the region of interest detection model adopts the VGG model.

[0021] Secondly, the present invention provides a sensor hardware Trojan detection system based on image processing, for implementing the above-mentioned sensor hardware Trojan detection method based on image processing.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] (1) This invention obtains sensor circuit images and original design files of different types and manufacturing processes, including those with and without Trojans implanted. It also introduces image quality defects and constructs a sample set covering multiple features, providing sufficient and diverse data for model training and improving model adaptability.

[0024] (2) The present invention uses different sample sets to train the target recognition model and the region of interest detection model respectively. One model focuses on identifying Trojan horse regions, and the other judges image quality defects. The functions are clear and the detection accuracy is improved.

[0025] (3) This invention combines two models and introduces image restoration technology, which can more accurately determine whether there is a Trojan horse in the circuit image of the sensor under test, and the automated detection process improves detection efficiency.

[0026] This invention provides a highly efficient and accurate method for detecting hardware Trojans in sensors. This method not only accurately identifies potential hardware Trojans but also effectively reduces false alarm rates through image quality assessment and feature extraction, ensuring reliable detection results. Simultaneously, the automated process significantly reduces manual intervention, improves detection efficiency, and is applicable to various types of sensor products, ensuring their security and functionality in different application scenarios. This greatly enhances the level of sensor security detection and provides strong protection against malicious tampering. In the presence of untrusted manufacturers, this method can effectively reduce threats, enhance sensor security, prevent malicious Trojans from causing sensor malfunctions or even damage, and avoid erroneous operations in intelligent devices such as automobiles and drones. This invention is applicable to Trojan detection at the sensor-level printed circuit board level as well as integrated circuit Trojan detection. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a sensor hardware Trojan detection method based on image processing proposed in this invention. Detailed Implementation

[0028] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0029] The accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] This invention proposes an image processing-based method for detecting sensor hardware Trojans. It utilizes high-resolution scanning electron microscopy (SEM) images / visible light images and advanced image registration technology to ensure precise correspondence between the physical device image and the original design file. A ResNet model is used to automatically identify and annotate subtle differences between the two, accurately locating potentially abnormal regions of hardware Trojans. Furthermore, a post-processing module is introduced to evaluate the image quality of suspected abnormal regions, correcting anomalies not caused by Trojans, effectively reducing false alarm rates and improving detection reliability.

[0032] like Figure 1As shown, the present invention proposes a sensor hardware Trojan detection method based on image processing, which mainly includes the following steps:

[0033] S1, Data preparation.

[0034] Obtain a batch of sensor samples and their original design files. These sensors contain one or more integrated circuits and printed circuits. The original design files should be considered clean versions without malicious backdoors. Ensure that this data covers sensor circuits of different types and manufacturing processes to improve the model's generalization ability. In this embodiment, different types of sensor circuits can be thermistor-type temperature sensor circuits, thermocouple-type temperature sensor circuits, piezoresistive pressure sensor circuits, capacitive pressure sensor circuits, inductive displacement sensor circuits, photoelectric displacement sensor circuits, etc.; sensor circuits with different manufacturing processes can be semiconductor process (integrated circuit structure) sensor circuits, thick film and thin film process (circuit printed circuit board structure) sensor circuits, etc.

[0035] For sensors with integrated circuit structures, the original design files are typically provided in GDSII format, serving as blueprints for manufacturing semiconductor devices. For sensors with printed circuit board structures, the original design files are typically provided in PCB format, which is also essential material for producing subsequent circuit boards.

[0036] Sensor samples and their original design files are used for subsequent analysis to determine if there are any unauthorized modifications or hardware trojans in the sensor circuitry.

[0037] S2, acquire the sensor circuit image.

[0038] For sensors with integrated circuit structures, preprocessing of the integrated circuit chip is required, including removing the top layer of packaging material and exposing the underlying circuit structure to facilitate subsequent imaging; while for sensors with printed circuit board structures, visible light images of the circuit section can be directly captured.

[0039] In one specific embodiment of the present invention, for integrated circuit chips, due to their high integration level, a scanning electron microscope (SEM) is required to obtain high-resolution images of the chip's internal structure. To cover the entire chip area, multiple SEM images need to be acquired and stitched together into a complete image. For printed circuit boards, however, clear and readable visible light images can be obtained using a high-resolution camera.

[0040] This step requires a large number of normal circuit images known to be free of malicious hardware trojans, as well as images containing trojans. It also requires knowledge of the trojan's implantation location for image annotation as described below. To improve model performance, defects such as localized dust, dirt, excessive contrast, or blurring can be introduced into the normal circuit images and / or images containing trojans; these do not need to be annotated below.

[0041] S3: Preprocessing of sensor circuit images and original design files to construct a training dataset.

[0042] Sensor circuit images can be affected by factors such as uneven lighting and noise interference, leading to a decrease in image quality. For example, SEM images may contain electronic noise, and visible light images may have inconsistent brightness due to uneven ambient light. This invention improves the clarity and contrast of sensor circuit images through preprocessing, making features in the image easier to identify.

[0043] Furthermore, SEM images and visible light images acquired by different devices may have different formats, resolutions, and sizes. Preprocessing can convert images into a uniform format and appropriate size, facilitating subsequent registration and processing, and ensuring consistency in the feature extraction and matching process.

[0044] In this embodiment, image processing tools are used to preprocess the acquired images, such as adjusting contrast (linear contrast stretching, histogram equalization) and removing noise (mean filtering, median filtering), to improve image quality and facilitate subsequent analysis.

[0045] The original design document specifies the standard layout, dimensions, and circuit routing of the circuit board. By registering the SEM image or visible light image with it, the differences between the actual circuit board and the design can be accurately compared, which helps to automatically identify and label subtle differences between the two. In this embodiment, the image registration method can be implemented using existing registration algorithms in the art, such as feature-based registration, template-based registration, or registration using Scale Invariant Feature Transform (SIFT) combined with the Random Sample Consensus Algorithm (RANSAC). Accurate registration of the SEM image or visible light image with the original design document ensures that the registered image and the design document are accurately aligned in spatial position.

[0046] For regions where malware is known to have been implanted, these regions are marked on the SEM or visible light images as areas of difference from the original design file. For clean sensors, no marking is required on the corresponding SEM or visible light images. It should be noted that only the implanted malware area needs to be marked; localized dust, dirt, excessive contrast, blur, etc., do not need to be marked. The registered image pairs are preprocessed, including but not limited to resizing images and normalizing pixel values, to ensure the image data has a uniform format and range. The labeled data also undergoes appropriate transformation to facilitate model input and processing, ultimately yielding the first training dataset for the target recognition model described below.

[0047] In addition, for local dust, dirt, excessively high contrast, and blurred areas on SEM images or visible light images, local regions of interest are extracted and mixed with normal local areas and Trojan implantation areas. The local regions of interest for local dust, dirt, excessively high contrast, and blurred areas are labeled with 1; the normal local areas and Trojan implantation areas are labeled with 0. Finally, a second training dataset is obtained for the region of interest detection model described below.

[0048] S4. Construct and train a target recognition model for labeling suspected abnormal regions.

[0049] A convolutional neural network (CNN) is trained using the first training dataset prepared above. This embodiment employs a ResNet architecture. ResNet demonstrates significant advantages in circuit board image difference annotation tasks. Its residual connections effectively address the vanishing gradient problem, allowing deep networks to learn complex and subtle features to accurately identify differences. Its powerful feature extraction capabilities enable it to grasp various image features and determine differences. This model is trained to identify and annotate minute differences between two input images, and the output includes the specific location of the differences, which may be the location of potential hardware trojans.

[0050] In one specific embodiment of the present invention, the process of training the model is as follows:

[0051] The preprocessed dataset was divided into 70% training set, 15% validation set, and 15% test set.

[0052] Setting parameters during training includes the learning rate, number of training epochs, and batch size. The learning rate determines the step size of parameter updates during training, the number of training epochs controls the number of times the model is trained, and the batch size represents the amount of data input into the model each time. Setting these parameters appropriately is crucial for the training effect of the model.

[0053] The ResNet model is trained using the training set data. During training, the model calculates a loss function (such as cross-entropy loss) between the predicted results and the ground truth labels based on the input image pairs and annotation information, and updates the model parameters using the backpropagation algorithm to minimize the loss function. Simultaneously, the model's performance is monitored using the validation set data, and hyperparameters are adjusted based on the validation results to prevent overfitting.

[0054] The trained ResNet model is evaluated using test set data. Registered image pairs are input into the model, which outputs predicted differential region annotations. The model is evaluated based on these results until it meets the requirements.

[0055] The trained ResNet model can compare differences between registered image pairs and label discrepancies, which are highly likely to be caused by suspicious malware circuits. Using this model, subtle differences in input registered images can be automatically identified and labeled, accurately locating potentially abnormal regions of hardware malware.

[0056] S5. Construct and train a region of interest detection model for the verification of suspected abnormal regions.

[0057] A convolutional neural network (CNN) is trained using the second training dataset prepared above. This embodiment employs the VGG model, which consists of multiple stacked convolutional and pooling layers. This enables it to automatically learn features at different levels in an image, gradually extracting semantic features from simple features such as edges and textures at the lower levels. It can accurately capture subtle textures and shapes of localized dust, dirt, excessive contrast, and blur in an image, identifying them through its deep feature extraction capabilities.

[0058] The training process for the VGG model is similar to that of the ResNet model, except that the dataset used to train the VGG model is a secondary training dataset, which is used to train a binary classifier to identify images containing dust, dirt, excessive contrast, or blur. This training process will not be elaborated further. Before training, data augmentation techniques, including rotation, scaling, and flipping, can be used to improve the model's generalization ability.

[0059] The VGG model can identify common issues in input suspicious images, such as variations, dust, dirt, excessive contrast, and blurriness, which can lead to false positives in target recognition models. This process primarily aims to improve detection quality and reduce the false positive rate.

[0060] S6: Obtain the sensor circuit image of the sample to be tested and the registration image of the original design file to identify whether there is a sensor hardware Trojan.

[0061] Specifically, this involves: acquiring sensor circuit images of sensor samples and registering them with the original design files;

[0062] After registration, the image pairs are input into a ResNet model to identify suspected anomalous regions. If a suspected anomalous region is found, a region of interest (ROI) image containing one of these regions is extracted. Each ROI image is then analyzed using the VGG model to determine if it suffers from image quality issues such as localized dust, dirt, excessive contrast, or blur. If so, the GatedConvolution method is used to repair the image. This algorithm allows the network to adaptively select the information to filter. The repair process aims to make the image as close as possible to its original, lossless state while preserving important details. This method effectively improves image quality. Each ROI image with image quality issues requires repair. After repair, the ResNet model is used again for identification until the suspected anomalous regions marked by the ResNet model are no longer considered image quality issues. At this point, the suspected anomalous regions are considered to be infected with malware. If the repaired image is not marked as a suspected anomalous region after being re-inputted into the ResNet model, the initial detection is considered a false positive due to image quality issues.

[0063] By assessing the image quality of suspected abnormal areas and repairing anomalies not caused by Trojans, the system ensures that even in the presence of image quality issues, the false alarm rate can be significantly reduced through an effective repair and re-detection process, thereby improving the overall reliability of detection.

[0064] Based on the same inventive concept, this embodiment also provides an image processing-based sensor hardware Trojan detection system, which is used to implement the above embodiments. The terms "module," "unit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible.

[0065] In this embodiment, a sensor hardware Trojan detection system based on image processing includes:

[0066] The data acquisition module is used to acquire sensor circuit images of different types and manufacturing processes and their corresponding original design files; the sensor circuit images include both cases with and without malware implanted; and image quality defects are introduced into some sensor circuit images.

[0067] The training sample set construction module is used to register the sensor circuit image and the original design file, label the region where the Trojan is implanted, and construct the first training sample set; and to crop the region of interest containing image quality defects, the region of interest containing Trojans but not image quality defects, and the region of interest containing neither image quality defects nor Trojans in the sensor circuit image to construct the second training sample set.

[0068] The target recognition model training module is used to train a target recognition model for identifying Trojan regions in registered image pairs using a first training sample set.

[0069] The Region of Interest (ROI) detection model training module is used to train a ROI detection model using a second training sample set to determine whether an image contains image quality defects.

[0070] The Trojan joint detection module is used to combine the target recognition model and the region of interest detection model, and introduces image restoration technology for image quality defects to determine whether there is a Trojan in the sensor circuit image of the sensor sample under test.

[0071] For the system embodiments, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments; the implementation methods of the remaining modules will not be repeated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0072] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0073] The above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A sensor hardware Trojan detection method based on image processing, characterized in that, Includes the following steps: (1) Obtain sensor circuit images of different types and manufacturing processes and their corresponding original design files; the sensor circuit images include two cases: those with and without Trojans; introduce image quality defects into some sensor circuit images; (2) Register the sensor circuit image and the original design file, mark the region where the Trojan is implanted, and construct the first training sample set; and crop the region of interest containing image quality defects, the region of interest containing Trojans but not image quality defects, and the region of interest containing neither image quality defects nor Trojans in the sensor circuit image to construct the second training sample set. (3) Use the first training sample set to train a target recognition model for identifying Trojan regions in registered image pairs, and use the second training sample set to train a region of interest detection model for determining whether the region of interest image contains image quality defects. (4) Obtain the sensor circuit image of the sensor sample to be tested and register it with the original design file; input the registered image into the target recognition model to identify the Trojan region as a suspected abnormal region; If a suspected abnormal region exists, the region of interest image containing the individual suspected abnormal region is captured. Each region of interest image is used to determine whether it belongs to the image quality defect using the region of interest detection model. If it does, the region of interest of the sensor circuit image is repaired and the suspected abnormal region is re-identified until the suspected abnormal region marked by the target recognition model does not belong to the image quality problem. At this time, the suspected abnormal region is judged to be infected with a Trojan. If no suspected abnormal area is marked after repair, it is determined that the false alarm is caused by image quality defects, and the sensor does not contain a Trojan horse.

2. The sensor hardware Trojan detection method based on image processing according to claim 1, characterized in that, The image restoration technique employs a gated convolutional image restoration algorithm.

3. The sensor hardware Trojan detection method based on image processing according to claim 1, characterized in that, The sensor circuit image is a scanning electron microscope image of a sensor with an integrated circuit structure, or a visible light image of a sensor with a printed circuit board structure.

4. The sensor hardware Trojan detection method based on image processing according to claim 3, characterized in that, The image quality defects mentioned include dust, dirt, excessive contrast, and blur.

5. The sensor hardware Trojan detection method based on image processing according to claim 1, characterized in that, Each sample in the first training sample set is a pair of images registered with a sensor circuit image and an original design file. If a Trojan horse is implanted in the sensor circuit image of the image pair, the region where the Trojan horse is implanted is marked. Otherwise, no annotation is required.

6. The sensor hardware Trojan detection method based on image processing according to claim 1 or 5, characterized in that, The target recognition model described uses a ResNet network.

7. The sensor hardware Trojan detection method based on image processing according to claim 1, characterized in that, Each sample in the second training sample set is a local region of interest of the sensor circuit image. If the region of interest contains image quality defects, the label is 1; otherwise, the label is 0.

8. The sensor hardware Trojan detection method based on image processing according to claim 1, characterized in that, The region of interest detection model described above adopts the VGG model.

9. A sensor hardware Trojan detection system based on image processing, used to implement the method of claim 1; characterized in that, The system includes: The data acquisition module is used to acquire sensor circuit images of different types and manufacturing processes and their corresponding original design files; the sensor circuit images include two cases: those with malware implanted and those without malware implanted; image quality defects are introduced into some sensor circuit images; The training sample set construction module is used to register the sensor circuit image and the original design file, label the region where the Trojan is implanted, and construct the first training sample set; and to crop the region of interest containing image quality defects, the region of interest containing Trojans but not image quality defects, and the region of interest containing neither image quality defects nor Trojans in the sensor circuit image to construct the second training sample set. The target recognition model training module is used to train a target recognition model for identifying Trojan regions in registered image pairs using a first training sample set. The Region of Interest (ROI) detection model training module is used to train a ROI detection model using a second training sample set to determine whether an image contains image quality defects. The Trojan joint detection module is used to combine the target recognition model and the region of interest detection model, and introduces image restoration technology for image quality defects to determine whether there is a Trojan in the sensor circuit image of the sensor sample under test.

Images