Real-time detection and recovery method and system for laser attack on vehicle-mounted vision system

Abstract

This invention discloses a method and system for real-time detection and recovery of laser attacks in vehicle-mounted vision systems, aiming to solve the problems of detection lag and difficulty in meeting real-time requirements for image restoration under laser attacks in vehicle-mounted vision perception systems. The solution includes: 1) constructing a dual-threshold detection benchmark based on Mahalanobis distance and inter-row brightness gradient using a small number of benign samples; 2) detecting abnormal frame images based on the dual-threshold detection benchmark and locating the laser grating stripe positions and affected color channels; 3) generating a precise laser interference mask under spatiotemporal consistency constraints based on the detection results; and 4) completing image structure reconstruction, edge constraint, and chromaticity suppression based on the laser interference mask to achieve recovery. This invention can accurately and in real-time eliminate laser interference under limited sample conditions, meeting the real-time requirements of autonomous driving vehicle terminals while restoring high-quality image geometric structure and texture details, and can be used in the field of security enhancement for vision perception systems.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and further relates to artificial intelligence security technology. Specifically, it is a method and system for real-time detection and recovery of laser attacks for vehicle vision systems, which can be applied to fields such as autonomous driving, intelligent transportation, and robot vision to ensure the reliable operation of visual perception systems. Background Technology

[0002] With the rapid development of autonomous driving technology, vehicle vision systems, as one of the core sensors for environmental perception, are playing an increasingly important role in core tasks such as target detection, lane keeping, and obstacle recognition. Visual information collected by vehicle cameras is a crucial input to the autonomous driving decision-making system and directly relates to driving safety. However, existing vehicle vision perception systems face a new type of security threat—laser attacks. Laser attacks refer to malicious attackers using high-power laser emitters to project laser beams at vehicle cameras, creating overexposed grating stripes on the CMOS / CCD sensors. This interferes with the normal function of the vision perception system, and in severe cases, may lead to system misjudgment or failure, endangering the safety of autonomous driving.

[0003] Current technologies for anomaly detection in videos or images have significant limitations. For example, patent application CN119625600A discloses an "unsupervised video anomaly behavior detection method, device, equipment, and medium." This method acquires interval frames of a monitoring video, performs optical flow estimation, predicts intermediate frames, and then compares the feature similarity between the predicted frames and the real frames to determine anomalies, achieving unsupervised anomaly behavior detection in monitoring scenes. However, it does not model the exposure grating features of laser attacks, and the computational complexity is high due to the need to extract optical flow features and perform complex inter-frame predictions, making it difficult to meet the real-time requirements of autonomous driving scenarios. Patent application CN118918013A discloses an "FPGA-based image median filtering method, system, equipment, and medium." This method identifies noise points by establishing a pixel-corresponding neuron processing model, then sorts the noise points and neighboring pixels and replaces them with the target pixel value, achieving noise filtering of static images. However, this method does not consider the spatiotemporal continuity of laser grating stripes in the video stream and relies on dedicated FPGA hardware, increasing system cost and deployment complexity, making it difficult to directly apply to ordinary vehicle-mounted systems. Patent document with application number CN118898771A discloses "an abnormal image detection method, device and related equipment". It reconstructs an abnormal image by using an adversarial neural network and compares it with the input image to achieve anomaly detection. It works well in customs and industrial inspection scenarios. However, this method requires a large number of samples for training. The difficulty in obtaining laser attack samples makes training difficult. In addition, the inference computation overhead of the adversarial neural network is large and the processing time of a single frame image is long, which cannot meet the strict real-time requirements of autonomous driving scenarios.

[0004] In summary, existing technologies have shortcomings in terms of adaptability to laser attack scenarios, sample dependence, computational efficiency, and feasibility of vehicle deployment. They are difficult to simultaneously achieve detection accuracy, image restoration quality, and real-time processing requirements on the vehicle side, and are prone to problems such as abrupt boundaries of restored images and cross-frame flickering, which affect the stability of subsequent perception tasks and cannot meet the laser attack detection needs in autonomous driving scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a real-time laser attack detection and recovery method and system for vehicle-mounted vision systems. This invention solves the technical problems of existing vehicle-mounted vision perception systems encountering laser attacks, such as detection lag due to a lack of prior attack samples, and the high computational complexity of deep learning-based image restoration methods, which are difficult to meet the real-time requirements of vehicle-mounted systems. It also addresses the difficulty of existing methods in balancing laser attack detection accuracy, image restoration quality, and real-time processing performance on the vehicle-mounted system. This invention does not rely on training with abnormal laser attack samples. It constructs a dual-threshold detection benchmark using only a small number of benign samples, and combines spatiotemporally consistent mask construction with multi-dimensional image restoration strategies to achieve accurate real-time laser attack detection and high-quality restoration of abnormal frames, ensuring the stability of subsequent tasks such as target detection and semantic segmentation in the vehicle-mounted vision perception system.

[0006] To achieve the above objectives, the technical solution of the present invention includes the following:

[0007] (1) Select N consecutive frames of benign samples, with the input frame being That is, a 3-channel image with height H and width W; a 9-dimensional feature vector is obtained after sampling each frame. Simultaneously calculate the inter-line brightness gradient of benign frames. :

[0008] ,

[0009] ;

[0010] in, , , This represents the RGB channel difference between adjacent benign samples; Indicates the difference between the brightness channels; Indicates the maximum channel difference; Indicates color deviation; Represents the normalized chromaticity variation values ​​of the RGB channels; Indicates the transpose operation; Max represents the inter-line brightness gradient of the i-th group of adjacent lines in a benign frame. i This indicates taking the maximum gradient value across all rows;

[0011] (2) Calculate the eigenvectors Mahalanobis distance And based on benign samples Distribution determines Mahalanobis distance threshold Meanwhile, based on benign samples Distribution determines inter-line brightness gradient threshold ; and by and To jointly construct a dual-threshold detection benchmark;

[0012] (3) For the current input frame to be detected by the vehicle vision system, repeat the feature extraction process in step (1), calculate the Mahalanobis distance and inter-line brightness gradient of the feature vector of the frame, and perform abnormal frame image detection based on the dual threshold detection benchmark, including... and contrast, and The following judgment is made based on the comparison: If and If it is not, it is determined to be an abnormal frame; otherwise, the background frame is updated. For the current frame, that is, let = ;

[0013] (4) Locate the position of the abnormal frame laser grating stripes and the affected color channels to obtain the detection results. The steps are as follows:

[0014] (4.1) Calculate the interline brightness gradient transformation of abnormal frames And use it to locate the starting line of the laser grating stripes in the abnormal frame. and end line :

[0015] like Then the i-th row will be recorded as the starting row. ,

[0016] like Then the i-th row will be recorded as the end row. ;

[0017] (4.2) Determine the spatial range of the laser grating fringes in the abnormal frame Within this range, the channel most severely affected by laser interference is selected as the main channel. :

[0018] ,

[0019] ;

[0020] in, c=0,1,2 correspond to channels B, G, and R respectively;

[0021] (5) Based on the detection results, calculate the overexposure mask of the current abnormal frame in sequence. Background frame overexposure mask Luminance comparison mask between the current abnormal frame and the background frame Noise in the non-interference area is filtered out by a triple mask; an initial two-dimensional laser mask is constructed using horizontal and vertical projections. And under spatial consistency constraints, for Perform normalization reconstruction:

[0022] ;

[0023] in This indicates vertical projection synthesis. It is a non-zero extreme small number;

[0024] Under the constraint of temporal consistency, the mask is updated through inter-frame cumulative smoothing to obtain an accurate laser interference mask. :

[0025] ;

[0026] in Let be the cumulative mask for frame t. For the cumulative mask of frame t-1, A two-dimensional mask for reconstructing the current frame. Indicates time smoothing parameters;

[0027] (6) Based on the precise laser interference mask, abnormal frame image restoration is performed, and image structure reconstruction, edge structure constraint and color suppression are completed in sequence to complete image restoration.

[0028] A laser attack real-time detection and recovery system that implements the method of claim 1, characterized in that it includes: a dual-threshold training module, an abnormal frame detection and localization module, a precise mask construction module, and a multi-dimensional image recovery module;

[0029] The dual-threshold training module is useful for calculating the Mahalanobis distance of feature vectors and the inter-row brightness gradient using a small number of benign samples, and calculates the dual thresholds of feature vectors and inter-row brightness gradients based on preset percentiles.

[0030] The abnormal frame detection and localization module calculates the Mahalanobis distance and interline brightness gradient of the feature vector of the current frame, judges the current frame as abnormal based on the dual thresholds of the Mahalanobis distance and interline brightness gradient, locates the range of the laser stripe area by the change of interline brightness gradient, and determines the affected channel by the brightness change of each channel.

[0031] The precise mask construction module generates a precise laser mask based on the detection results of the abnormal frame detection and localization module, and on the current frame and background frame, under the condition of spatiotemporal consistency constraints.

[0032] The multi-dimensional image restoration module performs image structure reconstruction, edge structure constraint, and chromaticity suppression operations on the current frame sequentially based on the laser interference mask, and outputs the restored normal frame.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] First, this invention employs a dual-threshold joint detection mechanism: on one hand, it utilizes the Mahalanobis distance of feature vectors to detect abnormal changes in inter-frame features; on the other hand, it captures the spatial distribution features of laser stripes through inter-line brightness gradients. Only when both thresholds are exceeded is the frame considered abnormal, effectively reducing the false detection rate. Compared to existing single-feature detection methods, this invention can more accurately distinguish between laser attacks and normal scene changes, significantly improving detection accuracy.

[0035] Secondly, this invention proposes a laser mask construction method under spatiotemporal consistency constraints: In the temporal dimension, an exponential moving average is used to update the laser mask, ensuring a smooth transition between masks from multiple laser attacks and preventing severe flickering in the recovery results; in the spatial dimension, the two-dimensional mask is reconstructed using the outer product operation of energy projections in the horizontal and vertical directions, guaranteeing the inherent coordination of energy distribution in the horizontal and vertical directions. Compared to the existing frame-by-frame independent processing scheme, this invention achieves a consistent recovery effect across frames.

[0036] Third, this invention proposes a multi-dimensional layered restoration method: abnormal frame restoration adopts a layered strategy of "structure reconstruction-edge constraint-chromaticity suppression". First, the geometric structure of the image is reconstructed by laser mask. Then, the inter-row brightness gradient of the grating stripe area is smoothed to achieve natural connection of the edge structure. Finally, by suppressing chromaticity noise, compared with the existing simple restoration method of directly subtracting the mask, this method effectively solves the problems of abrupt image boundaries and residual chromaticity noise, and restores the geometric structure and texture details of the image with high quality. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the overall implementation of the method of the present invention;

[0038] Figure 2 This is a schematic diagram of the system framework of the present invention;

[0039] Figure 3 These are representative abnormal frame images selected in the embodiments of the present invention;

[0040] Figure 4 This is the abnormal frame image after recovery processing in this embodiment of the invention;

[0041] Figure 5 This is a comparison chart of the detection performance of the present invention and existing methods;

[0042] Figure 6 This is a comparison chart of the image restoration performance of the present invention and existing methods. Detailed Implementation

[0043] The present invention will now be further described with reference to the accompanying drawings.

[0044] Example 1: Refer to Figure 1 The present invention proposes a method for real-time detection and recovery of laser attacks for vehicle-mounted vision systems, characterized by the following steps:

[0045] Step 1. Select N consecutive frames of benign samples, with the input frames being... That is, a 3-channel image with height H and width W; a 9-dimensional feature vector is obtained after sampling each frame. Simultaneously calculate the inter-line brightness gradient of benign frames. :

[0046] ,

[0047] ;

[0048] in, , , This represents the RGB channel difference between adjacent benign samples; Indicates the difference between the brightness channels; Indicates the maximum channel difference; Indicates color deviation; Represents the normalized chromaticity variation values ​​of the RGB channels; Indicates the transpose operation; Max represents the inter-line brightness gradient of the i-th group of adjacent lines in a benign frame. i This indicates taking the maximum gradient value across all rows.

[0049] The aforementioned color deviation in this embodiment Based on the RGB channel difference between adjacent benign samples , , Specifically, calculate according to the following formula:

[0050] ;

[0051] in .

[0052] Step 2. Calculate the eigenvectors Mahalanobis distance And based on benign samples Distribution determines Mahalanobis distance threshold Meanwhile, based on benign samples Distribution determines inter-line brightness gradient threshold ; and by and A dual-threshold detection benchmark is jointly constructed. The aforementioned feature vectors... Mahalanobis distance The result is obtained by calculation using the following formula:

[0053] ;

[0054] in, , , The regularization coefficient is . It is an identity matrix.

[0055] Step 3. For the currently input frame to be detected by the vehicle vision system, repeat the feature extraction process of step (1), calculate the Mahalanobis distance and inter-line brightness gradient of the feature vector of the frame, and perform abnormal frame image detection based on the dual threshold detection benchmark, including... and contrast, and The following judgment is made based on the comparison: If and If it is not, it is determined to be an abnormal frame; otherwise, the background frame is updated. For the current frame, that is, let = ;

[0056] Step 4. Locate the position of the abnormal frame laser grating stripes and the affected color channels to obtain the detection results. The steps are as follows:

[0057] (4.1) Calculate the interline brightness gradient transformation of abnormal frames And use it to locate the starting line of the laser grating stripes in the abnormal frame. and end line :

[0058] like Then the i-th row will be recorded as the starting row. ,

[0059] like Then the i-th row will be recorded as the end row. ;

[0060] (4.2) Determine the spatial range of the laser grating fringes in the abnormal frame Within this range, the channel most severely affected by laser interference is selected as the main channel. :

[0061] ,

[0062] ;

[0063] in, c=0,1,2 correspond to channels B, G, and R respectively;

[0064] Step 5: Based on the detection results, calculate the overexposure mask for the current abnormal frame sequentially. Background frame overexposure mask Luminance comparison mask between the current abnormal frame and the background frame Noise in the non-interference area is filtered out by a triple mask; an initial two-dimensional laser mask is constructed using horizontal and vertical projections. And under spatial consistency constraints, for Perform normalization reconstruction:

[0065] ;

[0066] in This indicates vertical projection synthesis. It is a non-zero extreme small number;

[0067] Under the constraint of temporal consistency, the mask is updated through inter-frame cumulative smoothing to obtain an accurate laser interference mask. :

[0068] ;

[0069] in Let be the cumulative mask for frame t. For the cumulative mask of frame t-1, A two-dimensional mask for reconstructing the current frame. Indicates the time smoothing parameter.

[0070] In this embodiment, the overexposure mask of the current abnormal frame is described above. Background frame overexposure mask Luminance comparison mask between the current abnormal frame and the background frame Specifically, it is calculated according to the following steps:

[0071] (5.1) Calculate the overexposure mask of the current abnormal frame. :

[0072] ;

[0073] in, and These are the first non-main channel and the second non-main channel, respectively. For overexposure parameters, The overexposure difference; Indicates an indicator function; For logical OR operator; Indicates the coordinates of the current abnormal frame. Pixel value at;

[0074] (5.2) Calculate the overexposure mask of the background frame ;

[0075] ;

[0076] in, Indicates the background frame at coordinates Pixel value at;

[0077] (5.3) Calculate the brightness comparison mask between the current abnormal frame and the background frame. :

[0078] ;

[0079] in, Indicates the current abnormal frame's luminance channel at coordinates Pixel value at; Indicates the background frame luminance channel in coordinates The pixel value at that location.

[0080] The above describes the construction of an initial two-dimensional laser mask using horizontal and vertical projections. The implementation steps are as follows:

[0081] (5.4) Calculate the horizontal projection :

[0082] ;

[0083] (5.5) Calculate the vertical projection :

[0084] ;

[0085] (5.6) Perform median filtering and Gaussian smoothing on the projection:

[0086] ,

[0087] ,

[0088] ,

[0089] ;

[0090] Where k is the preset kernel size for median filtering. Standard deviation Gaussian kernel;

[0091] (5.7) Construct a two-dimensional laser mask using horizontal and vertical projections. :

[0092] .

[0093] in , It is a non-zero extreme small number.

[0094] Step 6. Perform abnormal frame image restoration based on a precise laser interference mask, sequentially completing image structure reconstruction, edge structure constraint, and chromaticity suppression to complete image restoration.

[0095] The image structure reconstruction described in this embodiment is used to restore the basic structure of an image damaged by laser, and includes the following steps:

[0096] (6a1) Subtract the laser mask from the aberration frame to obtain the initial restored image. :

[0097] ;

[0098] (6a2) Apply overexposed area protection to crop the image to the effective area:

[0099] ;

[0100] (6a3) Calculate the inter-line brightness gradient in the normal region And calculate the mean and variance:

[0101] ;

[0102] ;

[0103] ;

[0104] (6a4) Calculate the inter-row brightness gradient in the fringe region:

[0105] ;

[0106] in, .

[0107] The aforementioned edge structure constraints include the following steps:

[0108] (6b1) Apply structural edge constraints to the striped region:

[0109] , ,

[0110] , ;

[0111] in, This indicates that all pixel values ​​in the i-th row of the image have been recovered; This represents the constraint strength value; This represents the structural direction vector, used to define the direction of edge adjustment and determine the direction of pixel value correction;

[0112] (6b2) For the recovered image with the structural edge constraints completed, calculate the excess term of the main channel for each pixel within the laser stripe region. :

[0113] ;

[0114] The above-mentioned chromaticity suppression is achieved by suppressing the intensity coefficient. The laser stripe region is subjected to main channel attenuation suppression and bilateral filtering edge-preserving noise reduction sequentially, including the following steps:

[0115] (6c1) The main channel is suppressed based on the excess term of the main channel to obtain the suppressed main channel:

[0116] ;

[0117] (6c2) Perform bilateral filtering and edge-preserving denoising on the image after main channel suppression:

[0118] ;

[0119] in, For pixels The filtering neighborhood at that point For the filtering neighborhood Any pixel within pixel values, It is a Gaussian kernel function based on the Euclidean distance between pixels. It is a Gaussian kernel function based on the difference in color values ​​between pixels. This is the normalization coefficient.

[0120] In this embodiment, the image restoration in this step specifically involves completing all the restoration steps of image structure reconstruction, edge structure constraint, and chroma suppression in sequence, and then using the image after chroma suppression processing as the final restored image to complete the restoration process.

[0121] Example 2: Refer to Figure 2This embodiment proposes a real-time laser attack detection and recovery system that implements the method described in Embodiment 1. Specifically, it includes: a dual-threshold training module, an abnormal frame detection and localization module, a precise mask construction module, and a multi-dimensional image recovery module. The dual-threshold training module is used to obtain the Mahalanobis distance and inter-line brightness gradient as dual thresholds. The abnormal frame detection and localization module is used to identify abnormal frames and locate the laser stripe positions and affected color channels. The precise mask construction module is used to construct a laser mask based on the detection results. The multi-dimensional image recovery module is used to recover abnormal frames based on laser overlay. The working principle of the above modules in this embodiment is as follows:

[0122] The dual-threshold training module is beneficial for calculating the Mahalanobis distance of feature vectors and the inter-row brightness gradient using a small number of benign samples, and calculates the dual thresholds of feature vectors and inter-row brightness gradients based on preset percentiles.

[0123] The abnormal frame detection and localization module calculates the feature vector Mahalanobis distance and interline brightness gradient of the current frame. Based on the dual thresholds of the feature vector Mahalanobis distance and interline brightness gradient, it judges the current frame as abnormal and locates the range of the laser stripe area by the change in interline brightness gradient. It also determines the affected channel by the brightness change of each channel.

[0124] The precise mask construction module generates a precise laser mask based on the detection results of the abnormal frame detection and localization module, using the current frame and background frame, under the constraint of spatiotemporal consistency.

[0125] The multi-dimensional image frame restoration module performs image structure reconstruction, edge structure constraint, and chromaticity suppression operations on the current frame sequentially based on the laser interference mask, and outputs the restored normal frame.

[0126] Example 3: The overall implementation steps of the detection and recovery method proposed in this example are the same as in Example 1. A specific example is given below to further describe the specific implementation process of this invention:

[0127] Step 1) Determine the dual-threshold detection benchmark of feature vector Mahalanobis distance and inter-line brightness gradient using a small number of benign samples; in this embodiment, the dual-threshold detection benchmark of feature vector and inter-line brightness gradient is obtained according to the following steps:

[0128] 1a) Select N consecutive frames of benign sample input frames from the vehicle vision system. Each frame of the image is sampled, and the mean value of each channel is calculated. , , The calculation formula is:

[0129] ,

[0130] ,

[0131] ;

[0132] in , , These are the RGB channels. This refers to the image size.

[0133] 1b) Calculate the inter-frame channel difference between adjacent benign sample frames. , , The calculation formula is:

[0134] ,

[0135] ,

[0136] ;

[0137] in Let be the channel mean of frame t. The channel mean of the (t-1)th frame.

[0138] 1c) Calculate the inter-frame brightness variation of adjacent benign sample frames. The calculation formula is:

[0139] ;

[0140] in Let be the grayscale value of the t-th frame. is the grayscale value of the (t-1)th frame.

[0141] 1d) Calculate the maximum channel difference The calculation formula is:

[0142] ;

[0143] 1e) Calculate color deviation The calculation formula is:

[0144] ;

[0145] in .

[0146] 1f) Calculate normalized chromaticity The calculation formula is:

[0147] ;

[0148] in It is a non-zero decimal, used to prevent division by zero.

[0149] 1g) Calculate the normalized colorimetric change , , The calculation formula is:

[0150] ,

[0151] ,

[0152] ;

[0153] in , .

[0154] 1h) to obtain the final feature vector :

[0155] ;

[0156] 1i) Calculate the inter-line brightness gradient between adjacent lines of a benign frame:

[0157] ;

[0158] in This represents the average brightness of the row.

[0159] 1j) Calculate the interline brightness gradient of benign frames. The calculation formula is:

[0160] ;

[0161] 1k) Calculate the Mahalanobis distance of the eigenvectors. :

[0162] ;

[0163] in

[0164] ,

[0165] , The regularization coefficient is . identity matrix

[0166] 1l) Determine the Mahalanobis distance threshold and interline brightness gradient threshold :

[0167] ,

[0168] ;

[0169] in The Mahalanobis distance quantile, This represents the inter-line brightness gradient quantile.

[0170] Step 2) Calculate the Mahalanobis distance and inter-line brightness gradient of the feature vector for the current input frame, and perform abnormal frame image detection based on the dual threshold detection benchmark, and locate the position of the laser grating stripes and the affected color channels; specifically according to the following steps, including:

[0171] 2a) Calculate the feature vector of the current input frame of the vehicle vision system Calculate Mahalanobis distance :

[0172] ;

[0173] 2b) Calculate the interline brightness gradient of the current input frame. :

[0174] ;

[0175] 2c) A frame is considered abnormal when the following two conditions are met:

[0176] ;

[0177] 2d) If condition 2c) is not met, then update the background frame. For the current frame:

[0178] = ;

[0179] 2e) Calculate the inter-line brightness gradient transformation of abnormal frames. :

[0180] ;

[0181] Where L(i+1) is the average brightness of the (i+1)th row and L(i) is the average brightness of the ith row.

[0182] 2f) Based on the inter-line brightness gradient transformation, locate the starting line of the laser stripe in the abnormal frame. and end line :

[0183] The i-th row is the starting row. ,

[0184] The i-th row is the end row. ;

[0185] 2g) Determine the range of laser stripes in abnormal frames :

[0186] ;

[0187] in .

[0188] 2h) within the spatial range of the laser grating stripes Inside, the channel most severely affected by laser interference was selected as the main channel. The formula is:

[0189] ;

[0190] Where c=0,1,2 correspond to channels B, G, and R, respectively.

[0191] Step 3) Based on the detection results, generate a precise laser interference mask under spatiotemporal consistency constraints; the steps for generating the precise laser mask are as follows:

[0192] 3a) Calculate the overexposure mask of the current abnormal frame. :

[0193] ;

[0194] in, and These are the first non-main channel and the second non-main channel, respectively. For overexposure parameters, The overexposure difference; Indicates an indicator function; For logical OR operator; Indicates the coordinates of the current abnormal frame. Pixel value at;

[0195] 3b) Calculate the background frame overexposure mask ;

[0196] ;

[0197] in, Indicates the background frame at coordinates Pixel value at;

[0198] 3c) Calculate the brightness comparison mask between the current abnormal frame and the background frame. :

[0199] ;

[0200] in, Indicates the current abnormal frame's luminance channel at coordinates Pixel value at; Indicates the background frame luminance channel in coordinates The pixel value at that location.

[0201] 3d) Calculate the horizontal projection :

[0202] ;

[0203] 3e) Calculate the vertical projection :

[0204] ;

[0205] 3f) Perform median filtering and Gaussian smoothing on the projection:

[0206] ,

[0207] ,

[0208] ,

[0209] ;

[0210] Where k is the preset kernel size for median filtering. Standard deviation Gaussian kernel.

[0211] 3g) Constructing a two-dimensional laser mask using horizontal and vertical projections. :

[0212] ;

[0213] 3h) Reconstructing a two-dimensional laser mask under spatial consistency constraints :

[0214] ;

[0215] in , It is a non-zero extreme small number.

[0216] 3i) Updating the 2D laser mask under time consistency constraints :

[0217] ;

[0218] in Let be the cumulative mask for frame t. The cumulative mask for the (t-1)th frame. A two-dimensional mask reconstructed for the current frame. For smoothing parameters.

[0219] Step 4): Abnormal frame image restoration is performed based on a laser interference mask, sequentially completing image structure reconstruction, edge structure constraint, and chromaticity suppression; the specific steps are as follows:

[0220] 4a) Subtract the laser mask from the aberration frame to obtain the initial restored image. :

[0221] ;

[0222] 4b) Apply overexposure area protection:

[0223] ;

[0224] 4c) Trim to the effective range:

[0225] ;

[0226] 4d) Calculate the inter-row brightness gradient in the normal region. And calculate the mean and variance:

[0227] ,

[0228] ,

[0229] ;

[0230] 4e) Calculate the inter-row brightness gradient in the fringe region:

[0231] ;

[0232] in, .

[0233] 4f) Apply structural edge constraints to the striped region:

[0234] , ,

[0235] , ;

[0236] 4g) For the recovered image with completed structural edge constraints, calculate the excess term of the main channel for each pixel within the laser stripe region. :

[0237] ;

[0238] 4h) Attenuation suppression of the main channel is performed based on the excess term of the main channel:

[0239] ;

[0240] in The suppression intensity coefficient;

[0241] 4i) Perform bilateral filtering and edge-preserving denoising on the image after main channel suppression:

[0242] ;

[0243] in, For pixels The filtering neighborhood at that point For the filtering neighborhood Any pixel within pixel values, It is a Gaussian kernel function based on the Euclidean distance between pixels. It is a Gaussian kernel function based on the difference in color values ​​between pixels. This is the normalization coefficient.

[0244] 4j) Obtain the final restored image :

[0245] ;

[0246] (4k) Update the background frame to the restored image:

[0247] = .

[0248] The effects of the present invention will be further explained below with reference to simulation experiments.

[0249] 1. Simulation conditions:

[0250] This invention employs computer simulation experiments to verify its technical effectiveness. The experimental hardware environment consists of an AMD Ryzen 7 5800H processor and an NVIDIA GeForce RTX 3060 graphics card. The software environment is based on a Windows operating system, with the algorithm developed using the Python programming language. It utilizes the PyTorch deep learning framework and incorporates the torchvision and OpenCV image processing libraries to perform tests related to anomaly detection and image restoration.

[0251] 2. Simulation content:

[0252] This invention uses the Udacity dataset as experimental data. A small number of benign samples are selected from this dataset to form a training set. From the remaining samples, multiple sets of samples with different acquisition environments are selected as test sets. Each test set consists of continuously acquired video frames. Based on the real-world laser stripe patterns, each test set image is divided into interference-free normal frames and abnormal frames with superimposed laser stripe interference. The training set is input into a dual-threshold training module to obtain the corresponding dual-threshold benchmark. Then, each test set is input into the system described in this invention. The abnormal frame detection and localization module identifies the abnormal frames and locates the abnormal regions, and then restores the identified abnormal frames. Figure 3 shows the selected representative abnormal frames, and Figure 4 shows the abnormal frames after restoration.

[0253] 3. Simulation results:

[0254] In this embodiment, PatchSVDD, Patchcore, and FAIR are selected as anomaly detection control algorithms, and AUROC (including Image ROC and Pixel ROC) and processing time are used as anomaly detection evaluation metrics. Restormer, Uformer, and NAFNet are selected as anomaly recovery control algorithms, and PSNR, SSIM, mAP50, and processing time are used as anomaly recovery evaluation metrics. Figure 5 The results show the comparison of anomaly detection algorithms. Figure 6 The results show the comparison of anomaly recovery algorithms.

[0255] Depend on Figure 5 As can be seen, compared with the PatchSVDD, Patchcore and FAIR comparison algorithms, the method described in this invention achieves the best results in both Image ROC and Pixel ROC metrics, and has higher accuracy in detecting abnormal frames. At the same time, the processing time per frame of this invention is significantly lower than that of the comparison algorithms, and the detection efficiency is better. It can effectively improve the detection speed while ensuring detection accuracy, and the overall detection performance is more outstanding.

[0256] Depend on Figure 6 As can be seen, the anomaly recovery method described in this invention outperforms Restormer, Uformer, and NAFNet in terms of mAP50, achieving higher image recovery accuracy and providing a more reliable data foundation and safety guarantee for subsequent autonomous driving-related vision tasks. The SSIM index remains at a high level, indicating good structural integrity of the recovered image. Although the PSNR index is slightly lower than some comparative algorithms, it still meets the needs of practical applications. In terms of processing efficiency, the single-frame processing time of this invention is only 0.060s, far lower than the comparative algorithms, enabling rapid recovery of abnormal images while balancing recovery accuracy and real-time performance, resulting in superior overall recovery performance.

[0257] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0258] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A method for real-time detection and recovery of laser attacks for vehicle-mounted vision systems, characterized in that, Includes the following steps: (1) Select N consecutive frames of benign samples, with the input frame being That is, a 3-channel image with height H and width W; after sampling each frame, a 9-dimensional feature vector is obtained. Simultaneously calculate the inter-line brightness gradient of benign frames. : ， ； in, , , This represents the RGB channel difference between adjacent benign samples; Indicates the difference between the brightness channels; Indicates the maximum channel difference; Indicates color deviation; Represents the normalized chromaticity variation values ​​of the RGB channels; Indicates the transpose operation; Max represents the inter-line brightness gradient of the i-th group of adjacent lines in a benign frame. i This indicates taking the maximum gradient value across all rows; (2) Calculate the eigenvectors Mahalanobis distance And based on benign samples Distribution determines Mahalanobis distance threshold Meanwhile, based on benign samples Distribution determines inter-line brightness gradient threshold ; and by and To jointly construct a dual-threshold detection benchmark; (3) For the current input frame to be detected by the vehicle vision system, repeat the feature extraction process in step (1), calculate the Mahalanobis distance and inter-line brightness gradient of the feature vector of the frame, and perform abnormal frame image detection based on the dual threshold detection benchmark, including... and contrast, and The following judgment is made based on the comparison: If and If it is not, it is determined to be an abnormal frame; otherwise, the background frame is updated. For the current frame, that is, let = ; (4) Locate the position of the abnormal frame laser grating stripes and the affected color channels to obtain the detection results. The steps are as follows: (4.1) Calculate the interline brightness gradient transformation of abnormal frames And use it to locate the starting line of the laser grating stripes in the abnormal frame. and end line : like Then the i-th row will be recorded as the starting row. , like Then the i-th row will be recorded as the end row. ; (4.2) Determine the spatial range of the laser grating fringes in the abnormal frame Within this range, the channel most severely affected by laser interference is selected as the main channel. : ， ； in, c=0,1,2 correspond to channels B, G, and R respectively; (5) Based on the detection results, calculate the overexposure mask of the current abnormal frame in sequence. Background frame overexposure mask Luminance comparison mask between the current abnormal frame and the background frame Noise in the non-interference area is filtered out by a triple mask; an initial two-dimensional laser mask is constructed using horizontal and vertical projections. And under spatial consistency constraints, for Perform normalization reconstruction: ； in This indicates vertical projection synthesis. It is a non-zero extreme small number; Under the constraint of temporal consistency, the mask is updated through inter-frame cumulative smoothing to obtain an accurate laser interference mask. : ； in Let be the cumulative mask for frame t. For the cumulative mask of frame t-1, A two-dimensional mask for reconstructing the current frame. Indicates time smoothing parameters; (6) Based on the precise laser interference mask, abnormal frame image restoration is performed, and image structure reconstruction, edge structure constraint and color suppression are completed in sequence to complete image restoration.

2. The method according to claim 1, characterized in that: The color deviation mentioned in step (1) Based on the RGB channel difference between adjacent benign samples , , Specifically, calculate according to the following formula: ； in .

3. The method according to claim 1, characterized in that: The feature vector in step (2) Mahalanobis distance The result is obtained by calculation using the following formula: ； in, , , The regularization coefficient is . It is an identity matrix.

4. The method according to claim 1, characterized in that: Step (5) The current abnormal frame overexposure mask Background frame overexposure mask Luminance comparison mask between the current abnormal frame and the background frame Specifically, it is calculated according to the following steps: (5.1) Calculate the overexposure mask of the current abnormal frame. : ； in, and These are the first non-main channel and the second non-main channel, respectively. For overexposure parameters, The overexposure difference; Indicates an indicator function; For logical OR operator; Indicates the coordinates of the current abnormal frame. Pixel value at; (5.2) Calculate the overexposure mask of the background frame ; ； in, Indicates the background frame at coordinates Pixel value at; (5.3) Calculate the brightness comparison mask between the current abnormal frame and the background frame. : ； in, Indicates the current abnormal frame's brightness channel at coordinates Pixel value at; Indicates the background frame luminance channel in coordinates The pixel value at that location.

5. The method according to claim 4, characterized in that: Step (5) involves constructing an initial two-dimensional laser mask using horizontal and vertical projections. The implementation steps are as follows: (5.4) Calculate the horizontal projection : ； (5.5) Calculate the vertical projection : ； (5.6) Perform median filtering and Gaussian smoothing on the projection: ， ， ， ； Where k is the preset kernel size for median filtering. Standard deviation Gaussian kernel; (5.7) Construct a two-dimensional laser mask using horizontal and vertical projections. : ， in , It is a non-zero extreme small number.

6. The method according to claim 1, characterized in that: Step (6) image structure reconstruction, used to restore the basic structure of the image damaged by the laser, includes the following steps: (6a1) Subtract the laser mask from the aberration frame to obtain the initial restored image. : ； (6a2) Apply overexposed area protection to crop the image to the effective area: ； (6a3) Calculate the inter-line brightness gradient in the normal region And calculate the mean and variance: ； ； ； (6a4) Calculate the inter-row brightness gradient in the fringe region: ； in, .

7. The method according to claim 6, characterized in that: The edge structure constraint in step (6) includes the following steps: (6b1) Apply structural edge constraints to the striped region: , ， , ； in, This indicates that all pixel values ​​in the i-th row of the image have been recovered; This represents the constraint strength value; This represents the structural direction vector, used to define the direction of edge adjustment and determine the direction of pixel value correction; (6b2) For the recovered image with the structural edge constraints completed, calculate the excess term of the main channel for each pixel within the laser stripe region. : 。 8. The method according to claim 7, characterized in that: The chromaticity suppression described in step (6) is achieved by suppressing the intensity coefficient. The laser stripe region is subjected to main channel attenuation suppression and bilateral filtering edge-preserving noise reduction sequentially, including the following steps: (6c1) Attenuation suppression of the main channel based on the excess term of the main channel: ； (6c2) Perform bilateral filtering and edge-preserving denoising on the image after main channel suppression: ； in, For pixels The filtering neighborhood at that point For the filtering neighborhood Any pixel within pixel values, It is a Gaussian kernel function based on the Euclidean distance between pixels. It is a Gaussian kernel function based on the difference in color values ​​between pixels. This is the normalization coefficient.

9. The method according to claim 8, characterized in that: The image restoration described in step (6) is specifically achieved by using the image after color suppression as the final restored image after completing all the restoration steps of image structure reconstruction, edge structure constraint and color suppression in sequence, and updating the background frame to the restored image to complete the restoration.

10. A real-time laser attack detection and recovery system implementing the method of claim 1, characterized in that, include: The module includes a dual-threshold training module, an abnormal frame detection and localization module, a precise mask construction module, and a multi-dimensional image restoration module. The dual-threshold training module is useful for calculating the Mahalanobis distance of feature vectors and the inter-row brightness gradient using a small number of benign samples, and calculates the dual thresholds of feature vectors and inter-row brightness gradients based on preset percentiles. The abnormal frame detection and localization module calculates the Mahalanobis distance and interline brightness gradient of the feature vector of the current frame, judges the current frame as abnormal based on the dual thresholds of the Mahalanobis distance and interline brightness gradient, locates the range of the laser stripe area by the change of interline brightness gradient, and determines the affected channel by the brightness change of each channel. The precise mask construction module generates a precise laser mask based on the detection results of the abnormal frame detection and localization module, and on the current frame and background frame, under the condition of spatiotemporal consistency constraints. The multi-dimensional image restoration module performs image structure reconstruction, edge structure constraint, and chromaticity suppression operations on the current frame sequentially based on the laser interference mask, and outputs the restored normal frame.

Images