Image data integrity detection method and device

Abstract

The invention discloses an image data integrity detection method and device. The method comprises the following steps: step 1, acquiring an image frame; 2, calculating the color variance of each local part of the image frame by using a sliding window, and calculating the statistical magnitude isanomallyratio of the variance; 3, calculating the edge energy of each row of the image, and calculating the statistical magnitude edgejumpratio of the edge energy; and step 4, carrying out joint discrimination by using isanomaloratio and edgejumpratio, and detecting whether the image is abnormal or not.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image data integrity detection method and apparatus. Background Technology

[0002] Camera / network video systems may experience decoding errors or data corruption during transmission or encoding, manifesting as bottom color blocks, horizontal stripes, color banding, blocky noise, or abrupt blurry areas. Existing detection methods suffer from both false negatives and false negatives.

[0003] In view of this, this invention is hereby proposed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an image data integrity detection method and apparatus for identifying abnormalities in image data (such as decoding errors, transmission interruptions, data packet loss, cable interference, etc.), which has a good accuracy rate for normal image detection and an abnormal image recognition rate.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides an image data integrity detection method, comprising the following steps:

[0007] Step 1: Acquire image frames;

[0008] Step 2: Using a sliding window, calculate the color variance of each local area of ​​the image frame, and calculate the variance statistic is_anomaly_ratio;

[0009] Step 3: Calculate the edge energy of each row of the image, and calculate the edge energy statistic edge_jump_ratio;

[0010] Step 4: Use is_anomaly_ratio and edge_jump_ratio for joint discrimination to detect whether the image is abnormal.

[0011] Furthermore, step 1 includes the following steps:

[0012] Step 1.1: Obtain the RGB image;

[0013] Step 1.2: Crop and scale the RGB image;

[0014] Step 1.3: Convert the preprocessed RGB image to HSV or YUV.

[0015] Furthermore, step 2 includes the following steps:

[0016] Step 2.1: Using a sliding window, calculate the color variance of each local area of ​​the RGB image according to a preset path and a preset step size. The formula for calculating variance is as follows:

[0017] ;

[0018] In the formula, W is the width of the sliding window, Wi,j is the number of pixels within the sliding window, I(u,v) is the channel value of each pixel within the sliding window, and μ Wi,j This represents the average channel values ​​of the pixels within the sliding window.

[0019] Step 2.2: Use the color variance of each local area of ​​the RGB image Construct a variance plot;

[0020] Step 2.3: Normalize the values ​​of each pixel in the variance map, and then perform Gaussian blurring or mean filtering.

[0021] Step 2.4: Set the value of pixels with a value greater than the preset threshold var_threshold to 1, and set the value of pixels with a value less than or equal to the preset threshold var_threshold to 0, and construct a binary heatmap;

[0022] Step 2.5: Calculate is_anomaly_ratio, which is the ratio of the number of pixels in the variance map whose value is greater than the preset threshold var_threshold to the total number of pixels.

[0023] Furthermore, step 3 includes the following steps:

[0024] Step 3.1: Calculate the horizontal gradient matrix G of the image using the Sobel operator. x and the gradient matrix G in the vertical direction y ;

[0025] Step 3.2: Convert matrix G x And matrix G y The gradient energy matrix G of the image is obtained by summing the results;

[0026] Step 3.3: Sum the matrix G row by row to obtain the energy vector E(r) with a length equal to the number of rows in the image;

[0027] Step 3.4: Normalize the values ​​of each element of E(r) and perform one-dimensional smoothing.

[0028] Step 3.5: Calculate edge_jump_ratio using the following formula:

[0029] ;

[0030] In the formula, max(E) is the maximum value in the energy vector E(r), mean(E[0:h / 2]) is the average value of the values ​​in the energy vector from row 0 to h / 2, and ε is a very small positive number.

[0031] Furthermore, step 4 includes the following steps:

[0032] Step 4.1: Determine whether edge_jump_ratio is greater than the preset threshold high_energy_threshold. If it is, it is normal; otherwise, proceed to step 4.2.

[0033] Step 4.2: Determine whether edge_jump_ratio is less than the preset threshold low_energy_threshold. If not, proceed to step 4.3; otherwise, proceed to step 4.4.

[0034] Step 4.3: Determine whether is_anomaly_ratio is greater than the preset threshold high_var_threshold. If not, it is normal; if so, it is an abnormal structural damage.

[0035] Step 4.4: Determine whether is_anomaly_ratio is less than the preset threshold var_ratio_threshold. If it is, it is a structural damage anomaly; otherwise, it is a conservative anomaly.

[0036] Furthermore, it also includes the following steps:

[0037] Step A5: Perform short-term fusion using a sliding window; if at least two frames show abnormalities, an alarm will be triggered.

[0038] Furthermore, it also includes the following steps:

[0039] Step B5: If an anomaly occurs in N consecutive frames, the maintenance process is triggered.

[0040] Furthermore, the preset threshold is calculated through the following steps:

[0041] Step C1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T ;

[0042] Step C2: Calculate the eigenvalues ​​x1, x2, ..., x T mean μ T and standard deviation σ T ;

[0043] Step C3: Calculate the preset threshold using the following formula:

[0044] threshold=μT +kσ T ;

[0045] In the formula, k is a constant.

[0046] Furthermore, the preset threshold is calculated through the following steps:

[0047] Step D1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T ;

[0048] Step D2: Calculate the preset threshold using the following quantile formula:

[0049] threshold=quantile(x1,x2,...,x T ).

[0050] In a second aspect, the present invention provides an image data integrity detection device, comprising a processor and a memory, wherein the memory stores an image data integrity detection program, and when the image data integrity detection program is executed by the processor, it implements the image data integrity detection method as described in any one of the first aspects of the present invention.

[0051] Compared with the prior art, the beneficial effects of this invention are as follows:

[0052] 1. Anomalies in image frames are jointly determined based on the local color variance statistic is_anomaly_ratio (local structural consistency) and the edge energy statistic edge_jump_ratio (vertical energy continuity) of each row. Local color variance can capture stripes, color bands, and block noise, while edge energy of each row can capture energy breaks and abrupt changes; the two are complementary.

[0053] 2. It features low computational complexity, facilitating real-time deployment. Suitable for real-time video surveillance and industrial production line inspection. Attached Figure Description

[0054] Figure 1 A flowchart of an image data integrity detection method;

[0055] Figure 2 This is a flowchart of step 4 in an image data integrity detection method. Detailed Implementation

[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0057] Example 1:

[0058] An image data integrity detection method, such as Figure 1 As shown, it includes the following steps:

[0059] Step 1: Acquire image frames.

[0060] Specifically, step 1 includes the following steps:

[0061] Step 1.1: Obtain the RGB image.

[0062] In this embodiment, the RGB image comes from the camera or the decoded video frame, and the resolution can be 2560×1440 or 1280×720.

[0063] Step 1.2: Crop and scale the RGB image.

[0064] In this embodiment, the RGB image can be cropped by removing the watermark area or fixing irrelevant areas on the side to improve computational efficiency; for RGB images wider than 2000 pixels, downsampling can be performed at a ratio of 1 / 2 or 1 / 4 to meet real-time performance, while ensuring that vertical structural information is preserved during downsampling.

[0065] Step 1.3: Convert the preprocessed RGB image to HSV or YUV.

[0066] In this embodiment, conversion to HSV allows the use of the V or S channel, which can reflect color stripes; conversion to YUV allows the use of the Y channel, which can reflect brightness gradients.

[0067] Step 2: Using a sliding window, calculate the color variance of each local area of ​​the image frame, and calculate the variance statistic is_anomaly_ratio.

[0068] Specifically, step 2 includes the following steps:

[0069] Step 2.1: Using a sliding window, calculate the color variance of each local area of ​​the RGB image according to a preset path and a preset step size. The formula for calculating variance is as follows:

[0070] ;

[0071] In the formula, W is the width of the sliding window, Wi,j is the number of pixels within the sliding window (which can be 64 or 32), and I(u,v) is the channel value of each pixel within the sliding window. When converting to HSV, the variances of the V and S channel values ​​of each pixel within the sliding window are calculated separately, and then the mean or the maximum of the two values ​​is taken. When converting to YUV, the variance of the Y channel values ​​of each pixel within the sliding window is calculated, μ... Wi,j This represents the average channel value of the pixels within the sliding window.

[0072] In this embodiment, the preset step size can be the width of the sliding window.

[0073] Step 2.2: Use the color variance of each local area of ​​the RGB image Construct a variance plot.

[0074] In this embodiment, the value of each pixel in the variance map is the local color variance calculated using a sliding window. .

[0075] Step 2.3: Normalize the values ​​of each pixel in the variance map, and then perform Gaussian blurring or mean filtering.

[0076] In this embodiment, Gaussian blurring or mean filtering is performed to reduce isolated noise.

[0077] Step 2.4: Set the value of pixels with a value greater than the preset threshold var_threshold to 1, and set the value of pixels with a value less than or equal to the preset threshold var_threshold to 0, and construct a binary heatmap.

[0078] In this embodiment, when an anomaly occurs, the region with a value of 1 in the binary heatmap can be outlined, marked as an anomaly region, and the vertex coordinates of the anomaly region can be output.

[0079] Step 2.5: Calculate is_anomaly_ratio, which is the ratio of the number of pixels in the variance map whose value is greater than the preset threshold var_threshold to the total number of pixels.

[0080] Step 3: Calculate the edge energy of each row of the image and calculate the edge energy statistic edge_jump_ratio.

[0081] Specifically, step 3 includes the following steps:

[0082] Step 3.1: Calculate the horizontal gradient matrix G of the image using the Sobel operator. x and the gradient matrix G in the vertical direction y .

[0083] In this embodiment, the gradient matrix G of the image in the horizontal direction can be calculated using the following formula. x and the gradient matrix G in the vertical direction y :

[0084] ;

[0085] ;

[0086] In the formula, I is the input image matrix.

[0087] In this embodiment, matrix G can be calculated using a preprocessed grayscale image or the Y channel in YUV. x And matrix G y .

[0088] Step 3.2: Convert matrix G x And matrix G y The gradient energy matrix G of the image is obtained by adding them together.

[0089] In this embodiment, the gradient energy matrix G of the image can be calculated using the following formula:

[0090] .

[0091] Step 3.3: Sum the matrix G row by row to obtain the energy vector E(r) with a length equal to the number of rows in the image.

[0092] In this embodiment, the energy vector E(r) can be calculated using the following formula:

[0093] ;

[0094] In the formula, r and c are the row index and column index, respectively.

[0095] Step 3.4: Normalize the values ​​of each element of E(r) and perform one-dimensional smoothing.

[0096] In this embodiment, when an anomaly occurs, rows with values ​​greater than k*mean(E[0:h / 2]) can be marked as abnormal rows, where mean(E[0:h / 2]) is the average value of the values ​​in rows 0 to h / 2 of the energy vector.

[0097] In this embodiment, high-frequency jitter is removed through one-dimensional smoothing. Specifically, this can be achieved by using a moving average or a small... Gaussian smoothing is applied.

[0098] Step 3.5: Calculate edge_jump_ratio using the following formula:

[0099] ;

[0100] In the formula, max(E) is the maximum value in the energy vector E(r), mean(E[0:h / 2]) is the average value of the values ​​in the energy vector from row 0 to row h / 2, and ε is a very small positive number to prevent the denominator from being 0.

[0101] Damaged frames typically produce sudden large peaks in certain rows or have breaks at the descent points, causing edge_jump_ratio to deviate significantly from the normal distribution. Anomalies are usually concentrated in smaller value ranges, and if the graph is blurred, the total energy decreases.

[0102] Step 4: Use is_anomaly_ratio and edge_jump_ratio for joint discrimination to detect whether the image is abnormal.

[0103] Specifically, such as Figure 2 As shown, step 4 includes the following steps:

[0104] Step 4.1: Determine whether edge_jump_ratio is greater than the preset threshold high_energy_threshold. If it is, it is normal; otherwise, proceed to step 4.2.

[0105] In this embodiment, edge_jump_ratio represents the row edge energy jump ratio (e.g., the degree of Sobel energy jump). If edge_jump_ratio is greater than high_energy_threshold, it means that: the image texture still exists; the physical edges (e.g., workpiece boundaries, mechanical structures) are still clear; and the transmission link has not caused significant "blurring or energy collapse". In other words, it means that the image structure is intact.

[0106] Step 4.2: Determine whether edge_jump_ratio is less than the preset threshold low_energy_threshold. If not, proceed to step 4.3; otherwise, proceed to step 4.4.

[0107] In this embodiment, if edge_jump_ratio is less than low_energy_threshold, it indicates that: the image may be darkened overall; image boundaries are lost; half of the image is black or severely blurred; or there may be a camera or decoding malfunction. At this point, we need to further examine another dimension of the feature: local variance is_anomaly_ratio.

[0108] Step 4.3: Determine whether is_anomaly_ratio is greater than the preset threshold high_var_threshold. If not, it is normal; if so, it is an abnormal structural damage.

[0109] In this embodiment, is_anomaly_ratio represents the proportion of low-variance pixels (the proportion of locally homogenized regions). If is_anomaly_ratio is greater than high_var_threshold, it indicates that a large area has "structure disappearance." This often corresponds to: black images; stripes; collapse; large-area encoder block loss.

[0110] Step 4.4: Determine whether is_anomaly_ratio is less than the preset threshold var_ratio_threshold. If it is, it is a structural damage anomaly; otherwise, it is a conservative anomaly.

[0111] In this embodiment, if is_anomaly_ratio is less than low_var_threshold, it indicates a single region, suggesting that the image exhibits "overall grayness," "overall blackness," or "overall flatness," thus belonging to conservative anomalies, such as compression encoding errors, decoder stuttering, or slight blurring within a small area. Because the energy is extremely low, it is highly suspicious, but the variance may not fully meet the strict anomaly conditions; to avoid missed detections, a conservative strategy can be adopted.

[0112] In this embodiment, by setting different preset thresholds, the settings can be flexibly configured between "better to kill the innocent than let the guilty go free" and "false alarms are controllable".

[0113] In this embodiment, when an anomaly occurs, an alarm package containing the original image, binary heatmap, energy curve, and decision reason can be output.

[0114] This embodiment of an image data integrity detection method jointly determines whether an image frame has anomalies based on the local color variance statistic is_anomaly_ratio (local structural consistency) and the edge energy statistic edge_jump_ratio (vertical energy continuity) of each row. The local color variance can capture stripes, color bands, and block noise, while the edge energy of each row can capture energy breaks and abrupt changes; the two are complementary.

[0115] This embodiment of the image data integrity detection method features low computational complexity and facilitates real-time deployment. It is suitable for real-time video surveillance and industrial production line inspection.

[0116] In an optional embodiment, the image data integrity detection method of this embodiment further includes the following steps:

[0117] Step A5: Perform short-term fusion using a sliding window; if at least two frames show abnormalities, an alarm will be triggered.

[0118] In this optional embodiment, the sliding window can be 3 to 5 frames. Steps 1 to 4 are performed on each frame within the sliding window to determine whether the image of each frame is abnormal. If at least 2 frames are abnormal, an alarm is triggered.

[0119] In this optional embodiment, short-term fusion via a sliding window can reduce false alarms caused by sporadic noise while ensuring that critical anomalies (that occur continuously) can be captured.

[0120] In an optional embodiment, the image data integrity detection method of this embodiment further includes the following steps:

[0121] Step B5: If an anomaly occurs in N consecutive frames, the maintenance process is triggered.

[0122] In this optional embodiment, steps 1 to 4 are executed consecutively to determine whether each frame is abnormal. If there are abnormalities in N consecutive frames, the maintenance process is triggered.

[0123] In an optional embodiment, an image data integrity detection method of this embodiment calculates a preset threshold by means of the following steps:

[0124] Step C1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T .

[0125] Step C2: Calculate the eigenvalues ​​x1, x2, ..., x T mean μ T and standard deviation σ T .

[0126] Step C3: Calculate the preset threshold using the following formula:

[0127] threshold=μ T +kσ T ;

[0128] In the formula, k is a constant.

[0129] In an optional embodiment, an image data integrity detection method of this embodiment calculates a preset threshold by means of the following steps:

[0130] Step D1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T .

[0131] Step D2: Calculate the preset threshold using the following quantile formula:

[0132] threshold=quantile(x1,x2,...,x T ).

[0133] In the above optional embodiments, the preset thresholds used to calculate one or more of the parameters involved in this embodiment include: high_energy_threshold, low_energy_threshold, high_var_threshold, var_ratio_threshold, and feature values ​​x1, x2, ..., x TThe value of the feature corresponding to the relevant preset threshold.

[0134] The preset threshold is calculated using the above optional embodiments. When the feature value dispersion of consecutive T frames is large, the preset threshold automatically widens, and vice versa, thereby achieving stability processing.

[0135] In practical implementation, the hardware options are: a regular inference server or an embedded industrial PC; for real-time processing of 2K resolution and multiple frames per second, it is recommended to use a CPU or GPU accelerated with SSE / AVX instructions (OpenCV + CUDA). The software options are: OpenCV (C++ / Python), NumPy, and scikit-learn; for online training, scikit-learn can be used. Parallelization: Variance windows and Sobel can be computed in parallel, fully utilizing multi-core processors; convolution operations can also be implemented on GPUs (CUDA / Metal / OpenCL). The interface provides gRPC / HTTP interfaces to output alarms, heatmaps, and metadata; and connects to the camera SDK or RTSP plugin to acquire frame data.

[0136] To verify the effectiveness of the image data integrity detection method proposed in this implementation, an industrial monitoring dataset with a resolution of 1920×1080 was used. This dataset contained 239 normal frames and 44 abnormal frames. The abnormal frames included: image transmission anomalies (such as missing frames, line synchronization errors, and half-screen blackouts); noise anomalies (such as block noise, local pixel drift, and edge jumps); and compression and decoding anomalies (such as color channel errors, low-frequency response anomalies, and sudden drops in overall image energy). The Sophon SE8288-EA6-72 microserver was used, equipped with a Sophon fourth-generation processor BM1684X, a domestic x86 processor: 2.8GHz, 8 cores and 16 threads, 16MB L3 cache, and came standard with two 32GB memory modules and a 1TB SSD system disk.

[0137] The preset threshold values ​​are as follows:

[0138] high_energy_threshold=8;

[0139] low_energy_threshold=3.0;

[0140] high_var_threshold = 0.12;

[0141] var_ratio_threshold = 0.03.

[0142] The test results are as follows:

[0143] Normal image detection accuracy: 97.5%;

[0144] Abnormal image recognition rate: 95.3%;

[0145] Processing time: approximately 12 ms / frame, can run in real time.

[0146] It should be noted that, in order to evaluate the single-frame detection performance of the algorithm, temporal sliding window, linear discriminator or other post-processing methods were not enabled during the verification.

[0147] Example 2:

[0148] An image data integrity detection device includes a processor and a memory. The memory stores an image data integrity detection program. When the image data integrity detection program is executed by the processor, it implements the image data integrity detection method as described in any one of Embodiment 1.

[0149] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting image data integrity, characterized in that, Includes the following steps: Step 1: Acquire image frames; Step 2: Using a sliding window, calculate the color variance of each local area of ​​the image frame, and calculate the variance statistic is_anomaly_ratio; Step 3: Calculate the edge energy of each row of the image and calculate the edge energy statistic edge_jump_ratio; Step 4: Use is_anomaly_ratio and edge_jump_ratio for joint discrimination to detect whether the image is abnormal.

2. The image data integrity detection method according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: Obtain the RGB image; Step 1.2: Crop and scale the RGB image; Step 1.3: Convert the preprocessed RGB image to HSV or YUV.

3. The image data integrity detection method according to claim 1, characterized in that, Step 2 includes the following steps: Step 2.1: Using a sliding window, calculate the color variance of each local area of ​​the RGB image according to a preset path and a preset step size. The formula for calculating variance is as follows: ； In the formula, W is the width of the sliding window, Wi,j is the number of pixels within the sliding window, I(u,v) is the channel value of each pixel within the sliding window, and μ Wi,j This represents the average channel values ​​of the pixels within the sliding window. Step 2.2: Use the color variance of each local area of ​​the RGB image Construct a variance plot; Step 2.3: Normalize the values ​​of each pixel in the variance map, and then perform Gaussian blurring or mean filtering. Step 2.4: Set the value of pixels with a value greater than the preset threshold var_threshold to 1, and set the value of pixels with a value less than or equal to the preset threshold var_threshold to 0, and construct a binary heatmap; Step 2.5: Calculate is_anomaly_ratio, which is the ratio of the number of pixels in the variance map whose value is greater than the preset threshold var_threshold to the total number of pixels.

4. The image data integrity detection method according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Calculate the horizontal gradient matrix G of the image using the Sobel operator. x and the gradient matrix G in the vertical direction y ; Step 3.2: Convert matrix G x And matrix G y The gradient energy matrix G of the image is obtained by summing the results; Step 3.3: Sum the matrix G row by row to obtain the energy vector E(r) with a length equal to the number of rows in the image; Step 3.4: Normalize the values ​​of each element of E(r) and perform one-dimensional smoothing. Step 3.5: Calculate edge_jump_ratio using the following formula: ； In the formula, max(E) is the maximum value in the energy vector E(r), mean(E[0:h / 2]) is the average value of the values ​​in the energy vector from row 0 to h / 2, and ε is a very small positive number.

5. The image data integrity detection method according to claim 1, characterized in that, Step 4 includes the following steps: Step 4.1: Determine whether edge_jump_ratio is greater than the preset threshold high_energy_threshold. If it is, it is normal; otherwise, proceed to step 4.

2. Step 4.2: Determine whether edge_jump_ratio is less than the preset threshold low_energy_threshold. If not, proceed to step 4.3; otherwise, proceed to step 4.

4. Step 4.3: Determine whether is_anomaly_ratio is greater than the preset threshold high_var_threshold. If not, it is normal; if so, it is an abnormal structural damage. Step 4.4: Determine whether is_anomaly_ratio is less than the preset threshold var_ratio_threshold. If it is, it is a structural damage anomaly; otherwise, it is a conservative anomaly.

6. The image data integrity detection method according to claim 1, characterized in that, It also includes the following steps: Step A5: Perform short-term fusion using a sliding window; if at least two frames show abnormalities, an alarm will be triggered.

7. The image data integrity detection method according to claim 1, characterized in that, It also includes the following steps: Step B5: If an anomaly occurs in N consecutive frames, the maintenance process is triggered.

8. The image data integrity detection method according to claim 5, characterized in that, Calculate the preset threshold using the following steps: Step C1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T ; Step C2: Calculate the eigenvalues ​​x1, x2, ..., x T mean μ T and standard deviation σ T ; Step C3: Calculate the preset threshold using the following formula: threshold=μ T +kσ T ; In the formula, k is a constant.

9. The image data integrity detection method according to claim 5, characterized in that, Calculate the preset threshold using the following steps: Step D1: Obtain the feature values ​​x1, x2, ..., x of consecutive T frames. T ; Step D2: Calculate the preset threshold using the following quantile formula: threshold=quantile(x1、x2、...、x T )。 10. An image data integrity detection device, characterized in that, The device includes a processor and a memory, wherein the memory stores an image data integrity detection program, and when the image data integrity detection program is executed by the processor, it implements the image data integrity detection method as described in any one of claims 1 to 9.

Images