Natural environment test image data cleaning optimization method and system and computer readable storage medium

Abstract

The invention discloses a natural environment test image data cleaning optimization method and system and a computer readable storage medium. The method comprises the following steps: acquiring sample image data of a natural environment test; performing missing detection on the image data by using the image missing recognition model and based on a key point slope algorithm; extracting central area image data from the image data; image virtual focus detection and image overexposure detection are carried out on the central area image data; when the missing identification result is no missing, the image virtual focus detection result is no virtual focus, and the image overexposure detection result is no overexposure, determining the image data as qualified data; and otherwise, determining that the image data is unqualified data. The image data cleaning efficiency and the judgment accuracy can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural environment tests, and particularly relates to a method, a system and a computer-readable storage medium for optimizing the cleaning of natural environment test image data. Background Art

[0002] Natural environment test image data is not only an effective means for evaluating product performance and environmental adaptability, but also an important tool in the fields of environmental assessment, fault diagnosis, decision-making support, etc. At present, the sources of environmental test image data are mainly collected by artificial or unmanned devices through instruments such as cameras and video cameras. Due to the influence of instruments, operations and weather conditions, the quality of the acquired image data varies, and it is necessary for experienced test personnel to review the image data to exclude unqualified data. This traditional data cleaning method is time-consuming and laborious, and when the data volume is very large, there may be review errors due to reasons such as personnel fatigue. Implementing automated cleaning of data through image detection algorithms is an idea worth exploring. Existing image data cleaning methods can detect defocus, overexposure, etc. of ordinary images. However, existing algorithms are not applicable in the actual outdoor scenarios of natural environment tests, and the misjudgment rate is high, which limits the use of this method. For example, according to the standard, a natural environment test is carried out using a high-low bar test stand, the sample placement height is about 1 meter, and the inclined plane angle is 45°. When taking pictures, the camera is perpendicular to the upper surface of the specimen, and inevitably the ground background will be taken in. There is a large height difference between the ground and the sample surface, and there will inevitably be defocus of a large area of the background while the sample surface is clearly photographed, resulting in algorithm misjudgment. Moreover, for different samples such as steel, aluminum alloy, and protective coatings, the reflectance degrees vary greatly, and traditional overexposure detection algorithms cannot make accurate judgments. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, a system and a computer-readable storage medium for optimizing the cleaning of natural environment test image data, which automatically detect the defocus, missing, overexposure, etc. of images through an optimized algorithm, set a quality judgment threshold, and comprehensively judge whether the image quality is qualified. Through this method, the efficiency of image data cleaning and the accuracy of judgment can be effectively improved.

[0004] One aspect of the present invention provides a method for optimizing the cleaning of natural environment test image data, including:

[0005] Obtaining sample image data of a natural environment test;

[0006] Performing missing detection on the image data by using an image missing recognition model and a key point slope-based algorithm;

[0007] Extracting central region image data from the image data;

[0008] Perform image defocus detection and image overexposure detection on the image data of the central region;

[0009] When the missing recognition result is no missing, the image defocus detection result is no defocus and the image overexposure detection result is no overexposure, determine that the image data is qualified data; otherwise, determine that the image data is unqualified data.

[0010] Furthermore, the image missing recognition model is based on the framework of the YOLOv11 algorithm. The image missing recognition model includes an input module, a backbone network, a feature pyramid network, a path aggregation network, and an output module. Among them,

[0011] The input module preprocesses the image data;

[0012] The backbone network adopts the CSPDarknet structure to extract features from the preprocessed image data;

[0013] The feature pyramid network performs multi-scale feature fusion on the extracted features;

[0014] The path aggregation network connects the features of each layer using a bottom-up path to achieve further feature fusion;

[0015] The output module selects the optimal detection result through the NMS method.

[0016] Furthermore, the missing detection specifically includes:

[0017] First, perform rough segmentation on the image data through the image missing recognition model to determine the region where missing may exist, that is, the ROI region;

[0018] Subsequently, for the region where missing may exist, perform the following operations:

[0019] Through the image missing recognition model, identify the key regions in the image. Among them, the selection of the key regions is based on the characteristics of the magnetic column used to fix the sample. The image missing recognition model can accurately detect the image region where the magnetic column is located and mark the image region where the magnetic column is located as the key region;

[0020] For the identified key regions, judge whether they exist in the ROI region obtained by rough segmentation, and only retain the qualified key regions. That is, perform a logical determination on the specific positions of the key regions, regard the key regions that exceed the upper, lower, left, and right boundary ranges of the image as unreasonable regions and exclude them, and perform integrity determination on the key regions according to the area size of the key regions, and remove the incomplete key regions, so as to obtain the final number of key regions;

[0021] If the final number of key regions is less than two, directly determine that the image has a missing;

[0022] If the number of final key regions is at least two, extract the center points of each final key region as key points, and calculate the slope between any two key points. When the slope meets the preset slope requirement, it is finally determined that the image has no missing parts; otherwise, it is determined that the image has missing parts.

[0023] Further, the image defocus detection for the central region image data includes:

[0024] Convert the central region image data into a grayscale image;

[0025] Perform a convolution operation on the grayscale image using a Laplacian operator convolution kernel to obtain a Laplacian image;

[0026] Calculate the variance of the Laplacian image;

[0027] Judge whether there is image defocus by comparing the variance of the Laplacian image with a preset defocus threshold.

[0028] Further, the preset defocus threshold includes a first defocus threshold and a second defocus threshold, and the first defocus threshold is less than the second defocus threshold. When the variance of the Laplacian image is between the first threshold and the second threshold, it is determined that there is defocus in the image data; otherwise, it is determined that there is no defocus in the image data.

[0029] Further, the image overexposure detection for the central region image data includes:

[0030] Convert the central region image data into a grayscale image;

[0031] Count the number of pixels at each brightness level in the grayscale image;

[0032] Determine the number of pixels with a brightness value higher than the preset brightness threshold, that is, the number of highlight pixels;

[0033] Obtain the highlight pixel ratio from the number of highlight pixels and the total number of pixels in the grayscale image;

[0034] Judge whether there is image overexposure by comparing the highlight pixel ratio with a preset ratio threshold.

[0035] Further, the preset ratio threshold is related to the samples of the natural environment test, and samples of different materials correspond to different preset brightness thresholds.

[0036] The present invention also provides a natural environment test image data cleaning and optimization system, including:

[0037] A memory configured to store a computer program;

[0038] A processor, configured to execute the computer program to implement the natural environment test image data cleaning and optimization method as described above.

[0039] The present invention also provides a computer-readable storage medium, on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the natural environment test image data cleaning and optimization method as described above.

[0040] Advantages of the present invention: The present invention automatically detects situations such as defocus, missing, and overexposure of images through an optimization algorithm. When performing missing detection, by calculating the slope characteristics of the key point distribution, it determines whether there is an abnormal gradient change, and at the same time evaluates the integrity of the key points to further optimize the determination result; when performing defocus detection and overexposure detection, it focuses on the central area of the image and comprehensively judges whether the image quality is qualified by setting different quality evaluation thresholds. Through this method, the efficiency of image data cleaning and the accuracy of judgment can be effectively improved.

[0041] Other advantages, objectives, and features of the present invention will to some extent be described in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be learned from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the drawings, where:

[0043] Figure 1 is a schematic flowchart of the data cleaning method;

[0044] Figure 2 is the time-consuming result of the sample missing detection method based on segmentation;

[0045] Figure 3 is the time-consuming result of the detection method based on key area segmentation and key point discrimination;

[0046] Figure 4 is a schematic diagram of the slope calculation of the key area;

[0047] Figure 5 is Picture Example 1 in the dataset;

[0048] Figure 6 is Picture Example 2 in the dataset;

[0049] Figure 7 is an example diagram of an image with unqualified blurriness in the scenario of constructing a blur detection algorithm;

[0050] Figure 8 It is an example diagram of the region of interest of the fuzzy detection algorithm;

[0051] Figure 9 It is an example diagram of significantly overexposed unqualified samples;

[0052] Figure 10 It is a comparison diagram of the brightness histogram distributions of normal images and overexposed images;

[0053] Figure 11 It is a schematic diagram of the sub-image of the central region;

[0054] Figure 12 It is a comparison of data cleaning results. Specific implementation manners

[0055] Hereinafter, with reference to the accompanying drawings, the preferred embodiments of the present invention will be described in detail. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than limiting the protection scope of the present invention.

[0056] Image data cleaning is to automatically discriminate natural environment test images with abnormalities such as defocus, overexposure, and incompleteness. The present invention provides an optimization method for natural environment test image data cleaning, and the method includes:

[0057] Obtain the image data of the samples of the natural environment test;

[0058] Use the image missing recognition model and the algorithm based on the key point slope to perform missing detection on the image data;

[0059] Extract the central region image data from the image data;

[0060] Perform image defocus detection and image overexposure detection on the central region image data;

[0061] When the missing recognition result is no missing, the image defocus detection result is no defocus and the image overexposure detection result is no overexposure, determine that the image data is qualified data; otherwise, determine that the image data is unqualified data.

[0062] It should be noted that when performing data cleaning, for missing detection, defocus detection, and overexposure detection, these three detections can be executed sequentially or simultaneously. For the case of sequential execution, the order can also be fixed or arbitrary.

[0063] In some embodiments, missing detection can be performed first, then defocus detection, and finally overexposure detection. For example Figure 1As shown, first perform missing detection (i.e., image missing discrimination). If the determination result is "missing", directly determine the image data as unqualified data; if the determination result is "not missing", then extract the central region of the image (or "image data") and perform defocus detection on the central region. If the defocus detection result is "defocus", then determine the image data as unqualified data; if the detection result is "no defocus", then proceed with overexposure detection. If the overexposure detection result is "overexposed", then determine the image data as unqualified data; if the overexposure detection result is "no overexposure", then determine the image data as qualified data.

[0064] Among them, during image missing detection, model discrimination can be performed first, that is, use the image missing recognition model to determine whether there is a missing. After the model discrimination is completed, the "determination method of the slope and integrity of the key-point image" is carried out. Only when both algorithms return the result of not missing will it be confirmed that the sample is not missing.

[0065] It should be noted that Figure 1 the execution order in

[0066] is only illustrative and does not limit the present invention. In some embodiments of the present invention, defocus detection can also be performed first, then overexposure detection, and finally missing detection; or overexposure detection can be performed first, then defocus detection, and finally missing detection. Those skilled in the art can understand that data cleaning can also be performed in other orders, which will not be elaborated here.

[0067] First, the input module preprocesses the image data. The preprocessing formula of the input image can be expressed as:

[0068]

[0069] where I represents the original input image, μ and σ respectively represent the mean and standard deviation of the image pixels, and I′ represents the normalized image (i.e., the result of preprocessing).

[0070] The backbone network extracts features from the preprocessed image data. The CSPDarknet structure can be used. The CSPDarknet network contains multiple convolutional layers and residual modules for extracting features at different levels.

[0071] The output of the residual block can be expressed as:

[0072] Y = X + F(X, W 0),

[0073] Among them, X represents the feature input to CSPDarknet; F(X, W 0 ) represents the convolution operation; W 0 represents the convolution kernel weight; Y represents the output of the residual block.

[0074] Through the CSPDarknet structure, the backbone network can effectively extract high-dimensional features and is suitable for object detection of different sizes.

[0075] The purpose of the Feature Pyramid Network (i.e., Feature Pyramid Network, FPN) is to achieve multi-scale feature fusion to adapt to objects of different sizes. There are defect regions of different sizes in image integrity detection, so multi-scale feature fusion is essential. FPN performs upsampling and weighted fusion on features of different scales, enabling the network to focus on large and small objects simultaneously.

[0076] The output of FPN can be expressed as:

[0077] P L = Conv(P L+1 ) + F L ,

[0078] where P L represents the L-th layer feature map generated by FPN, F L represents the L-th layer feature extracted by the backbone network, and Conv represents convolution. Through the step-by-step upsampling and fusion of multi-layer features, FPN can effectively enhance the attention to small objects and detailed features.

[0079] The Path Aggregation Network (i.e., Path Aggregation Network, PANet) further strengthens the aggregation of YOLOv11 for features of different scales. It uses a bottom-up path to connect the features of each layer and transfers the low-level features to the high-level features to achieve stronger feature fusion. The design of PANet enables the network to capture richer detailed features and better identify tiny defects or abnormal regions in image integrity detection.

[0080] The feature fusion process of PANet can be expressed as:

[0081] P L ' = Concat(P L , Conv(P L-1 ))

[0082] where P L and P L-1They respectively represent the feature maps of different scales generated by FPN; Concat represents the operation of feature concatenation. Through PANet, YOLOv11 can effectively utilize context information to make the object detection results more accurate.

[0083] The output module of YOLOv11 contains a prediction head, which is responsible for object detection on each feature map. The output module will generate multiple bounding boxes, and each bounding box contains position parameters, confidence, and class probabilities.

[0084] YOLOv11 uses the NMS (Non-Maximum Suppression) method to select the optimal detection results and suppress redundant bounding boxes. The main logic of NMS is as follows:

[0085] 1. Assume that the model outputs a set of candidate box collections:

[0086]

[0087] Each candidate box B i includes:

[0088] The coordinates of the box: b i =(x 1i , y 1i , x 2i , y 2i ), representing the upper left and lower right coordinates of the box;

[0089] The confidence score: s i .

[0090] 2. Sort in descending order according to the confidence score to obtain a new sorted set:

[0091]

[0092] 3. Select the box with the highest score:

[0093] Initialize the reserved set as:

[0094]

[0095] In each round, select the current candidate box B with the highest score (i) , and add it to the reserved set:

[0096]

[0097] 4. Calculate IoU:

[0098] For each box B in the remaining candidate box set (j) (j > i), calculate its IoU with the current box B with the highest score (i)Intersection over Union (IoU):

[0099]

[0100] 5. Remove high IoU boxes:

[0101] Remove the boxes with IoU exceeding the set threshold τ with respect to B (i) from the remaining candidate boxes:

[0102]

[0103] 6. Repeat steps:

[0104] Repeat steps 3 to 5 until no two boxes in the candidate box set have an IoU threshold greater than τ.

[0105] In image integrity detection, YOLOv11 evaluates the integrity of an image by detecting whether there are defective or abnormal regions in the image. The key to the integrity detection task lies in identifying all factors that affect the image quality, which usually appear as irregular shapes or boundaries. The multi-scale feature fusion mechanism of YOLOv11 enables it to detect large-scale damaged areas and subtle defects simultaneously, thus comprehensively evaluating the image integrity.

[0106] The loss function of YOLOv11 consists of three parts: classification loss, localization loss, and confidence loss.

[0107] The classification loss is used to measure the prediction accuracy of the model for target classes. The classification loss Loss cls is usually defined using cross-entropy loss as:

[0108]

[0109] where p i represents the probability of the actual class; C represents the total number of classes; represents the predicted class probability.

[0110] The localization loss is used to measure the deviation between the predicted bounding box and the ground truth bounding box, and usually uses CIoU (i.e., Complete IoU) loss. The localization loss Loss loc is defined as:

[0111]

[0112] where IoU represents the intersection over union of the ground truth bounding box and the predicted bounding box; b and b g represent the centers of the predicted bounding box and the ground truth bounding box respectively; ρ(b, b g) represents the distance between the center of the predicted bounding box and the center of the ground truth bounding box; d represents the diagonal distance of the smallest closed region that can simultaneously contain the predicted bounding box and the ground truth bounding box; α represents the balance coefficient; v is used to measure the consistency of the aspect ratio.

[0113] The confidence loss is used to measure the accuracy of the model's confidence prediction for the detection results. The confidence loss Loss conf is defined as:

[0114]

[0115] where y 0 represents the true confidence label; represents the predicted confidence score.

[0116] The total loss function of YOLOv11 is the weighted sum of the above three losses, which is expressed by the formula:

[0117] Loss = λ cls Loss cls + λ loc Loss loc + λ conf Loss conf ,

[0118] where λ cls , λ loc and λ conf are the weight coefficients of each loss.

[0119] YOLOv11 performs excellently in image integrity detection with its multi-scale feature extraction and efficient prediction module. The FPN and PANet modules in its architecture ensure the fusion of multi-scale features, adapt to defect regions of different sizes, and guarantee high precision and high efficiency in integrity detection. The loss function of YOLOv11 is reasonably designed, making the model more accurate in localization, classification, and confidence prediction when detecting integrity defects.

[0120] At the beginning of the experiment, the collected image dataset was used and labeled. To comprehensively understand the characteristics of the data, resolution statistics, sample distribution analysis, and refined classification of missing types were carried out on the images. It was found that about 90% of the missing samples in the collected image dataset belong to local problems, while the remaining 10% have no missing problems.

[0121] In an embodiment of the present invention, for missing detection, a determination method combining key point slope and integrity is proposed, which may include:

[0122] First, the image data is roughly segmented by an image missing recognition model to determine the regions where missing may exist (usually around the image), denoted as the ROI region;

[0123] Subsequently, for the regions where missing may exist, the following operations are performed:

[0124] Through the image missing recognition model, the key regions in the image are recognized. Among them, the selection of the key regions is based on the characteristics of the magnetic column used to fix the sample. The image missing recognition model can accurately detect the image region where the magnetic column is located and mark the image region where the magnetic column is located as the key region;

[0125] For the recognized key regions, it is judged whether they exist in the ROI region obtained by rough segmentation, and only the qualified key regions are retained. That is, a logical determination is made on the specific positions of the key regions, and the key regions that exceed the upper, lower, left, and right boundary ranges of the image are regarded as unreasonable regions and excluded. And according to the area size of the key regions, the integrity of the key regions is determined, and the incomplete key regions are removed, so as to obtain the final number of key regions;

[0126] If the final number of key regions is less than two, it is directly determined that the image has a missing;

[0127] If the final number of key regions is at least two, the center points of each final key region are extracted as key points, and the slope between any two key points is calculated. When the slope meets the preset slope requirement, it is finally determined that the image has no missing, otherwise, it is determined that the image has a missing.

[0128] Therefore, based on the slope and integrity of the key points, it can be comprehensively judged whether the sample has a missing.

[0129] For the missing detection method in the present invention, the correlation between the distribution of key points in the trend regions is fully considered. Especially in images with subtle missing or complex structures, the detection accuracy is significantly improved. Specifically, this method calculates the local slope characteristics of the key point distribution to judge whether there is an abnormal gradient change, and at the same time evaluates the integrity of the key points to further optimize the judgment result. When specifically implemented, the algorithm calculates the angle change between each key point and its neighborhood points, and if the abnormal change exceeds the threshold, it is marked as a missing region. Figure 4 It is the slope calculation of the key region, where, Figure 4The red dots in it represent two key points (i.e., the center points of the magnetic column areas for fixing the samples), and the line connecting these two red dots represents the slope. Here, according to the experiment, the slope threshold can be set to 8 degrees (or other appropriate angles). If it exceeds 8 degrees, it indicates a missing part. In addition, a global key point integrity score is introduced to quantify the overall consistency of the key point distribution, that is, when the key point integrity is less than 30% (or other appropriate values), it is considered that the key points are incomplete, and the sample area is also incomplete.

[0130] The missing detection method of the present invention has shown a significant improvement in comprehensive tests. The detection accuracy of local missing parts has been increased to 93%, and at the same time, the detection speed of the algorithm has been further optimized, and it only takes 1.1 seconds to process a single image on average. Especially in the scenarios of subtle missing parts and complex backgrounds, this method shows stronger robustness.

[0131] In order to demonstrate the superiority of the missing detection method proposed by the present invention, it will be compared with other methods below.

[0132] First, for the method based on sample segmentation, this method takes image segmentation as the core, extracts the possible missing areas and calculates their area ratios (i.e., the area ratio of the possible missing areas to the entire sample area) to determine whether there is an image missing phenomenon. However, the experimental results show that this method has great limitations in both detection efficiency and accuracy. Verified on the above - collected data set, the specific test data shows that the determination accuracy of the segmentation method for local missing parts is only 50%, and the detection time is relatively long, with an average of 5 seconds required to process a single image. The time - using results of the specific sample missing detection method based on segmentation are as Figure 2 shown. This result shows that simply relying on the segmentation method not only has a large amount of calculation, but also has limited ability to capture detailed missing parts, and it is difficult to meet the actual needs.

[0133] Next, compared with the composite determination method that combines sample segmentation and key point detection. The composite determination method that combines sample segmentation and key point detection also considers the key structures in the image. Therefore, the detection of key points is introduced. This method first roughly segments the image to locate the areas where missing parts may exist; then further detects the integrity of key points within these areas, and uses the distribution characteristics of key points to determine whether there are missing parts. This method has shown good results in preliminary tests. The results show that the detection accuracy of local missing parts has been increased to 70%, and the detection speed has also been improved, with the average time to process a single image reduced to 2 seconds. Figure 3 It is the time - using result of the detection method based on key area segmentation and key point discrimination.

[0134] By comparing with the above two methods, the final results prove that the missing detection method in the present invention can effectively solve the efficiency and accuracy problems of traditional segmentation methods.

[0135] Next, the defocus detection will be described.

[0136] The present invention adopts the Laplacian operator method with a region of interest, focusing on the Laplacian response value in the central region of the image. This region usually contains the main image information and details, while the edge region is more susceptible to noise interference. The advantage of the Laplacian operator defocus detection is that the calculation is simple and no complex parameter adjustment is required. Moreover, the Laplacian operator is sensitive to edges and details.

[0137] Specifically, the image defocus detection includes:

[0138] First, obtain the image data of the central region;

[0139] The image data of the central region can be a part of the center of the original image (for example, Figure 8 the image data within the red frame area shown), which usually contains the main image information and details, while the edge region is more susceptible to noise interference.

[0140] Figure 11 is a schematic diagram of the sub-image of the central region. Figure 11 In, the central region is restricted to the left and right 25% and the upper and lower 6% of the center of the image. The sub-image of the central region is the region of interest (ROI, Region of Interest).

[0141] In some embodiments, the central region of the image can also be defined as the 25% length and width range of the original image, and its specific extraction formula is:

[0142]

[0143] where W and H respectively represent the width and height of the original image.

[0144] Then, the sub-image of the central region is extracted as:

[0145] ROI = Image[y min :y max ,x min :x max .

[0147] This operation divides the original image Image into a sub-image with a size of sub-image.

[0148] Next, convert the image data of the central region into a grayscale image to reduce the computational complexity and remove the interference of color information. The grayscale image can better highlight the edges and brightness differences in the image, enabling the Laplacian operator to more effectively detect the edge information in the image.

[0149] Then, perform a convolution operation on the grayscale image using the Laplacian operator kernel to obtain the Laplacian image L(x, y). In this image, edges and high-frequency information are enhanced, while smooth regions are represented by smaller or near-zero pixel values. Thus, by analyzing the Laplacian image, the sharpness of the image can be quantitatively measured.

[0150] The Laplacian operator is a second-order differential operator, and its core idea is to detect the edges of an image through second derivatives. Compared with first-order derivative operators (such as the Sobel operator), the Laplacian operator can more sensitively detect fast-changing regions in the image (such as edges and details). Since the edge information in a defocused image is smoothed and the response value of the Laplacian operator is small, the response intensity of the Laplacian operator can be used to measure the sharpness of the image.

[0151] In a two-dimensional image, the definition of the Laplacian operator is as follows:

[0152]

[0153] where f(x, y) represents the pixel value of the image at point (x, y), represents the Laplacian value of the image at point (x, y). Using a discrete convolution kernel to approximate the Laplacian operator, by way of example only, the 3×3 Laplacian convolution kernel can be:

[0154] Or

[0155] Perform a convolution operation on the image using the Laplacian operator to obtain the Laplacian image, which is the image of edge and detail information. The larger the pixel value in the Laplacian image, the richer the edge and detail information; smaller or near-zero pixel values in the Laplacian image indicate smooth regions.

[0156] Next, calculate the variance of the Laplacian image to quantify the sharpness of the image. The larger the variance, the richer the details and edges of the image, that is, the clearer the image. The variance method can quickly and accurately detect the degree of defocus of the image and is suitable for application scenarios with high real-time requirements.

[0157] Among them, the variance of the Laplacian image can be expressed as:

[0158]

[0159] where M and N represent the width and height of the image in the central region respectively; represents the average pixel value of the Laplacian image. The variance σ 0 2 can reflect the distribution of pixel values in the image. The larger the value, the more details and clearer the edges of the image; the smaller the variance, the more defocused the image tends to be.

[0160] By comparing the variance of the Laplacian image with a preset defocus threshold, it is determined whether there is image defocus.

[0161] Furthermore, the preset defocus threshold includes a first defocus threshold and a second defocus threshold, and the first defocus threshold is less than the second defocus threshold. When the variance of the Laplacian image is between the first threshold and the second threshold, it is determined that there is defocus in the image data; otherwise, it is determined that there is no defocus in the image data.

[0162] The superiority of the defocus detection method in the present invention is illustrated below through specific experiments.

[0163] 80 representative image samples were collected from a variety of scenarios. These images cover different resolutions, texture complexities, and scene details, including aluminum alloy, coating surfaces, and steel sample images. Through statistical analysis, the resolutions of these samples vary from 4000*3000 to 3700×5200, and all of the image resolutions are above 4000×3000, as Figure 5 and Figure 6 shown. To clarify the experimental objectives, these images were manually divided into clear images and blurred images, with proportions of 70% and 30% respectively. Figure 7 is an example diagram of a blurred and unqualified image (i.e., a defocused image).

[0164] Experiments were carried out on the images obtained above and compared with other defocus detection methods.

[0165] Comparison method 1: Instead of using the central region to determine, global blur determination is performed, that is, determination is carried out on the entire image data. In this method, first, the entire image data is converted into a grayscale image, and then the Laplacian operator convolution kernel is convolved with the grayscale image to obtain a Laplacian image, and the variance of the Laplacian image is compared with a preset threshold. If it is lower than the threshold, it is determined as a blurred image. The correct determination rate of this method is only 30%. Through the analysis of the detection results, it is found that some clear images with large noise in the edge region are misjudged as blurred; moreover, some clear images with relatively simple textures (such as large-area single-color regions) are misclassified as blurred due to low Laplacian response values. This result shows that although the Laplacian operator can capture the blurred features of the image to a certain extent, its simple global threshold strategy cannot adapt to the diversity of the samples, and the global judgment method ignores the differences in local features of the image, resulting in insufficient capture of detailed information in the key regions.

[0166] Comparison method 2: Calculate the Laplacian response value of the central region, but still use a single fuzzy determination threshold. The accuracy of defocus determination by this method has been improved to 70%. Especially in the scenario of a large-area single-color background, this method shows stronger robustness. However, further tests show that there are still about 15% of the images with misjudgment problems. These images are mainly concentrated on the coating and aluminum alloy samples, because the surface gloss or reflection characteristics of such samples will cause the Laplacian operator response value to be abnormally high or low, and it is impossible to accurately determine through a single threshold.

[0167] Therefore, the present invention adopts a dual-threshold determination strategy. By setting the upper and lower limit ranges, when the Laplacian response value (i.e., variance) falls within this range, the determination accuracy can be improved. For example, it is determined that the samples with the determination threshold between 30 and 105 are fuzzy samples (i.e., defocus samples). When the method of the present invention is adopted, the determination rate of image blur determination is increased to 95%. More importantly, the misjudgment rate for coating and aluminum alloy samples has decreased by more than 30%. This method effectively reduces the misjudgment caused by the gloss and reflection characteristics of special materials.

[0168] The following describes the image overexposure detection.

[0169] Image overexposure detection is one of the important tasks in image quality detection. Its purpose is to determine whether the image loses detail information in some areas due to excessive light. Overexposure will saturate the highlight areas in the image, making it difficult to distinguish the details of these areas, thus affecting the subsequent image processing and application effects. Common overexposure problems occur in outdoor photography, face recognition, security monitoring and other scenarios, especially when the lighting environment changes greatly. Therefore, automating the image overexposure detection is of great significance for improving the image quality.

[0170] A simple and effective method for image overexposure detection is to analyze the brightness distribution in the image through the brightness histogram. The brightness histogram describes the distribution of the number of pixels at different brightness levels in the image. Generally speaking, if the highlight area of the image brightness histogram is concentrated and dense, it means that the image may be overexposed.

[0171] The following details the image overexposure detection method based on the brightness histogram.

[0172] The brightness histogram is a statistical representation of the brightness distribution of an image. The horizontal axis represents the brightness value (ranging from 0 to 255, representing black to white respectively), and the vertical axis represents the number of pixels with that brightness value. The brightness histogram can visually reflect the brightness distribution of the image, enabling us to determine whether there are obvious high-brightness or low-brightness regions in the image. Usually, the brightness histogram of an overexposed image will have a large number of pixels concentrated in the high-brightness part, that is, a relatively high peak is formed at a brightness value close to 255. Therefore, based on this feature, it can be determined whether the image is overexposed. The overexposure detection method based on the brightness histogram first grayscales the image to calculate the brightness histogram. Subsequently, it determines whether the image is overexposed by statistically calculating the proportion of the number of pixels in the high-brightness region (brightness value exceeding 240) to the total number of pixels. By setting a preliminary overexposure determination threshold, if the proportion of high-brightness pixels exceeds 10%, it is considered that the image has an overexposure phenomenon. Figure 10 shows the histogram distributions of two different images, a normal image and an overexposed image. Among them, Figure 10 the left image in [Figure reference] is the brightness histogram distribution result of the normal image; Figure 10 the right image in [Figure reference] shows the brightness histogram distribution of the overexposed image.

[0173] In the present invention, the image overexposure detection includes:

[0174] First, obtain the central region of the image data;

[0175] Next, convert the image data of the central region into a grayscale image and extract its brightness information, thereby simplifying the processing and focusing on the statistics of the brightness distribution. The grayscale image removes the color information, enabling the detection process to more accurately reflect the brightness characteristics of the image.

[0176] Count the number of pixels at each brightness level in the grayscale image to obtain the brightness histogram H(i), where i represents the brightness value, taking values from 0 to 255, and H(i) represents the number of pixel points with a brightness value of i. The brightness histogram can provide the distribution of different brightness regions in the image, thereby enabling the determination of whether there are obvious high-brightness or low-brightness regions in the image.

[0177] Then, determine the number of pixels with a brightness value higher than the preset brightness threshold T, that is, the number of high-brightness pixels N bright . For example, T can be any appropriate value. For example, it can be any value between 170 and 240, such as 170, 180, 190, 200, 210, 220, 230, etc. Then

[0178] Next, obtain the proportion of high-brightness pixels by dividing the number of high-brightness pixels by the total number of pixels in the grayscale image

[0179]

[0180] Compare the highlighted pixel ratio with a preset ratio threshold to determine whether there is image overexposure. For example, when P bright > 0.2, that is, the number of pixels with brightness exceeding the brightness threshold T (such as 220) accounts for more than 20% of the total number of pixels, it can be considered that the image is overexposed. By adjusting the ratio threshold, the sensitivity of overexposure detection can be flexibly controlled. If P bright has a high value, it indicates that a large area of the image is highlighted and overexposure may exist. Through this method, it is possible to quickly determine whether the image is overexposed and the severity of overexposure.

[0181] Furthermore, the preset ratio threshold is related to the samples of the natural environment test, and different samples of different materials correspond to different preset brightness thresholds. For example, for steel samples, the set brightness threshold is 180, and for aluminum alloy samples, the selected brightness threshold is 195. Such a selection enables the determination of the highlighted ratio to adapt to different samples.

[0182] The superiority of overexposure detection in the present invention is illustrated below through specific experiments.

[0183] First, 80 image samples collected in the same batch are also used, covering scenarios such as steel, coatings, and aluminum alloys. To ensure the diversity of the samples, these images include different resolutions, lighting conditions, and scene details, including both normal images with uniform light distribution and abnormal images with obvious overexposure. After manual annotation, the sample data is divided into normal images and overexposed images, where normal images account for 65% of the total samples and overexposed images account for 35%. Figure 9 It is an example diagram of unqualified samples with significant overexposure (i.e., obvious overexposure).

[0184] It can be known from the experiment that although the overexposure detection method directly based on the brightness histogram is simple to calculate and intuitive, can quickly analyze the image brightness distribution, and is suitable for scenarios with high real-time requirements; moreover, since this method does not rely on complex models or a large amount of computing resources, it can be widely applied to various image quality detection tasks. However, this method may produce misjudgments when processing images in local overexposure or high-light environments. Also, there are certain misjudgments for some coating materials or rate materials (such as outdoor images with uneven light spot distribution).

[0185] When using the overexposure detection method in the present invention (i.e., selecting the central area and setting different brightness thresholds according to different materials), the detection accuracy is significantly improved.

[0186] The present invention provides an optimized method for cleaning image data in a natural environment test. Through the improved image detection algorithm, automatic cleaning of image data is realized, effectively reducing the labor cost and improving the data cleaning efficiency.

[0187] Cleaning accuracy test: Select a total of 100 abnormal images and normal-quality images, including defocused, overexposed, incomplete images, etc., and perform real image data cleaning through the system program. Figure 12 This is the cleaning result of the above 100 images. Compare the cleaning result with the annotation result. Among them, the cleaning and annotation results of 90 images are consistent, and the accuracy of the image cleaning model is 90.00%.

[0188] In the image quality detection task, defocus and overexposure are two key factors affecting image visual quality and information expression. Traditional detection methods usually perform global analysis on the entire image. However, due to the edge area of the image being easily interfered by factors such as environmental light and motion blur, this method is prone to misjudgment in complex scenarios. In natural images, the main information is usually concentrated in the central area of the image, while the edge area is mostly background or secondary information. Therefore, for image quality detection, especially in defocused and overexposed scenarios, the clarity and brightness distribution in the central area have higher detection value. The changes in the edge area may be more affected by external factors such as environmental light and background complexity, resulting in an increased risk of misjudgment. Therefore, the present invention proposes an improved algorithm based on the central area attention mechanism, focusing the detection on the central area of the image to improve the robustness and accuracy of detection.

[0189] The present invention also provides a natural environment test image data cleaning optimization system, including:

[0190] A memory configured to store a computer program;

[0191] A processor configured to execute the computer program to implement the natural environment test image data cleaning optimization method as described above.

[0192] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the natural environment test image data cleaning optimization method as described above.

Claims

1. A natural environment test image data cleaning optimization method, characterized in that: include: Acquire image data of natural environment test samples; Use image missing recognition model and key point slope algorithm to detect missing images. Extracting central area image data from the image data; Perform image defocus detection and image overexposure detection on the image data in the center area; When the missing identification result is no missing, the image defocus detection result is no defocus, and the image overexposure detection result is no overexposure, the image data is determined to be qualified data; otherwise, the image data is determined to be unqualified data.

2. The natural environment test image data cleaning and optimization method according to claim 1 is characterized in that: The image missing recognition model is based on the framework of the YOLOv11 algorithm. The image missing recognition model includes an input module, a backbone network, a feature pyramid network, a path aggregation network, and an output module. The input module preprocesses the image data; The backbone network adopts the CSPDarknet structure to extract features from the preprocessed image data; The feature pyramid network performs multi-scale feature fusion on the extracted features; The path aggregation network uses a bottom-up path to connect the features of each layer to achieve further feature fusion; The output module selects the best detection results through the NMS method.

3. The natural environment test image data cleaning and optimization method according to claim 1 is characterized in that: Deletion detection specifically includes: First, the image data is roughly segmented through the image missing recognition model to determine the area where missing information may exist, which is recorded as the ROI area. Then, for the areas where there may be missing data, perform the following operations: The key areas in the image are identified through the image missing recognition model, wherein the selection of the key areas is based on the characteristics of the magnetic columns used to fix the sample. The image missing recognition model can accurately detect the image area where the magnetic columns are located, and mark the image area where the magnetic columns are located as the key area; For the identified key areas, determine whether they exist in the ROI area obtained by rough segmentation, and only retain the key areas that meet the conditions, that is, make a logical judgment on the specific location of the key areas, and regard the key areas beyond the upper, lower, left and right boundaries of the image as unreasonable areas and exclude them. In addition, according to the size of the key areas, make a judgment on the integrity of the key areas, remove the incomplete key areas, and thus obtain the final number of key areas; If the number of final key regions is less than two, the image is directly judged to be missing; If the final number of key areas is at least two, the center point of each key area is extracted as the key point, and the slope between any two key points is calculated. When the slope meets the preset slope requirement, it is finally determined that the image has no missing parts, otherwise, it is determined that the image has missing parts.

4. The natural environment test image data cleaning and optimization method according to claim 1 is characterized in that: The image out-of-focus detection of the central area image data includes: Convert the center area image data into a grayscale image; Use the Laplacian operator convolution kernel to perform convolution operation with the grayscale image to obtain the Laplacian image; Calculate the variance of the Laplacian image; By comparing the variance of the Laplace image with a preset defocus threshold, it is determined whether the image is defocused.

5. The natural environment test image data cleaning and optimization method according to claim 4 is characterized in that: The preset defocus threshold includes a first defocus threshold and a second defocus threshold, the first defocus threshold is less than the second defocus threshold, and when the variance of the Laplace image is between the first threshold and the second threshold, it is determined that defocus exists in the image data, otherwise, it is determined that defocus does not exist in the image data.

6. The natural environment test image data cleaning and optimization method according to claim 1 is characterized in that: Image overexposure detection of the center area image data includes: Convert the center area image data into a grayscale image; Count the number of pixels at each brightness level in a grayscale image; Determine the number of pixels whose brightness values ​​are higher than a preset brightness threshold, i.e., the number of highlighted pixels; The ratio of highlighted pixels is obtained by dividing the number of highlighted pixels by the total number of pixels in the grayscale image; The ratio of highlight pixels is compared with a preset ratio threshold to determine whether the image is overexposed.

7. The natural environment test image data cleaning and optimization method according to claim 6 is characterized in that: The preset ratio threshold is related to the samples of the natural environment test, and samples of different materials correspond to different preset brightness thresholds.

8. A natural environment test image data cleaning and optimization system, characterized in that: include: a memory configured to store a computer program; A processor is configured to execute the computer program to implement the natural environment test image data cleaning optimization method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the natural environment test image data cleaning optimization method according to any one of claims 1 to 7 is implemented.

Images