Camera black spot detection method and device and electronic evaluation device

Abstract

The invention discloses a camera black spot detection method and device and an electronic evaluation device, and belongs to the technical field of industrial vision, and the method comprises the steps: obtaining a camera detection image through a white matte uniform color light source plate, carrying out the image enhancement processing, obtaining an enhanced image, carrying out the manual marking of the enhanced image, and carrying out the manual marking of the enhanced image. A target detection model fused with a wavelet pyramid structure is constructed, a pre-training model is obtained by training manual annotation data, a to-be-detected image is enhanced and input into the pre-training model to preliminarily obtain black spot detection coordinates and confidence, and then a k-neighborhood contrast analysis result is calculated. And obtaining a final black spot detection result in combination with a deep learning model detection result. According to the method, deep learning and traditional algorithm fusion are adopted, the camera black spots are detected through the device, the method can be integrated into automatic equipment, the camera is directly evaluated, errors caused by visual evaluation by people are effectively reduced, the labor cost is greatly saved, and automation of a production line is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial vision technology, and in particular relates to a method, a device and an electronic evaluation device for detecting black spots on a camera. Background Art

[0002] Camera black spot detection is one of the important evaluation indicators for measuring camera imaging quality and an important evaluation parameter in the camera quality inspection process. Its testing method is mainly to evaluate its performance by shooting a white panel with a lens. In current industrial production, many companies use manual methods to shoot white panel images to obtain images, and then use human eyes to search for black spots in the image. However, this method of judgment is not only inefficient, but also has inconsistent standards, visual fatigue and other phenomena, resulting in fluctuations in the quality of produced products.

[0003] To address the above issues, the mainstream methods at home and abroad currently use sliding window detection and contrast methods for evaluation. These methods are greatly affected by imaging quality, especially the halo produced by the image against a white panel background, which can easily cause false detection and lead to large evaluation errors. Summary of the invention

[0004] In view of the problems in the prior art, the present invention provides a method, device and electronic evaluation device for camera black spot detection, aiming to solve one of the technical problems in the related art at least to a certain extent.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for detecting black spots in a camera, comprising the following steps:

[0007] S1 obtains a camera black spot detection image through a matte white and uniform light source plate, inputs the obtained camera black spot detection image into an image preprocessing module for grayscale uniform images, obtains an enhanced image, constructs a feature image, and annotates the feature image to obtain an annotated data set;

[0008] S2 is a training set of a black spot detection model composed of annotated data sets, and trains a black spot detection model M combined with a pyramid enhancement module;

[0009] S3 inputs the image to be detected into the image preprocessing module to obtain an enhanced image, and then inputs it into the model M to output the black spot detection result;

[0010] S4 dynamically distinguishes whether there are black spots based on the model output results, combined with the camera black spot detection confidence and k-neighborhood contrast.

[0011] The image preprocessing module described in step S1 is configured with an image enhancement algorithm and a multi-scale image enhancement algorithm.

[0012] In some embodiments, the multi-scale image enhancement algorithm includes:

[0013] (1) Image segmentation: The image is segmented into blocks according to k1*k1 and k2*k2, respectively, to form image block sets A1 and A2;

[0014] (2) Counting A1, camera black spot detection method and device A2 each block of image histogram information to generate a balanced histogram H;

[0015] (3) Setting a threshold, traversing the equalized histogram, subtracting the part greater than the threshold, and counting the number of pixels subtracted;

[0016] (4) Calculate the average pixel value of the subtracted pixels, then add all the average pixel values ​​to generate the enhanced image E.

[0017] The pyramid enhancement module described in step S2 is used to perform wavelet transform on the input image to obtain decomposed images in four directions of LL, HL, LH, and HH. The decomposed images are spliced ​​in the channel direction as the input of the model M.

[0018] The black spot detection model M described in step S2 includes at least one of a traditional algorithm, a supervised algorithm and an unsupervised algorithm.

[0019] In some embodiments, the model M outputs the black spot detection coordinates and the corresponding confidence, and the model M body includes at least one of the yolo and vgg main network structures.

[0020] The method for calculating the k-neighborhood contrast in step S4 includes:

[0021] The detection model M obtains the black spot area S0 and its k-neighboring areas S1-Sk;

[0022] Calculate the mean grayscale image of the S1-Sk region and calculate the contrast with S0 respectively;

[0023] Calculate the mean. The specific process is to regularize the pixel values ​​S0-Sk and calculate the grayscale mean G0-Gk of S0-Sk;

[0024] Calculate the regional contrast; calculate the k-neighborhood mean E.

[0025]

[0026] Where E is the neighborhood mean, k is the number of neighbors, and G(i) is the grayscale mean of the neighborhood Si (i ranges from 1 to k);

[0027] In step S4, the model detection confidence c and the k-neighborhood contrast calculation method are combined to obtain the final confidence R, where the range of R is between 0 and 1. Then, whether it is a black spot is determined according to the set threshold. If R is greater than the threshold, it is a black spot, and if R is less than the threshold, it is not a black spot.

[0028] R=c*E

[0029] Among them, c is the detection confidence and E is the k-neighborhood mean.

[0030] In a second aspect, the present application also provides a camera black spot detection model device, comprising:

[0031] Data module: obtain white matte monochrome image data through the camera to be tested, enhance the data, and then annotate it to obtain annotated data;

[0032] Training module: Integrate the wavelet pyramid enhancement module to build a camera black spot detection model, input the acquired labeled data into the network for model training, and obtain a pre-trained model;

[0033] Prediction module: The camera to be tested takes a picture of a white matte light source background, and after data enhancement, it is input into the model M to obtain the black spot detection result: the black spot coordinates and the corresponding confidence level;

[0034] Calculation module: The confidence of the model detection result is integrated with the k-neighborhood contrast, and the presence or absence of black spots is confirmed based on the preset threshold calculation.

[0035] In the third aspect, the present application also provides an electronic evaluation device, which is composed of a camera to be detected, a readable and writable storage medium, a processor, a communication interface and a communication bus, wherein: the communication bus communicates between the camera and the processor through the communication interface; the readable and writable storage medium is used to store images and instructions, and the processor is used to implement the steps of the method provided in the first aspect when executing the camera black spot detection and evaluation instructions.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention adopts the fusion of deep learning and traditional algorithms and detects camera black spots through the device of the invention. The method can be integrated into automation equipment to directly evaluate the camera, effectively reducing the errors caused by human intuitive evaluation, and greatly saving labor costs and realizing production line automation.

[0038] The present invention adopts the "wavelet pyramid enhancement + deep learning + multi-parameter fusion" strategy and uses the white light source board image collected by the camera for detection and evaluation. This method can be integrated into automated equipment to directly evaluate the camera, effectively reducing the error caused by human intuitive evaluation, greatly saving labor costs and realizing production line automation. It is relatively less affected by the environment and can detect black spot areas of various resolutions and imaging qualities. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the process of the present invention.

[0040] Figure 2 This is the method for obtaining the 8-neighborhood region.

[0041] Figure 3 It is a schematic diagram of the device module of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below in conjunction with the drawings in the preferred embodiments of the present application. In the drawings, the same or similar reference numerals throughout represent the same or similar parts or parts with the same or similar functions. The described embodiments are part of the embodiments of the present application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0043] The present invention provides a method and device for detecting and evaluating black spots in a camera. In order to better understand the above technical solution, the present invention is described in detail below in conjunction with the embodiments and the accompanying drawings. Figure 1 :

[0044] like Figure 1 As shown, a method for detecting black spots in a camera can be applied to the detection of a camera device, and specifically includes the following steps:

[0045] S1. The method and device for camera black spot detection obtain the camera black spot detection image through a white matte and uniform light source plate, input the obtained image into a preprocessing module for grayscale uniform images, obtain an enhanced image, construct a feature image, and annotate the image to obtain a labeled data set.

[0046] In this embodiment, the specific implementation is: use multiple groups of different cameras to shoot the white uniform light source board, each camera obtains a single frame image to form an original data set, and then the original data is processed by a multi-scale enhancement algorithm to obtain an enhanced image, and use the open source Labelimg annotation tool to annotate the black spot area with a rectangular box, and the annotation format is xml.

[0047] The above-mentioned multi-scale image enhancement algorithm includes:

[0048] (1) Image segmentation: The image is segmented into blocks according to k1*k1 and k2*k2 to form image block sets A1 and A2;

[0049] (2) Counting A1, camera black spot detection method and device A2 each block of image histogram information to generate a balanced histogram H;

[0050] (3) Setting a threshold, traversing the equalized histogram, subtracting the part greater than the threshold, and counting the number of pixels subtracted;

[0051] (4) Calculate the average pixel value of the subtracted pixels, then add all the average pixel values ​​to generate an enhanced image E;

[0052] S2. A method and device for detecting dark spots on a camera obtains an enhanced image set and annotated data according to step S1, trains a dark spot detection model until the network model converges, and obtains a pre-trained detection model;

[0053] In this embodiment, the specific implementation is to normalize the labeled data.

[0054] In the specific implementation, the yolo series neural network is selected to construct a 4-channel input.

[0055] In the model input part, a wavelet pyramid enhancement module is constructed to perform wavelet decomposition on the preprocessed image in S1, and then combine them in the channel direction to form a 4-channel feature image, which is input into the YOLO main network for training. The pre-trained network M is obtained.

[0056] S3. In the application stage, the image to be detected is input into the image preprocessing module to obtain an enhanced image, which is then input into the model M to output the black spot detection result;

[0057] In the specific implementation, the image to be detected is input into the preprocessing module to obtain an enhanced image, and then wavelet decomposition is performed to form a four-channel feature image, which is then input into the pre-trained model to obtain the detection result, which includes the black spot coordinates and the corresponding confidence.

[0058] S4. Based on the model output results, the camera black spot detection confidence and K-neighborhood contrast are combined to dynamically distinguish whether there are black spots and determine the location of the black spots;

[0059] In the specific implementation, the k-neighborhood contrast calculation method is used, wherein the value of k is generally 8 or a multiple of 8. In this embodiment, the value of k is 8. The black spot area S0 is obtained by the detection model M, and its k-neighborhood areas S1-Sk are obtained. The acquisition method is as follows: Figure 1 , calculate the mean of the grayscale image of the S1-Sk region, calculate the contrast with S0 respectively, and then calculate the mean. The specific process is to normalize the S0-Sk pixel values ​​to the range of 0-1, calculate the S0-Sk grayscale mean G0-Gk, then calculate the regional contrast, and calculate the 8-neighborhood mean E, which ranges from 0 to 1.

[0060]

[0061] Where E is the neighborhood mean, k is the number of neighbors, and G(i) is the grayscale mean of the neighborhood Si (i ranges from 1 to 8);

[0062] Finally, step S4 combines the model detection confidence c (ranging from 0 to 1) and the 8-neighborhood contrast calculation method to obtain the final confidence R, which ranges from 0 to 1. In this embodiment, the threshold is set to 0.5. If R is greater than 0.5, black spots are detected, and if R is less than 0.5, no black spots are detected.

[0063] R=c*E

[0064] Among them, c is the detection confidence, and E is the 8-neighborhood mean.

[0065] The camera detection image is obtained through a white matte uniform color light source plate, and the image is enhanced to obtain the enhanced image. The enhanced image is manually annotated, and a target detection model integrating a wavelet pyramid structure is constructed. The pre-trained model is obtained by training the manually annotated data. The image to be tested is enhanced and input into the pre-trained model to initially obtain the black spot detection coordinates and confidence. Then, the 8-neighborhood contrast analysis results are calculated, and the final black spot detection results are obtained by combining the detection results of the deep learning model.

[0066] Embodiment 2:

[0067] This embodiment discloses a camera black spot detection device, including:

[0068] Data module: obtain white matte monochrome image data through the camera to be tested, enhance the data, and then annotate it to obtain annotated data.

[0069] In a specific implementation, multiple groups of images are acquired using multiple groups of cameras and white uniform light source panels, and then the images are enhanced, and the enhanced images are annotated using an annotation tool to obtain annotation data.

[0070] Furthermore, in the specific implementation method of multi-scale image enhancement,

[0071] (1) Image segmentation: The image is segmented into blocks according to k1*k1 and k2*k2, respectively, to form image block sets A1 and A2;

[0072] (2) Counting A1, camera black spot detection method and device A2 each block of image histogram information to generate a balanced histogram H;

[0073] (3) Setting a threshold, traversing the equalized histogram, subtracting the part greater than the threshold, and counting the number of pixels subtracted;

[0074] (4) Calculate the average pixel value of the subtracted pixels, then add all the average pixel values ​​to generate an enhanced image E;

[0075] Training module: Integrate the wavelet pyramid enhancement module to build a camera black spot detection model, input the acquired labeled data into the network for model training, and obtain a pre-trained model.

[0076] In the specific implementation, we first construct a wavelet pyramid to constitute the input module of the deep learning model, and select the YOLO series deep learning neural network to train the data module to obtain labeled data until the model converges to obtain a black spot detection model.

[0077] Prediction module: The camera to be tested takes a picture of a white matte light source background, which is then input into the network model after data enhancement to obtain the black spot detection result: the black spot coordinates and the corresponding confidence level.

[0078] In a specific implementation, the same processing operation is first performed on the data module for obtaining the original data to obtain an enhanced image, which is then input into the pre-trained model to obtain the black spot detection coordinates and the corresponding confidence.

[0079] Calculation module: The confidence of the model detection result is integrated with the k-neighborhood contrast, and the presence and coordinates of black spots are confirmed according to the preset threshold calculation.

[0080] In a specific implementation, the k-neighborhood contrast calculation method obtains the black spot area S0 from the detection model M, and obtains its k-neighborhood areas S1-Sk. The acquisition method is as follows: Figure 1, calculate the mean of the grayscale image of the S1-Sk region, calculate the contrast with S0 respectively, and then calculate the mean. The specific process is to normalize the S0-Sk pixel values ​​to the range of 0-1. The method and device for camera black spot detection calculate the S0-Sk grayscale mean G0-Gk, then calculate the regional contrast, and calculate the k-neighborhood mean E. In this embodiment, k=8.

[0081]

[0082] Where E is the neighborhood mean, k=8 is the number of neighborhoods, and G(i) is the grayscale mean of neighborhood Si (i ranges from 1 to 8);

[0083] Furthermore, step S4 combines the model detection confidence c (ranging from 0 to 1) and the 8-neighborhood contrast calculation method to obtain the final confidence R, which ranges from 0 to 1. In this embodiment, the threshold is set to 0.5. If R is greater than 0.5, black spots are detected, and if R is less than 0.5, no black spots are detected.

[0084] R=c*E

[0085] Among them, c is the detection confidence, and E is the 8-neighborhood mean.

[0086] Embodiment 3:

[0087] This embodiment provides an electronic evaluation device consisting of a white matte light source board, a camera to be detected, a readable and writable storage medium, a processor, a communication interface and a communication bus, wherein: the communication bus communicates between the camera and the processor through the communication interface; the readable and writable storage medium is used to store images and instructions, and the processor is used to execute the camera black spot detection evaluation instructions.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting black spots in a camera. It is characterized in that The following steps are involved: S1 obtains the camera black spot detection image and inputs it into the image preprocessing module for grayscale uniform images to obtain an enhanced image, form a feature image, and annotate the feature image to obtain an annotated data set; S2 is a training set of a black spot detection model composed of annotated data sets, and trains a black spot detection model M combined with a pyramid enhancement module; S3 inputs the image to be detected into the image preprocessing module to obtain an enhanced image, and then inputs it into the model M to output the black spot detection result; S4 dynamically distinguishes whether there are black spots based on the model output results, combined with the camera black spot detection confidence c and k neighborhood contrast.

2. A method for detecting dark spots on a camera according to claim 1, It is characterized in that The image preprocessing module described in step S1 is configured with an image enhancement algorithm and a multi-scale image enhancement algorithm.

3. A method for detecting dark spots on a camera according to claim 2, It is characterized in that The multi-scale image enhancement algorithm comprises: (1) Image segmentation: The image is segmented into blocks according to k1*k1 and k2*k2, respectively, to form image block sets A1 and A2; (2) Counting A1, camera black spot detection method and device A2 each block of image histogram information to generate a balanced histogram H; (3) Setting a threshold, traversing the equalized histogram, subtracting the part greater than the threshold, and counting the number of pixels subtracted; (4) Calculate the average pixel value of the subtracted pixels, then add all the average pixel values ​​to generate the enhanced image E.

4. The method for detecting dark spots on a camera according to claim 1, It is characterized in that The pyramid enhancement module described in step S2 is used to perform wavelet transform on the input image to obtain decomposed images in four directions of LL, HL, LH, and HH. The decomposed images are spliced ​​in the channel direction as the input of the model M.

5. A camera black spot detection method according to claim 1, It is characterized in that The black spot detection model M described in step S2 includes at least one of a traditional algorithm, a supervised algorithm and an unsupervised algorithm.

6. A method for detecting dark spots in a camera according to claim 5, It is characterized in that Model M outputs the black spot detection coordinates and corresponding confidences, and the main body of model M includes at least one of the main network structures of yolo and vgg.

7. A camera black spot detection method according to claim 1, It is characterized in that The method for calculating the k-neighborhood contrast in step S4 includes: The detection model M obtains the black spot area S0 and its k-neighboring areas S1-Sk; Calculate the mean grayscale image of the S1-Sk region and calculate the contrast with S0 respectively; Calculate the mean. The specific process is to regularize the pixel values ​​S0-Sk and calculate the grayscale mean G0-Gk of S0-Sk; Calculate regional contrast; Calculate k-neighborhood mean E; Among them, E is the neighborhood mean, k is the number of neighbors, and G(i) is the grayscale mean of neighborhood Si (i ranges from 1 to k).

8. A camera black spot detection method according to claim 7, It is characterized in that The method for calculating the confidence c and the k-neighborhood contrast ratio in combination with the model detection in step S4 includes obtaining the final confidence R, RC*E Where c is the detection confidence, c∈0 to 1, E is the k-neighborhood mean; R∈0 to 1; When R is greater than the preset threshold, black spots are detected, and when R is less than the threshold, black spots are not detected.

9. A camera black spot detection model device, It is characterized in that include: Data module: obtain white matte monochrome image data through the camera to be tested, enhance the data, and then annotate it to obtain annotated data; Training module: Integrate the wavelet pyramid enhancement module to build a camera black spot detection model, input the acquired labeled data into the network for model training, and obtain a pre-trained model; Prediction module: The camera to be tested takes a picture of a white matte light source background, and after data enhancement, it is input into the model M to obtain the black spot detection result; Calculation module: The confidence of the model detection result is integrated with the k-neighborhood contrast, and the presence or absence of black spots is confirmed based on the preset threshold calculation.

10. An electronic evaluation device, It is characterized in that The device is composed of a camera to be detected, a readable and writable storage medium, a processor, a communication interface and a communication bus, wherein: the communication bus communicates between the camera and the processor through the communication interface; the readable and writable storage medium is used to store images and instructions, and the processor is used to implement the steps of the method described in any one of claims 1 to 8 when executing the camera black spot detection and evaluation instructions.

Images