Image-based defect index determination method and apparatus

Abstract

The application discloses an image-based defect index determination method and device, in which a first to-be-recognized image is subjected to image reconstruction to obtain a first reconstructed image; difference calculation is performed on the first to-be-recognized image and the first reconstructed image to determine a plurality of defect pixel points and a pixel difference value corresponding to each defect pixel point; based on the pixel difference value of each defect pixel point, a virtual pixel value of each defect pixel point is determined; and based on the total number of the plurality of defect pixel points and the virtual pixel value of each defect pixel point, a defect index of the first to-be-recognized image is determined. The method first obtains the defect pixel points through the difference between the to-be-recognized image and the reconstructed image, determines that the to-be-recognized image is an abnormal image, and determines the position of the defect pixel points; in addition, the defect index is determined through the total number of the defect pixel points and the virtual pixel value to represent the defect degree of the to-be-recognized image, which not only meets the real scene requirement but also is convenient for landing deployment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for determining an image-based defect index. Background Technology

[0002] With the rapid development of information and technology, image defect recognition technology is widely used in the field of industrial appearance defect recognition to identify potential defects in the texture, structure, brightness, and darkness of various products.

[0003] In existing technologies, image defect recognition methods are as follows: First, with reconstruction as the goal, a deep learning model is trained based on a normal image; then, the image to be identified is reconstructed using the trained deep learning model; finally, the reconstruction result is compared with a normal image to be identified to determine whether they are consistent, and a first anomaly score is obtained; the reconstruction result is compared with an abnormal image to be identified to determine whether they are consistent, and a second anomaly score is obtained; the image to be identified is determined to be normal or abnormal based on the first or second anomaly score.

[0004] It should be noted that existing solutions can only determine whether the image to be detected is abnormal, but cannot determine the degree of abnormality. Therefore, there is an urgent need for a method that can not only identify whether there are defects in the image to be identified (i.e., whether the image to be identified is an abnormal image), but also determine the degree of abnormality (i.e., the defect index), so that users can take the next step according to the actual situation. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and apparatus for determining an image-based defect index, which aims to identify the degree of abnormality in an image.

[0006] In a first aspect, embodiments of this application provide an image-based method for determining a defect index, the method comprising:

[0007] The first image to be identified is reconstructed to obtain the first reconstructed image;

[0008] Difference calculation is performed on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel;

[0009] Based on the pixel difference of each defective pixel, determine the virtual pixel value of each defective pixel;

[0010] The defect index of the first image to be identified is determined based on the total number of defective pixels and the virtual pixel value of each defective pixel.

[0011] Optionally, determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel specifically involves:

[0012] The absolute value of the pixel difference of each defective pixel is determined as the virtual pixel value of each defective pixel.

[0013] Optionally, determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel includes:

[0014] Filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel;

[0015] The absolute value of the filtered value of each defective pixel is determined as the virtual pixel value of each defective pixel.

[0016] Optionally, determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel includes:

[0017] Filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel;

[0018] The normalized value of each defective pixel is obtained by normalizing the absolute value of the filtered value of each defective pixel.

[0019] The normalized value of each defective pixel is determined as the virtual pixel value of each defective pixel.

[0020] Optionally, the step of performing filtering based on the pixel difference or the absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel specifically involves:

[0021] Gaussian filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel.

[0022] Optionally, determining the defect index of the first image to be identified based on the total number of defective pixels and the virtual pixel value of each defective pixel includes:

[0023] Defective pixels with virtual pixel values ​​less than a preset threshold are excluded to obtain the remaining defective pixels.

[0024] The defect index is determined based on the total number of remaining defective pixels and the virtual pixel value of each remaining defective pixel.

[0025] Optionally, determining the defect index of the first image to be identified based on the total number of defective pixels and the virtual pixel value of each defective pixel includes:

[0026] The defect index of the first image to be identified is determined based on the total number of pixels of the plurality of defective pixels, the virtual pixel value of each defective pixel, a first coefficient, a second coefficient, and a preset defect index formula, wherein the first coefficient is an adjustable coefficient based on the total number of pixels of the plurality of defective pixels, and the second coefficient is an adjustable coefficient based on the virtual pixel value of each defective pixel.

[0027] Optionally, the formula for the preset defect index is as follows:

[0028] Y = A × X area +B×∑X 2

[0029] Where Y is the defect index, A is the first coefficient, and X... area B is the total number of defective pixels, B is the second coefficient, and X is the virtual pixel value of each defective pixel.

[0030] Optionally, the step of reconstructing the first image to be identified to obtain a first reconstructed image includes:

[0031] If the size of the first image to be identified is larger than the preset size, the first image to be identified is cropped according to the preset strategy to obtain multiple second images to be identified;

[0032] Image reconstruction is performed on the plurality of second images to be identified to obtain a plurality of second reconstructed images;

[0033] The plurality of second reconstructed images are stitched together according to the preset strategy to obtain the first reconstructed image.

[0034] Optionally, the method further includes: training a preset encoding and decoding model based on a normal image and a random mask image of the normal image, and obtaining the trained encoding and decoding model as a reconstructed image model, wherein the pixel values ​​of the random mask image are random values ​​within a preset pixel value range;

[0035] The step of reconstructing the first image to be identified to obtain the first reconstructed image includes:

[0036] The first image to be identified is reconstructed using the reconstructed image model to obtain the first reconstructed image.

[0037] Optionally, the step of training a preset encoding / decoding model based on a normal image and a random mask image of the normal image to obtain the trained encoding / decoding model as a reconstructed image model includes:

[0038] The preset encoding / decoding model is trained based on the normal image, a random mask image of the normal image, and a preset loss function to obtain the trained encoding / decoding model as the reconstructed image model. Specifically, the preset loss function is:

[0039] Loss = C × PSNR + D × SSIM + E × JND

[0040] Wherein, Loss is the preset loss function, C is the third coefficient, PSNR is the peak signal-to-noise ratio of the normal image and the reconstructed image corresponding to the normal image during training, D is the fourth coefficient, SSIM is the structural similarity of the normal image and the reconstructed image corresponding to the normal image during training, E is the fifth coefficient, and JND is the minimum perceptible difference between the normal image and the reconstructed image corresponding to the normal image during training. The third, fourth, and fifth coefficients can all be adjusted.

[0041] Secondly, embodiments of this application provide an image-based defect index determination device, the device comprising:

[0042] The reconstruction module is used to reconstruct the first image to be identified and obtain the first reconstructed image.

[0043] The calculation module is used to perform difference calculation on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel.

[0044] The first determining module is used to determine the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel.

[0045] The second determining module is used to determine the defect index of the first image to be identified based on the total number of pixels of the plurality of defective pixels and the virtual pixel value of each defective pixel.

[0046] Thirdly, embodiments of this application provide an electronic device, the device comprising:

[0047] Memory, used to store computer programs;

[0048] A processor is configured to execute the computer program to cause the device to perform the image-based defect index determination method described in the first aspect above.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image-based defect index determination method described in the first aspect.

[0050] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0051] This application provides a method and apparatus for determining an image-based defect index. In this method, a first image to be identified is reconstructed to obtain a first reconstructed image; the difference between the first image to be identified and the first reconstructed image is calculated to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel; based on the pixel difference value of each defective pixel, a virtual pixel value for each defective pixel is determined; and based on the total number of defective pixels and the virtual pixel value of each defective pixel, the defect index of the first image to be identified is determined. It can be seen that after reconstructing the image to be identified to obtain the reconstructed image, this method first obtains the defective pixels through the difference between the image to be identified and the reconstructed image, that is, it not only determines that the image to be identified is an abnormal image, but also determines the location of the defective pixels; furthermore, it determines the defect index of the image to be identified by using the total number of defective pixels and the virtual pixel value of each defective pixel to characterize the degree of defect in the image to be identified. This method not only better meets the needs of real-world scenarios but also facilitates practical deployment. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This application scenario illustrates an image-based defect index determination method provided in this embodiment.

[0054] Figure 2 A flowchart illustrating an image-based defect index determination method provided in this application embodiment;

[0055] Figure 3 This is a schematic diagram of the structure of an image-based defect index determination method device provided in an embodiment of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0057] Current image defect identification methods work as follows: First, a deep learning model is trained based on a normal image, with reconstruction as the goal. Then, the image to be identified is reconstructed using the trained deep learning model. Finally, the reconstructed image is compared with a normal image to determine if they are consistent, resulting in a first anomaly score. A second anomaly score is then obtained by comparing the reconstructed image with an abnormal image. The image to be identified is then determined to be either normal or abnormal based on either the first or second anomaly score. Existing methods can only determine whether the image is abnormal, but not the degree of anomaly. Therefore, there is an urgent need for a method that can not only identify whether the image contains defects (i.e., whether the image is abnormal), but also determine the degree of anomaly (i.e., the defect index), allowing users to take further action based on the specific situation.

[0058] To address the aforementioned issues, this application provides a method and apparatus for determining an image-based defect index. The method involves: reconstructing a first image to be identified to obtain a first reconstructed image; calculating the difference between the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference corresponding to each defective pixel; determining the virtual pixel value of each defective pixel based on its pixel difference; and determining the defect index of the first image to be identified based on the total number of defective pixels and the virtual pixel value of each defective pixel. As can be seen, after reconstructing the image to be identified to obtain a reconstructed image, this method first obtains the defective pixels through the difference between the image to be identified and the reconstructed image, thus not only identifying the image to be identified as an abnormal image but also determining the location of the defective pixels. Furthermore, by using the total number of defective pixels and the virtual pixel value of each defective pixel, the defect index of the image to be identified is determined to characterize the degree of defect in the image to be identified. This method not only better meets the needs of real-world scenarios but also facilitates practical deployment.

[0059] For example, one scenario in the embodiments of this application can be applied to, such as Figure 1The scenario shown includes a database 101 and a server 102. The database 101 includes a first image to be identified. The server 102 uses the implementation method provided in this application to obtain the first image to be identified from the database 101 and uses the implementation method provided in this application to determine the defect index of the first image to be identified.

[0060] First, in the above application scenarios, although the action descriptions of the implementation methods provided in this application are executed by the server 102, the implementation methods of this application are not limited in terms of the execution subject, as long as the actions disclosed in the implementation methods provided in this application are executed.

[0061] Secondly, the above scenario is only one example provided by the embodiments of this application, and the embodiments of this application are not limited to this scenario.

[0062] The following detailed description, in conjunction with the accompanying drawings, illustrates the specific implementation of the image-based defect index determination method and apparatus in this application.

[0063] See Figure 2 The figure is a flowchart of an image-based defect index determination method provided in an embodiment of this application, combined with... Figure 2 As shown, it can specifically include:

[0064] S201: Perform image reconstruction on the first image to be identified to obtain the first reconstructed image.

[0065] Image reconstruction is performed on the first image to be identified to obtain a reconstructed image corresponding to the first image to be identified. Here, the image to be identified refers to the image whose defects need to be identified, and the reconstructed image refers to the image with almost no defects obtained after reconstructing the image to be identified.

[0066] In this application, the process of obtaining the first reconstructed image is not specifically limited. For ease of understanding, a possible implementation method is described below.

[0067] In one possible implementation, the size of the first image to be identified is first measured. If its size is larger than a preset size, the first image to be identified can be cropped to obtain multiple smaller second images to be identified. Then, image reconstruction is performed on the multiple second images to obtain multiple second reconstructed images corresponding to the multiple second images to be identified. Finally, the multiple second reconstructed images are stitched together according to the cropping strategy of the first image to be identified to obtain a first reconstructed image corresponding to the first image to be identified. Therefore, S201 may specifically include: if the size of the first image to be identified is larger than a preset size, cropping the first image to be identified according to a preset strategy to obtain multiple second images to be identified; performing image reconstruction on each of the multiple second images to obtain multiple second reconstructed images; and stitching the multiple second reconstructed images together according to a preset strategy to obtain a first reconstructed image. Cropping the first image to be identified before image reconstruction can greatly improve the speed of image reconstruction, so as to facilitate the execution of subsequent steps.

[0068] For example, the preset size can be obtained based on the results of multiple image reconstructions. Images of different sizes are reconstructed, and the reconstructed images of different sizes are compared. The preset size is then set based on the size of the reconstructed image with poor reconstruction results. The preset strategy could be to crop the first image to be recognized into multiple squares of equal size, or to randomly crop the first image to be recognized. Of course, other methods can also be used as described above, without affecting the implementation of the embodiments of this application.

[0069] S202: Perform difference calculation on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel.

[0070] Based on all pixels in the first image to be identified and all pixels in the first reconstructed image, the difference between the pixel values ​​of the corresponding pixels is calculated to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel.

[0071] S203: Determine the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel.

[0072] Based on the pixel difference value of each defective pixel obtained from the difference calculation, the virtual pixel value of each defective pixel is determined.

[0073] In the embodiments of this application, step S203 can be implemented in several ways, which will be described below. It should be noted that the implementation methods given below are only illustrative examples and do not represent all implementation methods of the embodiments of this application.

[0074] In one possible implementation, the absolute value of the pixel difference for each defective pixel is determined as a virtual pixel value for that defective pixel. This allows for lateral comparison of defects between different image datasets based on the pixel differences.

[0075] In one possible implementation, filtering is performed based on the pixel difference or the absolute value of the pixel difference for each defective pixel to obtain a filtered value for each defective pixel. The absolute value of the filtered value for each defective pixel is then determined as the virtual pixel value for that defective pixel. This allows for subsequent filtering based on the filtered values ​​of multiple defective pixels to remove defective pixels with indistinct defect characteristics and retain those that represent the core defects.

[0076] In one possible implementation, filtering is performed based on the pixel difference or the absolute value of the pixel difference for each defective pixel to obtain a filtered value for each defective pixel; normalization is then performed based on the absolute value of the filtered value for each defective pixel to obtain a normalized value for each defective pixel; the normalized value for each defective pixel is then determined as a virtual pixel value for each defective pixel. Further normalization of the filtered value, resulting in a virtual pixel value for each defective pixel, ensures that the virtual pixel value for each defective pixel is within the range of 0 to 1, thus narrowing the difference space and facilitating subsequent processing of multiple defective pixels.

[0077] In this application, the filtering process is not specifically limited. For ease of understanding, a possible implementation method is described below.

[0078] In one possible implementation, Gaussian filtering is performed on the pixel difference or absolute value of the pixel difference for each defective pixel to obtain a filtered value for each defective pixel. Gaussian filtering refers to the process of weighting multiple defective pixels according to a Gaussian kernel. The filtered value of each defective pixel is obtained by weighting its own pixel value and the pixel values ​​of other defective pixels in its neighborhood. Specifically, the Gaussian kernel is used to scan each defective pixel using its center point, then the region of the Gaussian kernel excluding the center point is used to determine the pixels in the neighborhood of each defective pixel. The filtered value obtained by weighting the pixel values ​​of the defective pixel at the center of the Gaussian kernel and the pixels in the vicinity of the Gaussian kernel is then used to replace the pixel value of the defective pixel at the center of the Gaussian kernel.

[0079] In one possible implementation, the filtering process can also employ linear filtering methods such as box filtering and mean filtering, or nonlinear filtering methods such as median filtering and bilateral filtering. Of course, other methods can also be used without affecting the implementation of the embodiments of this application.

[0080] S204: Determine the defect index of the first image to be identified based on the total number of pixels of multiple defective pixels and the virtual pixel value of each defective pixel.

[0081] In this application embodiment, the process of determining the defect index of the first image to be identified is not specifically limited. For ease of understanding, the following description is based on a possible implementation method.

[0082] In one possible implementation, processing can be performed based on the virtual pixel value of each defective pixel. Defective pixels with virtual pixel values ​​less than a preset threshold are excluded, i.e., defective pixels with indistinct defect features are eliminated, resulting in multiple remaining defective pixels that can characterize the core defect. Therefore, S204 may specifically include: excluding defective pixels with virtual pixel values ​​less than a preset threshold to obtain multiple remaining defective pixels; and determining a defect index based on the total number of remaining defective pixels and the virtual pixel value of each remaining defective pixel.

[0083] In this application embodiment, the specific process for determining the defect index of the first image to be identified is not limited. For ease of understanding, a possible implementation method is described below.

[0084] In one possible implementation, the defect index of the first image to be identified can be determined based on the total number of defective pixels, the pixel difference of each defective pixel, an adjustable coefficient configured for the total number of defective pixels, an adjustable coefficient configured for the virtual pixel value of each defective pixel, and a preset defect index formula. Therefore, S204 specifically includes: determining the defect index of the first image to be identified based on the total number of defective pixels, the virtual pixel value of each defective pixel, a first coefficient, a second coefficient, and a preset defect index formula, wherein the first coefficient is an adjustable coefficient based on the total number of defective pixels, and the second coefficient is an adjustable coefficient based on the virtual pixel value of each defective pixel.

[0085] In this application, the embodiment may not be specifically limited to the preset defect index formula. For ease of understanding, the following description is based on a possible implementation method.

[0086] In one possible implementation, the formula for the preset defect index can be:

[0087] Y = A × X area +B×∑X 2

[0088] Where Y is the defect index, A is the first coefficient, and X is the defect index. area Let B be the total number of defective pixels, B be the second coefficient, and X be the virtual pixel value of each defective pixel.

[0089] The first and second coefficients can be set by the user according to their emphasis on the defect area and defect brightness.

[0090] In addition, in an optional embodiment of this application, multiple mask images can be filled with multiple normal images and random values ​​within a preset pixel value range to train a preset encoding and decoding model, thereby obtaining a trained encoding and decoding model, which is then determined as the reconstructed image model. Correspondingly, the first image to be identified can be reconstructed using the reconstructed image model to obtain a first reconstructed image. Therefore, the image-based defect index determination method may further include: training a preset encoding and decoding model based on a normal image and a random mask image of the normal image to obtain a trained encoding and decoding model as the reconstructed image model, wherein the pixel values ​​of the random mask image are random values ​​within a preset pixel value range; correspondingly, S201 may specifically be: reconstructing the first image to be identified using the reconstructed image model to obtain a first reconstructed image.

[0091] In one possible implementation, the process of obtaining a random mask image can be as follows: First, a variety of different shapes are set to form a shape library, which may include circles, rectangles, hammer shapes, and various random shapes; then, the random shapes in the shape library are filled with any value within a preset pixel value range, and the pixel values ​​of each pixel in each shape may be exactly the same or different; finally, multiple random mask images are obtained to train a preset encoder-decoder network. Of course, other methods can also be used, which does not affect the implementation of the embodiments of this application.

[0092] In this application, the training process of the preset encoding and decoding network is not specifically limited. For ease of understanding, a possible implementation method is described below.

[0093] In one possible implementation, a preset encoder-decoder model is trained based on a normal image, a random mask image of the normal image, and a preset loss function. Specifically, the random mask image is first superimposed on the normal image to obtain a defective image; then, multiple defective images are input into the preset encoder-decoder network to obtain multiple reconstructed images corresponding to the multiple defective images; next, the multiple reconstructed images are compared with the corresponding multiple normal images, and the parameters of the preset encoder-decoder network are adjusted based on the comparison results; finally, the trained preset encoder-decoder network is used as the reconstructed image model. Of course, other methods can also be used without affecting the implementation of the embodiments of this application.

[0094] In this application, the embodiment may not be specifically limited to a preset loss function. For ease of understanding, a possible implementation method will be described below.

[0095] In one possible implementation, the preset loss function can specifically be:

[0096] Loss = C × PSNR + D × SSIM + E × JND

[0097] In this model training algorithm, Loss is the preset loss function, C is the third coefficient, PSNR is the peak signal-to-noise ratio between the normal image and its corresponding reconstructed image during training, D is the fourth coefficient, SSIM is the structural similarity between the normal image and its corresponding reconstructed image during training, E is the fifth coefficient, and JND is the minimum perceptible difference between the normal image and its corresponding reconstructed image during training. The third, fourth, and fifth coefficients are all adjustable. PSNR refers to the overall brightness consistency of the image, SSIM refers to the consistency of brightness contrast, shape, and structure in different parts of the image, and JND refers to the complexity of the image texture. By considering multiple image characteristics and setting a preset loss function for model training, the training effect of the model is improved.

[0098] In one possible implementation, multiple defect images obtained by superimposing normal images and random mask images can be input into a preset encoder-decoder network for model training. Based on multiple comparison results between the reconstructed images and normal images, a preset loss function Loss is calculated. Then, multiple defect images are input into the preset encoder-decoder network multiple times, and the preset loss function Loss is calculated for each model training session. After each model training session, the model parameters are adjusted, and finally, the trained preset encoder-decoder network is determined as the reconstructed image model.

[0099] In one possible implementation, the model can be trained multiple times according to a preset number of times, and a preset loss function Loss can be calculated uniformly; then, the model can be trained for a preset number of times again, and the preset loss function Loss can be calculated uniformly after each preset number of training sessions; after each multiple training sessions, the model parameters can be adjusted, and finally, the trained preset encoder-decoder network can be determined as the image reconstruction model.

[0100] In one possible implementation, the adjustment method for the third, fourth, and fifth coefficients of the preset loss function can be as follows: Before training the preset encoder-decoder network each time based on multiple defect images obtained by superimposing normal images and random mask images, the third, fourth, and fifth coefficients are set according to the characteristics of different defect images. For example, when there is only overall brightness variation in the defect image, the third coefficient of PSNR can be adjusted to a larger coefficient; when the brightness contrast between different parts of the defect image is relatively strong, the fourth coefficient of SSIM can be adjusted to a larger coefficient; when the texture of the defect image is relatively complex, the fifth coefficient of JND can be adjusted to a larger coefficient. The sum of the third, fourth, and fifth coefficients can be equal to 1 or greater than 1, and the specific ratio is set according to the characteristics of the image.

[0101] For example, when the texture of the defective image is complex and the contrast between light and dark in different parts of the image is strong, the third coefficient of PSNR can be set to 0.1, the fourth coefficient of SSIM can be set to 0.4, and the third coefficient of JND can be set to 0.6.

[0102] Of course, other methods can also be used as described above, which does not affect the implementation of the embodiments of this application.

[0103] Based on the above-mentioned S201-S204, it is known that in this embodiment, image reconstruction is performed on the first image to be identified to obtain a first reconstructed image; the difference between the first image to be identified and the first reconstructed image is calculated to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel; based on the pixel difference value of each defective pixel, the virtual pixel value of each defective pixel is determined; and based on the total number of pixels of multiple defective pixels and the virtual pixel value of each defective pixel, the defect index of the first image to be identified is determined. It can be seen that after image reconstruction of the image to be identified to obtain the reconstructed image, this method first obtains the defective pixels through the difference between the image to be identified and the reconstructed image, that is, it not only determines that the image to be identified is an abnormal image, but also determines the location of the defective pixels; in addition, the defect index of the image to be identified is determined by the total number of pixels of the defective pixels and the virtual pixel value of each defective pixel to characterize the degree of defect of the image to be identified. This method is not only more in line with the needs of real-world scenarios, but also easier to deploy.

[0104] The above describes some specific implementations of the image-based defect index determination method provided in this application. Based on this, this application also provides a corresponding apparatus. The apparatus provided in this application will be described below from the perspective of functional modularity.

[0105] See Figure 3The figure is a schematic diagram of the structure of an image-based defect index determination device 300 provided in an embodiment of this application. The device 300 may include:

[0106] Reconstruction module 301 is used to reconstruct the first image to be identified and obtain the first reconstructed image;

[0107] The calculation module 302 is used to perform difference calculation on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel.

[0108] The first determining module 303 is used to determine the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel.

[0109] The second determining module 304 is used to determine the defect index of the first image to be identified based on the total number of pixels of multiple defective pixels and the virtual pixel value of each defective pixel.

[0110] In this embodiment, through the cooperation of four modules—reconstruction module 301, calculation module 302, first determination module 303, and second determination module 304—after reconstructing the image to be identified, the defective pixels are first obtained by the difference between the image to be identified and the reconstructed image. That is, not only is the image to be identified identified as an abnormal image, but the location of the defective pixels can also be determined. In addition, the defect index of the image to be identified is determined by the total number of defective pixels and the virtual pixel value of each defective pixel, so as to characterize the degree of defect of the image to be identified. This not only better meets the needs of real-world scenarios, but also facilitates deployment.

[0111] As one implementation method, the first determining module 303 may specifically include:

[0112] The first determining unit is used to determine the virtual pixel value of each defective pixel by the absolute value of the pixel difference of each defective pixel.

[0113] As one implementation method, the first determining module 303 may specifically include:

[0114] The first filtering unit is used to perform filtering based on the pixel difference or the absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel.

[0115] The second determining unit is used to determine the virtual pixel value of each defective pixel by the absolute value of the filtered value of each defective pixel.

[0116] As one implementation method, the first determining module 303 may specifically include:

[0117] The second filtering unit is used to perform filtering based on the pixel difference or the absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel.

[0118] The normalization unit is used to normalize the filter value of each defective pixel to obtain the normalized value of each defective pixel.

[0119] The third determining unit is used to determine the normalized value of each defective pixel as the virtual pixel value of each defective pixel.

[0120] As one implementation method, the first filtering unit or the second filtering unit can specifically be used for:

[0121] Gaussian filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel.

[0122] As one implementation, the second determining module 304 may specifically include:

[0123] The obtaining unit is used to exclude defective pixels whose virtual pixel values ​​are less than a preset threshold, and obtain the remaining multiple defective pixels;

[0124] The fourth determining unit is used to determine the defect index based on the total number of remaining defective pixels and the virtual pixel value of each remaining defective pixel.

[0125] As one implementation, the second determining module 304 may specifically include:

[0126] The fifth determining unit is used to determine the defect index of the first image to be identified based on the total number of pixels of multiple defective pixels, the virtual pixel value of each defective pixel, the first coefficient, the second coefficient, and the preset defect index formula, wherein the first coefficient is an adjustable coefficient based on the total number of pixels of multiple defective pixels, and the second coefficient is an adjustable coefficient based on the virtual pixel value of each defective pixel.

[0127] As one implementation method, the preset defect index formula of the fifth determining unit can specifically be:

[0128] Y = A × X area +B×∑X 2

[0129] Where Y is the defect index, A is the first coefficient, and X is the defect index. area Let B be the total number of defective pixels, B be the second coefficient, and X be the virtual pixel value of each defective pixel.

[0130] As one implementation method, the reconstruction module 301 may specifically include:

[0131] The cropping unit is used to crop the first image to be recognized according to a preset strategy if the size of the first image to be recognized is larger than a preset size, thereby obtaining multiple second images to be recognized.

[0132] The reconstruction unit is used to perform image reconstruction based on multiple second images to be identified, thereby obtaining multiple second reconstructed images;

[0133] The stitching unit is used to stitch together multiple second reconstructed images according to a preset strategy to obtain a first reconstructed image.

[0134] As one implementation, the image-based defect index determination device 300 may further include:

[0135] The training module is used to train a preset encoding and decoding model based on a normal image and a random mask image of the normal image, and obtain the trained encoding and decoding model as the reconstructed image model. The pixel values ​​of the random mask image are random values ​​within a preset pixel value range.

[0136] Accordingly, the reconstruction module 301 can be specifically used for:

[0137] The first image to be identified is reconstructed by reconstructing the image model to obtain the first reconstructed image.

[0138] As one implementation method, the training module can specifically be used for:

[0139] A pre-defined encoder-decoder model is trained based on a normal image, a random mask image of the normal image, and a pre-defined loss function. The trained encoder-decoder model is then used as the reconstructed image model. The pre-defined loss function is as follows:

[0140] Loss = C × PSNR + D × SSIM + E × JND

[0141] Wherein, Loss is the preset loss function, C is the third coefficient, PSNR is the peak signal-to-noise ratio of the normal image and the corresponding reconstructed image during training, D is the fourth coefficient, SSIM is the structural similarity between the normal image and the corresponding reconstructed image during training, E is the fifth coefficient, and JND is the minimum perceptible difference between the normal image and the corresponding reconstructed image during training. The third, fourth, and fifth coefficients can all be adjusted.

[0142] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0143] The device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to enable the device to perform the image-based defect index determination method according to any embodiment of this application.

[0144] The computer-readable storage medium stores a computer program that, when run by a processor, implements the image-based defect index determination method described in any embodiment of this application.

[0145] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0146] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0147] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0148] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining a defect index based on an image, characterized in that, The method includes: The first image to be identified is reconstructed to obtain the first reconstructed image; Difference calculation is performed on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel; Based on the pixel difference of each defective pixel, determine the virtual pixel value of each defective pixel; The defect index of the first image to be identified is determined based on the total number of defective pixels and the virtual pixel value of each defective pixel. The step of determining the defect index of the first image to be identified based on the total number of defective pixels and the virtual pixel value of each defective pixel is as follows: The defect index of the first image to be identified is determined based on the total number of pixels of the plurality of defective pixels, the virtual pixel value of each defective pixel, a first coefficient, a second coefficient, and a preset defect index formula, wherein the first coefficient is an adjustable coefficient based on the total number of pixels of the plurality of defective pixels, and the second coefficient is an adjustable coefficient based on the virtual pixel value of each defective pixel.

2. The method according to claim 1, characterized in that, The process of determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel is specifically as follows: The absolute value of the pixel difference of each defective pixel is determined as the virtual pixel value of each defective pixel.

3. The method according to claim 1, characterized in that, Determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel includes: Filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel; The absolute value of the filtered value of each defective pixel is determined as the virtual pixel value of each defective pixel.

4. The method according to claim 1, characterized in that, Determining the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel includes: Filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel; The normalized value of each defective pixel is obtained by normalizing the absolute value of the filtered value of each defective pixel. The normalized value of each defective pixel is determined as the virtual pixel value of each defective pixel.

5. The method according to claim 3 or 4, characterized in that, The filtering process, which involves performing filtering on the pixel difference or absolute value of the pixel difference for each defective pixel to obtain a filtered value for each defective pixel, specifically involves: Gaussian filtering is performed on the pixel difference or absolute value of the pixel difference of each defective pixel to obtain the filtered value of each defective pixel.

6. The method according to claim 1, characterized in that, Determining the defect index of the first image to be identified based on the total number of defective pixels and the virtual pixel value of each defective pixel includes: Defective pixels with virtual pixel values ​​less than a preset threshold are excluded to obtain the remaining defective pixels. The defect index is determined based on the total number of remaining defective pixels and the virtual pixel value of each remaining defective pixel.

7. The method according to claim 1, characterized in that, The formula for the preset defect index is as follows: Where Y is the defect index, A is the first coefficient, and X... area B is the total number of defective pixels, B is the second coefficient, and X is the virtual pixel value of each defective pixel.

8. The method according to claim 1, characterized in that, The step of reconstructing the first image to be identified to obtain the first reconstructed image includes: If the size of the first image to be identified is larger than the preset size, the first image to be identified is cropped according to the preset strategy to obtain multiple second images to be identified; Image reconstruction is performed on the plurality of second images to be identified to obtain a plurality of second reconstructed images; The plurality of second reconstructed images are stitched together according to the preset strategy to obtain the first reconstructed image.

9. The method according to claim 1, characterized in that, The method further includes: A preset encoding and decoding model is trained based on a normal image and a random mask image of the normal image to obtain the trained encoding and decoding model as a reconstructed image model. The pixel values ​​of the random mask image are random values ​​within a preset pixel value range. The process of reconstructing the first image to be identified to obtain the first reconstructed image specifically involves: The first image to be identified is reconstructed using the reconstructed image model to obtain the first reconstructed image.

10. The method according to claim 9, characterized in that, The step of training a preset encoding / decoding model based on a normal image and a random mask image of the normal image to obtain the trained encoding / decoding model as the reconstructed image model includes: The preset encoding / decoding model is trained based on the normal image, a random mask image of the normal image, and a preset loss function to obtain the trained encoding / decoding model as the reconstructed image model. Specifically, the preset loss function is: Wherein, Loss is the preset loss function, C is the third coefficient, PSNR is the peak signal-to-noise ratio of the normal image and the reconstructed image corresponding to the normal image during training, D is the fourth coefficient, SSIM is the structural similarity of the normal image and the reconstructed image corresponding to the normal image during training, E is the fifth coefficient, and JND is the minimum perceptible difference between the normal image and the reconstructed image corresponding to the normal image during training. The third, fourth, and fifth coefficients can all be adjusted.

11. An image-based defect index determination device, characterized in that, The device includes: The reconstruction module is used to reconstruct the first image to be identified and obtain the first reconstructed image. The calculation module is used to perform difference calculation on the first image to be identified and the first reconstructed image to determine multiple defective pixels and the pixel difference value corresponding to each defective pixel. The first determining module is used to determine the virtual pixel value of each defective pixel based on the pixel difference of each defective pixel. The second determining module is used to determine the defect index of the first image to be identified based on the total number of pixels of the plurality of defective pixels and the virtual pixel value of each defective pixel. The second determining module is specifically used to determine the defect index of the first image to be identified based on the total number of pixels of the plurality of defective pixels, the virtual pixel value of each defective pixel, a first coefficient, a second coefficient, and a preset defect index formula, wherein the first coefficient is an adjustable coefficient based on the total number of pixels of the plurality of defective pixels, and the second coefficient is an adjustable coefficient based on the virtual pixel value of each defective pixel.

12. An electronic device, characterized in that, The device includes: Memory, used to store computer programs; A processor for executing the computer program to cause the device to perform the steps of the image-based defect index determination method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image-based defect index determination method as described in any one of claims 1 to 10.

Images