Defect depth determination method and device and electronic equipment
By determining the average grayscale of defect edges, bright and dark areas in image processing, and calculating the weighted grayscale contrast ratio in combination with the background influence weight, the error problem of conventional grayscale contrast algorithms in the case of high defect polymorphism and background complexity is solved, and more accurate defect depth information acquisition is achieved.
Patent Information
- Application Number
- CN202411994931.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-29
AI Technical Summary
In the prior art, when determining the depth of the defect through conventional grayscale contrast algorithms, there are large errors and relative linearity cannot be guaranteed. Especially when the defect polymorphism and background complexity are high, the application scenarios are becoming narrower and narrower.
A method for determining the depth of defects is proposed. By determining the edge area, bright area and dark area from the original defect area of the image, the respective average grayscale is calculated, and the weighted grayscale contrast is calculated based on these average grayscales, area area and background influence weights, thereby obtaining more accurate defect depth information.
This method reduces background interference around the defective area, can adapt to background changes, and is also effective when used in overexposure, overdarkness and edge backgrounds. It takes into account the influence of the grayscale difference of the internal bright and dark area and the area of the bright and dark area. Compared with conventional methods, the error is smaller and the accuracy is higher.
Smart Images
Figure CN119904495A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular relates to a method, device and electronic device for determining defect depth. Background Art
[0002] In industrial quality inspection scenarios, 2D images are generally taken of products, and defects of the products are detected by identifying the 2D images. The depth information of the defects is an important indicator for judging the defects.
[0003] The related technology determines the depth information of the defect through a conventional grayscale contrast algorithm. The conventional grayscale contrast algorithm constructs a linear relationship between the average grayscale difference (difference in average grayscale) between the defect area and the background area and the depth information. However, due to the polymorphism of defects and the complexity of the background, the conventional grayscale contrast algorithm will cause large errors and cannot guarantee relative linearity, and the application scenarios are becoming increasingly narrow. Summary of the invention
[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a method, device and electronic device for determining the depth of a defect, which reduces the interference of the background around the defect area, can adapt to the changes in the background, and also considers the influence of the grayscale difference and the area of the internal bright and dark areas, so that the depth information of the defect obtained is more accurate.
[0005] In a first aspect, the present application provides a method for determining a defect depth, the method comprising:
[0006] Determine a defect edge region, a defect bright region, and a defect dark region from the original defect region of the image;
[0007] Determine a first average grayscale of a defect edge region, a second average grayscale of a defect bright region, and a third average grayscale of a defect dark region;
[0008] Obtaining a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale;
[0009] Obtaining a weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area;
[0010] According to the weighted grayscale contrast of the original defect area, the depth information of the defect corresponding to the original defect area is obtained.
[0011] According to the defect depth determination method of the present application, the first influence weight of the background of the image where the original defect area is located is determined by the first average grayscale and the target grayscale of the defect edge area, and the weighted grayscale contrast of the original defect area is obtained according to the first influence weight, the second average grayscale of the defect bright area, the third average grayscale of the defect dark area, the area of the defect bright area and the area of the defect dark area. The weighted grayscale contrast reduces the interference of the background around the defect area, can adapt to the changes in the background, can be used in overexposure, overdarkness, and edge backgrounds, and also considers the influence of the grayscale difference between internal bright and dark areas and the area of bright and dark areas. Compared with the depth information determined by the conventional grayscale contrast algorithm, the depth information of the defect determined by the weighted grayscale contrast is more accurate and has a smaller error with the actual depth information.
[0012] According to one embodiment of the present application, determining a defect edge region from an original defect region of an image includes:
[0013] Determine the number of indented pixels according to the size of the original defect area;
[0014] According to the number of shrunk pixels, the original defect area is shrunk to obtain the shrunk defect area;
[0015] The original defect area and the indented defect area are subjected to difference processing to obtain the defect edge area.
[0016] According to one embodiment of the present application, the weighted grayscale contrast of the original defect area is obtained according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area, including:
[0017] According to the first influence weight and the first average grayscale, a first grayscale contrast is obtained; the first grayscale contrast is a compensation grayscale contrast of the background to the original defect area;
[0018] According to the second average grayscale, the third average grayscale, the area of the defect bright region and the area of the defect dark region, a second grayscale contrast is obtained; the second grayscale contrast is the grayscale contrast inside the original defect region;
[0019] The sum of the first grayscale contrast and the second grayscale contrast is used as the weighted grayscale contrast of the original defect area.
[0020] According to one embodiment of the present application, obtaining a second grayscale contrast ratio according to the second average grayscale, the third average grayscale, the area of the defect bright region, and the area of the defect dark region includes:
[0021] Obtaining a grayscale difference between a bright defect area and a dark defect area according to the second average grayscale and the third average grayscale;
[0022] According to the area of the defect bright area and the area of the defect dark area, second influence weights of the areas of the defect bright area and the defect dark area are obtained;
[0023] The product of the grayscale difference between the bright and dark areas and the second influence weight is taken as the second grayscale contrast.
[0024] According to one embodiment of the present application, determining a defect bright area and a defect dark area from an original defect area of an image includes:
[0025] According to the average grayscale of the original defect area, the original defect area is divided into a defect bright area and a defect dark area;
[0026] The grayscale of each pixel included in the defect dark area is less than the average grayscale of the original defect area;
[0027] The grayscale of each pixel included in the defect bright area is smaller than the average grayscale of the original defect area.
[0028] According to an embodiment of the present application, obtaining a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale includes:
[0029] The ratio of the target grayscale to the first average grayscale is used as the first influence weight of the background of the image where the original defective area is located.
[0030] According to an embodiment of the present application, before obtaining the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area, the method further includes:
[0031] Obtaining depth information of defects in a plurality of defective products measured;
[0032] Collect sample defect images corresponding to multiple defective products respectively;
[0033] Determine the weighted grayscale contrast of the defect area in each defect sample image;
[0034] Performing linear fitting on the weighted grayscale contrast and depth information of the defective area in the defective sample image of each defective product to obtain a linear fitting function;
[0035] According to the weighted grayscale contrast of the original defect area, the depth information of the defect corresponding to the original defect area is obtained, including:
[0036] The weighted grayscale contrast of the original defect area is substituted into the linear fitting function to obtain the depth information of the defect corresponding to the original defect area.
[0037] In a second aspect, the present application provides a device for determining defect depth, the device comprising:
[0038] A first processing module is used to determine a defect edge area, a defect bright area and a defect dark area from an original defect area of the image;
[0039] A second processing module, used to determine a first average grayscale of a defect edge region, a second average grayscale of a defect bright region, and a third average grayscale of a defect dark region;
[0040] A third processing module, used for obtaining a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale;
[0041] A fourth processing module, configured to obtain a weighted grayscale contrast of an original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of a defect bright area, and the area of a defect dark area;
[0042] The fifth processing module is used to obtain the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area.
[0043] According to the defect depth determination device of the present application, the first influence weight of the background of the image where the original defect area is located is determined by the first average grayscale and the target grayscale of the defect edge area, and the weighted grayscale contrast of the original defect area is obtained according to the first influence weight, the second average grayscale of the defect bright area, the third average grayscale of the defect dark area, the area of the defect bright area and the area of the defect dark area. The weighted grayscale contrast reduces the interference of the background around the defect area, can adapt to the changes in the background, can be used in overexposure, overdarkness, and edge backgrounds, and also considers the influence of the grayscale difference between internal bright and dark areas and the area of bright and dark areas. Compared with the depth information determined by the conventional grayscale contrast algorithm, the depth information of the defect determined by the weighted grayscale contrast is more accurate and has a smaller error with the actual depth information.
[0044] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the defect depth provided in the first aspect above is implemented.
[0045] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining the defect depth provided in the first aspect above.
[0046] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method for determining the defect depth as provided in the first aspect.
[0047] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining the defect depth provided in the first aspect above.
[0048] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0049] The first influence weight of the background of the image where the original defect area is located is determined by the first average grayscale and the target grayscale of the defect edge area. The weighted grayscale contrast of the original defect area is obtained according to the first influence weight, the second average grayscale of the defect bright area, the third average grayscale of the defect dark area, the area of the defect bright area and the area of the defect dark area. The weighted grayscale contrast reduces the interference of the background around the defect area and can adapt to changes in the background. It can be used in overexposure, overdarkness, and edge backgrounds. The influence of the grayscale difference between internal bright and dark areas and the area of bright and dark areas is also considered. Compared with the depth information determined by the conventional grayscale contrast algorithm, the depth information of the defect determined by the weighted grayscale contrast is more accurate and has a smaller error with the actual depth information.
[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0052] Figure 1 This is one of the flow charts of the method for determining the defect depth provided in the embodiment of the present application;
[0053] Figure 2 It is one of the schematic diagrams of a grayscale image containing defects provided in an embodiment of the present application;
[0054] Figure 3 This is a second schematic diagram of a grayscale image containing defects provided in an embodiment of the present application;
[0055] Figure 4 is a schematic diagram of multiple defect sample images collected for each defect level provided in an embodiment of the present application;
[0056] Figure 5 It is a schematic diagram of the original defect area, the defect area after shrinkage, and the defect edge area provided in the embodiment of the present application;
[0057] Figure 6 This is the second flow chart of the method for determining the defect depth provided in the embodiment of the present application;
[0058] Figure 7 is a schematic diagram of the structure of a device for determining defect depth provided in an embodiment of the present application;
[0059] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0061] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0062] As people's living standards improve and the electronic products industry continues to develop and optimize, consumers are pursuing more and more extreme requirements for beauty. Appearance quality has become an important factor for consumers to choose electronic products, which has pushed manufacturers to have higher and higher requirements for product surface defect detection capabilities, even reaching the limit of the human eye. The defects in product appearance are diverse, multi-dimensional, and multi-angle.
[0063] Multidimensionality refers to the determination of defects, which needs to be judged from multiple dimensions such as area, length, width, depth, curvature, etc. Since the images taken of the product are 2D images, the depth information of the defect is generally determined by a conventional grayscale contrast algorithm. The conventional grayscale contrast algorithm constructs a linear relationship between the average grayscale difference (difference in average grayscale) of the defect area and the background area and the depth information. The two are usually positively correlated. However, defects are multi-morphological and background complex. Conventional grayscale contrast algorithms will cause large errors and cannot guarantee relative linearity, and the application scenarios are becoming increasingly narrow.
[0064] See also Figure 2 , the present application embodiment provides one of the schematic diagrams of a grayscale image containing defects, Figure 2The defect in the middle is an extremely serious defect with a deep depth. However, due to the grayscale offset between the dark area of the inner circle defect and the bright area of the defect, the average grayscale difference between the defect and the background is extremely small, which causes a huge error in the characterization of the depth of the defect, and the contrast of the deep defect is small.
[0065] See also Figure 3 , the present application embodiment provides a second schematic diagram of a grayscale image containing defects, Figure 3 The defect in the figure is a shallow defect, but because the defect is close to the edge of the complex background, the background structure changes greatly, resulting in an increase in the average grayscale difference between the defect and the background, and the contrast of the shallow defect is even greater.
[0066] To solve the above problems, embodiments of the present application provide a method for determining defect depth, an apparatus for determining defect depth, an electronic device, and a readable storage medium.
[0067] In combination with the accompanying drawings, the defect depth determination method, defect depth determination device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.
[0068] The method for determining the defect depth may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.
[0069] The method for determining the defect depth provided in the embodiments of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the method for determining the defect depth. The electronic devices mentioned in the embodiments of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The method for determining the defect depth provided in the embodiments of the present application is described below using an electronic device as an example of an execution subject.
[0070] like Figure 1 As shown, the method for determining the defect depth includes: step 110, step 120, step 130, step 140 and step 150.
[0071] Step 110: determine a defect edge region, a defect bright region, and a defect dark region from the original defect region of the image.
[0072] In the process of industrial quality inspection, images of the products to be inspected can be continuously taken. For images with defects, the original defect area can be cropped out from the image, where the original defect area is the area where the defect is located.
[0073] The defect edge area in the embodiment of the present application refers to the area corresponding to the edge of the original defect area. The defect edge area is usually narrow, and the grayscale of the defect edge area is affected by the background around the original defect area, which is one of the factors that cause errors in the calculated depth information.
[0074] The embodiment of the present application can determine the defect edge area in the original defect area through the existing artificial intelligence algorithm, and can also obtain the shrunk defect area by shrinking the original defect area (generally by 3-20 pixels). The difference area between the original defect area and the shrunk defect area is recorded as the defect edge area.
[0075] In the embodiment of the present application, the defective bright area and the defective dark area are divided based on the average grayscale of the defective area. Specifically, the average grayscale of the original defective area can be calculated based on the grayscale of the pixels contained in the original defective area. The pixels in the original defective area whose grayscale is less than the average grayscale of the original defective area can be divided into the defective bright area, and the pixels in the defective area whose grayscale is greater than the average grayscale of the original defective area can be divided into the defective bright area.
[0076] For pixels whose grayscale in the original defect area is equal to the average grayscale of the original defect area, they are classified into the defect bright area or the defect bright area.
[0077] Step 120 : Determine a first average grayscale of the defect edge region, a second average grayscale of the defect bright region, and a third average grayscale of the defect dark region.
[0078] Based on the number of pixels contained in the defect edge area, a portion of smaller grayscale pixels (such as the first 15% of the minimum grayscale pixels) and a portion of larger grayscale pixels (such as the last 15% of the maximum grayscale pixels) can be removed, and the average grayscale value of the remaining pixels is calculated as the first average grayscale of the defect edge area.
[0079] The second average grayscale of the defective bright area can be obtained according to the grayscales of the pixels included in the defective bright area.
[0080] A third average grayscale of the defective dark area may be obtained according to the grayscales of the pixels included in the defective dark area.
[0081] Step 130: Obtain a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale.
[0082] The target grayscale of the embodiment of the present application is a preset reference grayscale based on the grayscale of the background surrounding the original defect area. It is set according to actual conditions and can be equal to the grayscale of the surrounding background (the grayscale of the background can be represented by the average grayscale of the surrounding background), or it can be slightly larger or smaller than the grayscale of the background, for example, the target grayscale is 128.
[0083] The ratio of the target grayscale and the first average grayscale can be used as the first influence weight of the background of the image where the original defect area is located. The first influence weight participates in the subsequent weighted contrast calculation, so that when calculating the depth information of the defect, it can adapt to background changes and compensate for the influence of the background on the depth information calculation.
[0084] Step 140: Obtain a weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area.
[0085] Since the first average grayscale of the defect edge area is affected by the grayscale of the background surrounding the original defect area, and the first influence weight is to compensate for the influence of the background, the first grayscale contrast can be obtained based on the first influence weight and the first average grayscale; the first grayscale contrast is the compensatory grayscale contrast of the background to the original defect area.
[0086] Since the brightness and area occupied by the defective bright area and the defective dark area inside the original defect area are also likely to cause calculation errors of the depth information, in order to obtain more accurate depth information later, the grayscale difference between the bright and dark areas of the defective bright area and the defective dark area can be obtained according to the second average grayscale of the defective bright area and the third average grayscale of the defective dark area. According to the area of the defective bright area and the area of the defective dark area, the second influence weight of the area size of the defective bright area and the defective dark area is obtained. The area of the defective bright area and the defective dark area can be determined according to the number of pixels contained therein. Subsequently, the second grayscale contrast can be obtained according to the grayscale difference between the bright and dark areas and the second influence weight; the second grayscale contrast is the grayscale contrast inside the original defect area.
[0087] After the first grayscale contrast and the second grayscale contrast are obtained, the sum of the first grayscale contrast and the second grayscale contrast may be used as the weighted grayscale contrast of the original defective area.
[0088] Step 150: Obtain depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area.
[0089] The embodiment of the present application predetermines the linear relationship between the weighted grayscale contrast of the original defect area and the depth information of the defect. Specifically, the depth information of the defects in multiple defective products can be obtained; sample defect images corresponding to multiple defective products are collected; based on the process of the aforementioned steps 110 to 140, the weighted grayscale contrast of the defect area in each defect sample image is determined; then, the weighted grayscale contrast and depth information of the defect area in the defect sample image of each defective product can be linearly fitted to obtain a linear fitting function; the linear fitting function is used to characterize the linear relationship between the weighted grayscale contrast of the defect area and the depth information of the defect.
[0090] The weighted grayscale contrast of the original defect area can be substituted into the linear fitting function to accurately obtain the depth information of the defect corresponding to the original defect area.
[0091] The embodiment of the present application determines the first influence weight of the background of the image where the original defect area is located through the first average grayscale and the target grayscale of the defect edge area, and obtains the weighted grayscale contrast of the original defect area according to the first influence weight, the second average grayscale of the defect bright area, the third average grayscale of the defect dark area, the area of the defect bright area, and the area of the defect dark area. The weighted grayscale contrast reduces the interference of the background around the defect area, can adapt to changes in the background, and can be used in overexposure, overdarkness, and edge backgrounds. The influence of the grayscale difference between internal bright and dark areas and the area of bright and dark areas is also considered. Compared with the depth information determined by the conventional grayscale contrast algorithm, the depth information of the defect determined by the weighted grayscale contrast is more accurate, and the error with the actual depth information is smaller.
[0092] In some embodiments, before obtaining the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area, the method further includes:
[0093] Obtaining depth information of defects in a plurality of defective products measured;
[0094] Collect sample defect images corresponding to multiple defective products respectively;
[0095] Determine the weighted grayscale contrast of the defect area in each defect sample image;
[0096] Performing linear fitting on the weighted grayscale contrast and depth information of the defective area in the defective sample image of each defective product to obtain a linear fitting function;
[0097] According to the weighted grayscale contrast of the original defect area, the depth information of the defect corresponding to the original defect area is obtained, including:
[0098] The weighted grayscale contrast of the original defect area is substituted into the linear fitting function to obtain the depth information of the defect corresponding to the original defect area.
[0099] The present application is an embodiment in which the depth information of defects in defective products can be measured manually or by machine, and there is no limitation on this.
[0100] The defect sample images include defects of defective products. The defects in each defect sample image can be of any defect level. For example, the defect level can be divided into a serious level (NG), a less serious level (MNG) and a not serious level (OK) from high to low. During the sampling stage, multiple defect sample images can be collected for each defect level.
[0101] See also Figure 4 , Figure 4 This is a schematic diagram of multiple defect sample images collected for each defect level in an embodiment of the present application, including three rows of defect sample images, each row of defect sample images includes 9 defect sample images, wherein the defect level of the defect sample images in the first row is a serious level (NG), the defect level of the defect sample images in the second row is a sub-serious level (MNG), and the defect level of the defect sample images in the third row is a non-serious level (OK). It can be found from the figure that the higher the depth of the defect, the higher the severity of the defect.
[0102] After collecting sample defect images, Figure 1 The embodiment of step 110 to step 140 in the embodiment determines the weighted grayscale contrast of the defect area in each defect sample image.
[0103] See Table 1, which records Figure 4 The weighted grayscale contrast of defects in each row of defect sample images in ,
[0104]
[0105] Table 1
[0106] By comparison Figure 4 It can be found from Table 1 that the greater the weighted grayscale contrast, the deeper the defect, and the two show a positive correlation.
[0107] In some embodiments, determining a defect edge region from an original defect region in an image includes:
[0108] Determine the number of indented pixels according to the size of the original defect area;
[0109] According to the number of shrunk pixels, the original defect area is shrunk to obtain the shrunk defect area;
[0110] The original defect area and the indented defect area are subjected to difference processing to obtain the defect edge area.
[0111] The embodiment of the present application can establish a corresponding relationship between the original size and the number of shrunk pixels. The number of shrunk pixels refers to the number of pixels reduced in each direction after the image size is reduced. If the size of the original defect area is large, the number of shrunk pixels is slightly larger, such as 10-15; if the size of the original defect area is large, the number of shrunk pixels is slightly smaller, such as 3-5. In general, the number of shrunk pixels can be set to a fixed value, such as 5.
[0112] The original defect area can be cropped out from the image, and the original defect area can be shrunk according to the number of shrunk pixels to obtain the shrunk defect area, and then the difference area between the original defect area and the shrunk defect area is determined, and the difference area is the defect edge area.
[0113] like Figure 5 As shown, the embodiment of the present application provides a schematic diagram of the original defect area, the defect area after shrinkage, and the defect edge area. Figure 5 It can be found that, considering the occupied area, the original defect area = the defect area after shrinkage + the defect edge area.
[0114] The embodiment of the present application determines the defect edge area of the original defect area by performing an indentation process on the original defect area, so that the detected defect edge area has stronger anti-interference capability.
[0115] In some embodiments, obtaining a weighted grayscale contrast of an original defective area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of a defective bright area, and the area of a defective dark area includes:
[0116] According to the first influence weight and the first average grayscale, a first grayscale contrast is obtained; the first grayscale contrast is a compensation grayscale contrast of the background to the original defect area;
[0117] According to the second average grayscale, the third average grayscale, the area of the defect bright region and the area of the defect dark region, a second grayscale contrast is obtained; the second grayscale contrast is the grayscale contrast inside the original defect region;
[0118] The sum of the first grayscale contrast and the second grayscale contrast is used as the weighted grayscale contrast of the original defect area.
[0119] Specifically, the first grayscale contrast is the product of the first influence weight and the target absolute value, and the target absolute value is the absolute value of the difference between the target grayscale and the first average grayscale of the defect edge area, that is: the first grayscale contrast = first influence weight * |target grayscale - first average grayscale|, where "||" represents the absolute value.
[0120] In some embodiments, obtaining a second grayscale contrast ratio according to the second average grayscale, the third average grayscale, the area of the defect bright region, and the area of the defect dark region includes:
[0121] Obtaining a grayscale difference between a bright defect area and a dark defect area according to the second average grayscale and the third average grayscale;
[0122] According to the area of the defect bright area and the area of the defect dark area, second influence weights of the areas of the defect bright area and the defect dark area are obtained;
[0123] The product of the grayscale difference between the bright and dark areas and the second influence weight is taken as the second grayscale contrast.
[0124] Specifically, the grayscale difference between the bright and dark areas is the grayscale difference between the second average grayscale of the defective bright area and the third average grayscale of the defective dark area. The second influence weight is the area ratio of the bright and dark areas, that is, the ratio of the area of the defective bright area to the area of the defective dark area. From this, it can be determined that: the second grayscale contrast = (second average grayscale - third average grayscale) * second influence weight.
[0125] In summary, it can be determined that weighted grayscale contrast = first weight coefficient * | target grayscale - first average grayscale | + second average grayscale - third average grayscale) * second influence weight;
[0126] The first weight coefficient = target grayscale / first average grayscale;
[0127] The second weight coefficient=the area of the bright defect region / the area of the dark defect region.
[0128] From the above, it can be found that when calculating the weighted grayscale contrast, the embodiment of the present application takes into account the influence of the background around the defect and the grayscale difference between light and dark inside the defect. The weighted grayscale contrast reduces the interference of the background around the original defect area, can adapt to changes in the background, and can be used in overexposure, overdarkness, and edge backgrounds. In actual application scenarios, it can be applied to appearance detection, size detection, etc. In algorithm application scenarios, it can be applied to deep learning algorithms, traditional detection algorithms, etc.
[0129] In some embodiments, determining a defect bright area and a defect dark area from an original defect area in an image includes:
[0130] According to the average grayscale of the original defect area, the original defect area is divided into a defect bright area and a defect dark area;
[0131] The grayscale of each pixel included in the defect dark area is less than the average grayscale of the original defect area;
[0132] The grayscale of each pixel included in the defect bright area is smaller than the average grayscale of the original defect area.
[0133] The above-mentioned embodiments have explained that the average grayscale of the original defect area can be calculated based on the grayscale of the pixels contained in the original defect area. The pixels in the original defect area whose grayscale is less than the average grayscale of the original defect area can be divided into the defect bright area, and the pixels in the original defect area whose grayscale is greater than the average grayscale of the original defect area can be divided into the defect bright area. No further details will be given here.
[0134] See also Figure 6 , the present application embodiment provides a second flow chart of a method for determining a defect depth, including:
[0135] Input the original defect area; perform indentation processing on the original defect area to obtain the indented defect area; perform difference processing on the original defect area and the indented defect area to obtain the defect edge area; remove a part of pixels with smaller grayscale and larger grayscale in the defect edge area, take the average value of the grayscale values of the remaining pixels as the first average grayscale of the defect edge area, take the ratio of the target grayscale and the first average grayscale as the first influence weight of the background of the image where the original defect area is located, and obtain the first grayscale contrast according to the first influence weight and the first average grayscale; the first grayscale contrast is the compensatory grayscale contrast of the background to the original defect area.
[0136] Gaussian denoising is performed on the original defect area. According to the grayscale of the pixels contained in the original defect area, the average grayscale of the defect area is obtained. According to the average grayscale of the defect area, the original defect area is divided into a defect bright area and a defect dark area; according to the second average grayscale of the defect bright area and the third average grayscale of the defect dark area, the grayscale difference between the bright and dark areas of the defect bright area and the defect dark area is obtained; according to the area of the defect bright area and the area of the defect dark area, the second influence weight of the area of the defect bright area and the area of the defect dark area is obtained; the product of the grayscale difference between the bright and dark areas and the second influence weight is taken as the second grayscale contrast; the second grayscale contrast is the grayscale contrast inside the original defect area
[0137] The sum of the first grayscale contrast and the second grayscale contrast is used as the weighted grayscale contrast of the original defect area;
[0138] The weighted grayscale contrast of the original defect area is substituted into the linear fitting function to obtain the depth information of the defect corresponding to the original defect area.
[0139] The method for determining the defect depth provided in the embodiment of the present application may be executed by a device for determining the defect depth. In the embodiment of the present application, the device for determining the defect depth provided in the embodiment of the present application is described by taking the method for determining the defect depth performed by the device for determining the defect depth as an example.
[0140] The embodiment of the present application also provides a device for determining defect depth.
[0141] like Figure 7 As shown, the device for determining the defect depth includes: a first processing module 710 , a second processing module 720 , a third processing module 730 , a fourth processing module 740 and a fifth processing module 750 .
[0142] A first processing module 710 is used to determine a defect edge region, a defect bright region, and a defect dark region from an original defect region of the image;
[0143] The second processing module 720 is used to determine a first average grayscale of the defect edge area, a second average grayscale of the defect bright area, and a third average grayscale of the defect dark area;
[0144] The third processing module 730 is used to obtain a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale;
[0145] The fourth processing module 740 is used to obtain a weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area;
[0146] The fifth processing module 750 is used to obtain the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area.
[0147] According to the device for determining the depth of a defect provided by an embodiment of the present application, the first influence weight of the background of the image where the original defect area is located is determined by the first average grayscale of the defect edge area and the target grayscale. According to the first influence weight, the second average grayscale of the defect bright area, the third average grayscale of the defect dark area, the area of the defect bright area, and the area of the defect dark area, the weighted grayscale contrast of the original defect area is obtained. The weighted grayscale contrast reduces the interference of the background around the defect area, can adapt to changes in the background, can be used in overexposure, overdarkness, and edge backgrounds, and also considers the influence of the grayscale difference of the internal bright and dark areas and the area of the bright and dark areas. Compared with the depth information determined by the conventional grayscale contrast algorithm, the depth information of the defect determined by the weighted grayscale contrast is more accurate and has a smaller error with the actual depth information. In some embodiments, the first processing module 710 is used to:
[0148] Determine the number of indented pixels according to the size of the original defect area;
[0149] According to the number of shrunk pixels, the original defect area is shrunk to obtain the shrunk defect area;
[0150] The original defect area and the indented defect area are subjected to difference processing to obtain the defect edge area.
[0151] In some embodiments, the fourth processing module 740 is used to:
[0152] According to the first influence weight and the first average grayscale, a first grayscale contrast is obtained; the first grayscale contrast is a compensation grayscale contrast of the background to the original defect area;
[0153] According to the second average grayscale, the third average grayscale, the area of the defect bright region and the area of the defect dark region, a second grayscale contrast is obtained; the second grayscale contrast is the grayscale contrast inside the original defect region;
[0154] The sum of the first grayscale contrast and the second grayscale contrast is used as the weighted grayscale contrast of the original defect area.
[0155] In some embodiments, the fourth processing module 740 is further configured to:
[0156] Obtaining a grayscale difference between a bright defect area and a dark defect area according to the second average grayscale and the third average grayscale;
[0157] According to the area of the defect bright area and the area of the defect dark area, second influence weights of the areas of the defect bright area and the defect dark area are obtained;
[0158] The product of the grayscale difference between the bright and dark areas and the second influence weight is taken as the second grayscale contrast.
[0159] In some embodiments, the first processing module 710 is further configured to:
[0160] According to the average grayscale of the original defect area, the original defect area is divided into a defect bright area and a defect dark area;
[0161] The grayscale of each pixel included in the defect dark area is less than the average grayscale of the original defect area;
[0162] The grayscale of each pixel included in the defect bright area is smaller than the average grayscale of the original defect area.
[0163] In some embodiments, the third processing module 730 is specifically used to:
[0164] The ratio of the target grayscale to the first average grayscale is used as the first influence weight of the background of the image where the original defective area is located.
[0165] In some embodiments, the apparatus for determining defect depth further includes a sixth processing module;
[0166] The sixth processing module is used for:
[0167] Obtaining depth information of defects in a plurality of defective products measured;
[0168] Collect sample defect images corresponding to multiple defective products respectively;
[0169] Determine the weighted grayscale contrast of the defect area in each defect sample image;
[0170] Performing linear fitting on the weighted grayscale contrast and depth information of the defective area in the defective sample image of each defective product to obtain a linear fitting function;
[0171] The fifth processing module 750 is specifically used for:
[0172] The weighted grayscale contrast of the original defect area is substituted into the linear fitting function to obtain the depth information of the defect corresponding to the original defect area.
[0173] The device for determining the defect depth in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0174] The device for determining the defect depth in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0175] The device for determining the defect depth provided in the embodiment of the present application can achieve Figures 1 to 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0176] In some embodiments, Figure 8As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, each process of the above-mentioned defect depth determination method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0177] The processor 801 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
[0178] The memory 802 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.
[0179] The memory 802 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer program stored in the memory 802 to implement the steps shown in the above method embodiment.
[0180] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0181] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned defect depth determination method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0182] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0183] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned method for determining the defect depth when executed by a processor.
[0184] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0185] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned defect depth determination method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0186] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0187] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0188] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0189] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0190] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0191] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A method for determining defect depth, characterized in that: include: Determine a defect edge region, a defect bright region, and a defect dark region from the original defect region of the image; Determine a first average grayscale of the defect edge area, a second average grayscale of the defect bright area, and a third average grayscale of the defect dark area; Obtaining a first influence weight of a background of the image where the original defective area is located according to the first average grayscale and the target grayscale; Obtaining a weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area; According to the weighted grayscale contrast of the original defect area, the depth information of the defect corresponding to the original defect area is obtained.
2. The method for determining defect depth according to claim 1, characterized in that: The step of determining the defect edge region from the original defect region of the image comprises: Determining the number of shrunk pixels according to the size of the original defect area; Performing a shrinking process on the original defective area according to the number of shrinking pixels to obtain a shrunken defective area; The original defect area and the indented defect area are subjected to difference processing to obtain the defect edge area.
3. The method for determining defect depth according to claim 1, characterized in that: The step of obtaining the weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area includes: According to the first influence weight and the first average grayscale, a first grayscale contrast is obtained; the first grayscale contrast is a compensation grayscale contrast of the background to the original defect area; According to the second average grayscale, the third average grayscale, the area of the defect bright region and the area of the defect dark region, a second grayscale contrast is obtained; the second grayscale contrast is the grayscale contrast inside the original defect region; The sum of the first grayscale contrast and the second grayscale contrast is used as the weighted grayscale contrast of the original defect area.
4. The method for determining defect depth according to claim 3, characterized in that: The step of obtaining a second grayscale contrast ratio according to the second average grayscale, the third average grayscale, the area of the defect bright region, and the area of the defect dark region comprises: Obtaining a grayscale difference between the bright defect area and the dark defect area according to the second average grayscale and the third average grayscale; According to the area of the defect bright area and the area of the defect dark area, obtaining second influence weights of the areas of the defect bright area and the defect dark area; The product of the bright and dark area grayscale difference and the second influence weight is used as the second grayscale contrast.
5. The method for determining defect depth according to claim 1, characterized in that: Determine the defect bright area and the defect dark area from the original defect area of the image, including: According to the average grayscale of the original defect area, the original defect area is divided into a defect bright area and a defect dark area; Wherein, the grayscale of each pixel included in the defective dark area is less than the average grayscale of the original defective area; The grayscale of each pixel included in the defect bright area is smaller than the average grayscale of the original defect area.
6. The method for determining defect depth according to claim 1, characterized in that: Obtaining a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale includes: The ratio of the target grayscale to the first average grayscale is used as the first influence weight of the background of the image where the original defective area is located.
7. The method for determining defect depth according to any one of claims 1 to 6, characterized in that: Before obtaining the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area, the method further includes: Obtaining depth information of defects in a plurality of defective products measured; Collect sample defect images corresponding to multiple defective products respectively; Determine the weighted grayscale contrast of the defect area in each defect sample image; Performing linear fitting on the weighted grayscale contrast and depth information of the defective area in the defective sample image of each defective product to obtain a linear fitting function; The step of obtaining depth information of a defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area includes: The weighted grayscale contrast of the original defect area is substituted into the linear fitting function to obtain the depth information of the defect corresponding to the original defect area.
8. A device for determining defect depth, characterized in that: include: A first processing module is used to determine a defect edge area, a defect bright area and a defect dark area from an original defect area of the image; A second processing module, used to determine a first average grayscale of the defect edge area, a second average grayscale of the defect bright area, and a third average grayscale of the defect dark area; A third processing module, configured to obtain a first influence weight of the background of the image where the original defective area is located according to the first average grayscale and the target grayscale; a fourth processing module, configured to obtain a weighted grayscale contrast of the original defect area according to the first influence weight, the first average grayscale, the second average grayscale, the third average grayscale, the area of the defect bright area, and the area of the defect dark area; The fifth processing module is used to obtain the depth information of the defect corresponding to the original defect area according to the weighted grayscale contrast of the original defect area.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for determining the defect depth according to any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the defect depth according to any one of claims 1 to 7 is implemented.