A video image defect detection and identification processing method and system

Abstract

This invention discloses a video image defect detection and recognition processing method and system. The method includes the following steps: acquiring an image of a workpiece to be inspected from industrial video network data; converting the image of the workpiece to be inspected to grayscale to obtain a grayscale image of the workpiece to be inspected; calculating the grayscale values ​​of the pixels in the grayscale image of the workpiece to be inspected; distinguishing suspected pixels based on the grayscale values ​​of the pixels; clustering the suspected pixels to obtain suspected regions; calculating the defect probability based on the suspected regions; and determining whether the image of the workpiece to be inspected has a defect based on the defect probability.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method and system for detecting and recognizing defects in video images. Background Technology

[0002] Product quality inspection is a crucial step in industrial production. With the development of industrial automation and intelligence, machine vision-based automatic defect detection technology has been widely applied. However, some challenges remain in practical applications.

[0003] Video images acquired in industrial environments often contain interference such as shadows and light spots, affecting detection accuracy; different types of defects exhibit different characteristics in images, making them difficult to detect using a single method; simple thresholding methods are prone to false detections or missed detections, requiring more complex algorithms to improve detection accuracy; and production line inspections require rapid response, making algorithm efficiency crucial.

[0004] To address these issues, researchers have proposed several improved methods. Image preprocessing improves image quality through operations such as grayscale conversion and filtering. Then, grayscale statistical features and texture features are calculated to distinguish between defects and normal areas. The study found that predefined features (such as grayscale statistics and texture features) may not be able to fully capture the features of all types of defects. Defects of different sizes may require different processing strategies. Traditional methods have difficulty handling multi-scale shape problems simultaneously, and it is difficult to quickly detect the shapes of shadows and light spots generated in high-resolution images. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes an intelligent defect detection and recognition method and system based on video images. This method can detect defects in shadows and light spots of different shapes, achieving rapid detection in video images.

[0006] The purpose of this invention is to provide a video image defect detection and recognition processing method and system, which solves the above-mentioned technical problems pointed out in the prior art.

[0007] This invention provides a method for detecting and identifying defects in video images, comprising the following steps:

[0008] The process involves acquiring an image of the workpiece to be inspected from industrial video network data, converting the image to grayscale to obtain a grayscale image of the workpiece, and calculating the grayscale value of each pixel in the grayscale image of the workpiece.

[0009] Suspected pixels are distinguished based on their grayscale values, and these suspected pixels are clustered to obtain suspected regions; the probability of defects is then calculated based on these suspected regions.

[0010] The presence of defects in the image of the workpiece to be inspected is determined based on the defect probability.

[0011] Preferably, suspected pixels are distinguished based on their grayscale values, and these suspected pixels are clustered to obtain suspected regions. The defect probability is then calculated based on these suspected regions. The specific steps are as follows:

[0012] Based on the gray values ​​of the pixels in the grayscale image of the workpiece to be detected, they are sorted in order from low to high gray values; a preset pixel distinction threshold q is used to determine the gray value boundary of the pixels, thereby obtaining the first set of pixels and the second set of pixels.

[0013] Determine the difference between the number of pixels in the first pixel set and the number of pixels in the second pixel set;

[0014] If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, then the first pixel set is determined to be a suspected pixel set, and the second pixel set is determined to be a normal pixel set.

[0015] If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, then the second pixel set is determined to be a suspected pixel set, and the first pixel set is determined to be a normal pixel set.

[0016] The normal pixels are clustered using the mean offset algorithm to obtain multiple normal pixel regions; the suspected pixels are clustered using the mean offset algorithm to obtain multiple suspected regions.

[0017] The suspected region is optimized by deleting isolated suspected pixels at the edges; the local grayscale standard deviation of the optimized suspected region is calculated; the area of ​​the suspected region is calculated based on the suspected pixels in the suspected region, and different suspected region areas are calculated based on suspected regions of different shapes;

[0018] The defect probability is calculated by combining the local grayscale standard deviation and the area of ​​the suspected region using a weighted method.

[0019] The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated, and then the workpiece images to be detected in the previous and next frames are collected. The defect probability is further verified by the workpiece images to be detected in the previous and next frames to see if it reaches the final output defect probability.

[0020] Preferably, the formula for the defect probability is:

[0021]

[0022] s k The probability of a defect in the k-th suspected region;

[0023] H k Let be the weighted local grayscale standard deviation of the k-th suspected region;

[0024] A k ' is the area of ​​the k-th suspected region;

[0025] e is a natural constant.

[0026] Preferably, the specific steps for calculating the local grayscale standard deviation of the optimized post-suspected region are as follows:

[0027] Within the optimized suspected region, a local window is preset with a size of w*w; each suspected pixel within the suspected region is traversed, and the standard deviation of the gray values ​​of all suspected pixels within the local window is calculated with each suspected pixel as the center; the standard deviations of all local windows are weighted to obtain the local gray standard deviation of the suspected region.

[0028] The local grayscale standard deviation weighting formula is as follows:

[0029]

[0030] In the formula, H k The weighted local grayscale standard deviation of the k-th suspected region;

[0031] m k Let be the number of suspected pixels in the k-th suspected region;

[0032] Let be the local grayscale standard deviation of the i-th suspected pixel within the k-th suspected region;

[0033] w is the weighting coefficient; e is the natural constant.

[0034] Preferably, the area of ​​the suspected region is calculated based on the suspected pixels of the suspected region, and different suspected region areas are calculated based on suspected regions of different shapes. The specific operation steps are as follows:

[0035] Connect the suspected pixels at the edges of the suspected region to obtain the outline of the suspected region; use the closing operation to smooth the outline of the suspected region, and fill the outline of the suspected region with small holes to obtain a new outline of the suspected region.

[0036] Green's theorem is used to transform the new contour of the suspected region into a closed curve on a plane. The suspected region is then enclosed by the closed curve of the suspected region, and the basic area of ​​the suspected region is obtained.

[0037] Preferably, the basic area expression for the suspected region is:

[0038]

[0039] In the formula, C is a closed curve on the plane of the suspected region; ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol;

[0040] D is the suspected area surrounded by C;

[0041] Pdx and Qdy are represented as the integrands of the line integral; P and Q are bivariate functions on the suspected region of D, and the values ​​of P and Q vary through the two variables x and y.

[0042] x and y represent the coordinates of a point on the closed curve C;

[0043] ∫∫_D denotes the double integral over the suspected region D; It is represented as the integrand of a double integral; dxdy represents the area elements in the x and y directions;

[0044] When P = -y and Q = x

[0045] Enclosing the surface using line integrals, the calculation formula is as follows:

[0046] Basic area = ∮_C(xdy-ydx);

[0047] In the formula, ∮_C represents the line integral along the closed curve C;

[0048] x and y represent the coordinates of a point on the closed curve C;

[0049] dx and dy represent small changes in x and y, respectively;

[0050] (xdy-ydx) represents the area swept along the closed curve C by a randomly selected suspected pixel.

[0051] Preferably, the area of ​​the first target to be determined calculated by Green's theorem is subjected to a second verification process. If the verification result is correct, the area of ​​the first target to be determined is determined to be the area of ​​the second target. Specifically, this includes:

[0052] The suspected region is decomposed into multiple triangles using a triangulation algorithm;

[0053] Then calculate the area of ​​each triangle, and finally add up the areas of all the triangles to output the area calculation result of the suspected region.

[0054] The area calculation result of the suspected area is compared with the area of ​​the first target to be determined calculated by Green's theorem. If the ratio of the difference between the two to the value of the first target area is less than the standard error ratio threshold G%, the verification result is determined to be correct, and the current area of ​​the first target to be determined is determined to be the area of ​​the second target.

[0055] Accordingly, the present invention also provides a video image defect detection and recognition processing system, comprising: a module to be detected; a calculation module; and a confirmation module;

[0056] The detection module is used to acquire the image of the workpiece to be detected from industrial video network data, convert the image of the workpiece to be detected to grayscale to obtain a grayscale image of the workpiece to be detected, and calculate the grayscale value of the pixels in the grayscale image of the workpiece to be detected.

[0057] The calculation module is used to distinguish suspected pixels based on their grayscale values, cluster the suspected pixels to obtain suspected regions, and calculate the defect probability based on the suspected regions.

[0058] The confirmation module is used to determine whether the image of the workpiece to be inspected has defects based on the defect probability.

[0059] Accordingly, the present invention also provides a storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the steps of the video image defect detection and recognition processing method described in any of the above claims.

[0060] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0061] Analysis of the video image defect detection and recognition processing method provided by the present invention shows that, in specific applications, the image of the workpiece to be inspected is acquired from industrial video network data. The grayscale value of each pixel in the image is calculated by converting it to grayscale. The grayscale value of each pixel can be used to determine the shadow and spot areas of the image of the workpiece to be inspected. The pixels are sorted by grayscale value, and a pixel distinction threshold is set to distinguish the sorted pixels by boundary, resulting in a first set of pixels and a second set of pixels. The number of pixels within the first set of pixels and the second set of pixels is determined, and the set with more pixels is identified as a suspected pixel. In this way, based on the characteristic that the density of pixels in the defect area is higher than that in other areas, the shadow and spot areas appearing on the image of the workpiece to be inspected during the shooting process of the industrial video network data can be effectively identified, and normal pixels and suspected pixels that may have defects can be quickly identified.

[0062] Furthermore, suspected pixels are clustered to obtain suspected regions. Based on the suspected regions, it can be determined that there may be defects in the region. The local grayscale standard deviation and the area of ​​the suspected regions are calculated. The defect probability is obtained through the local grayscale standard deviation and the area of ​​the suspected regions. The presence of defects in the workpiece image is determined based on the level of the defect probability.

[0063] Furthermore, the contour of the suspected region is calculated using suspected pixels at the edges. A closing operation is then performed on the contour of the suspected region: first, dilation and filling, then erosion to restore the contour, resulting in a new contour without gaps. Green's theorem is then used to convert the new contour into a closed curve that encloses the suspected region. Based on selected suspected pixels on the closed curve, a circumferential scan is performed on the suspected region to obtain its basic area. This basic area is then converted to actual unit area (i.e., suspected region area) based on the resolution of the workpiece image. This allows for further confirmation of the basic area of ​​the suspected region, thereby improving its accuracy. Moreover, the area can be calculated for suspected regions of different shapes, solving the problem of detecting defects such as shadows and light spots in the workpiece image. Attached Figure Description

[0064] Figure 1 This is a flowchart of a video image defect detection and recognition processing method in Embodiment 1;

[0065] Figure 2 This is a flowchart illustrating the calculation of defect probability in a video image defect detection and recognition processing method according to Embodiment 1.

[0066] Figure 3 This is a schematic diagram before the clustering of normal pixels and suspected pixels in a video image defect detection and identification processing method in Embodiment 1.

[0067] Figure 4 This is a schematic diagram showing the clustering of normal pixels and suspected pixels in a video image defect detection and recognition processing method according to Embodiment 1.

[0068] Figure 5 This is a flowchart illustrating the calculation of the suspected region area in a video image defect detection and identification processing method according to Example 1.

[0069] Figure 6 This is a flowchart illustrating the verification of the suspected region area in a video image defect detection and identification processing method according to Example 1.

[0070] Figure 7This is a schematic diagram of a triangle constructed from the irregular edges of a suspected region in a video image defect detection and recognition processing method according to Embodiment 1.

[0071] Figure 8 This is a flowchart of a video image defect detection and recognition processing system according to Embodiment 2;

[0072] Figure 9 This is a schematic diagram of the structure of a storage medium for applying the above-mentioned video image defect detection and recognition processing method.

[0073] Labels: Module to be tested 10; Calculation module 20; Confirmation module 30; Memory 1130; Communication interface 1120; Processor 1110; Computer storage medium 1140. Detailed Implementation

[0074] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0076] Example 1

[0077] See Figure 1 Embodiment 1 of the present invention provides a video image defect detection and recognition processing method, including the following operation steps:

[0078] S10: Acquire the image of the workpiece to be inspected from industrial video network data, convert the image of the workpiece to be inspected to grayscale to obtain a grayscale image of the workpiece to be inspected; calculate the grayscale value of the pixels in the grayscale image of the workpiece to be inspected.

[0079] S20: Differentiate suspected pixels based on their grayscale values, cluster the suspected pixels to obtain suspected regions, and calculate the defect probability based on the suspected regions;

[0080] It should be noted that when acquiring images of the workpiece to be inspected from industrial video network data, the images are inspected to determine if defects exist. However, due to the dim lighting environment in industrial workshops, only clearly visible areas in the workpiece image can be quickly detected. Meanwhile, shadows and light spots in the images captured by the recording equipment can severely impact defect detection. Therefore, the grayscale values ​​of pixels are used to distinguish suspected areas of shadows and light spots, and the probability of defects in these areas is calculated. This probability is then used to determine if defects exist in the workpiece image. Furthermore, suspected pixels can be used to represent potential defects in shadow and light spot areas, allowing for the determination of whether defects exist in the workpiece image.

[0081] S30: Determine whether the image of the workpiece to be inspected has a defect based on the defect probability.

[0082] It should be noted that when the defect probability is obtained, a probability threshold is set, and the presence of defects in the workpiece image to be inspected is determined by comparing the defect probability with the probability threshold. When the defect probability is greater than or equal to the probability threshold, the workpiece image to be inspected has a defect problem; when the defect probability is less than the probability threshold, the workpiece image to be inspected has no defect problem.

[0083] Specifically, see Figure 2 In step S20, suspected pixels are distinguished based on their grayscale values, and these suspected pixels are clustered to obtain suspected regions. The defect probability is then calculated based on these suspected regions. The specific steps are as follows:

[0084] S21: Sort the pixels of the grayscale image of the workpiece to be detected in order from low to high according to the grayscale value; set a preset pixel distinction threshold q, and determine the grayscale value boundary of the pixels according to the preset pixel distinction threshold q to obtain the first pixel set and the second pixel set (that is, the pixel set refers to all pixels in the whole set, so the pixel set is a set).

[0085] S22: Determine the difference between the number of pixels in the first pixel set and the number of pixels in the second pixel set;

[0086] If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, then the first pixel set is determined to be a suspected pixel set, and the second pixel set is determined to be a normal pixel set.

[0087] If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, then the second pixel set is determined to be a suspected pixel set, and the first pixel set is determined to be a normal pixel set.

[0088] It should be noted that, for example: the gray values ​​of pixels are sorted from 1, 2, 3, ..., 100, and a pixel discrimination threshold q is set (assuming q = 47). The gray values ​​of pixels are then used as a discrimination boundary (i.e., the pixel discrimination threshold q (i.e., 47) is the discrimination boundary). The set of pixels with gray values ​​from 1 to 47 is the first set of pixels, and the set of pixels with gray values ​​from 48 to 100 is the second set of pixels. The set of pixels with more pixels is then determined, and the set with more pixels is considered a suspected set of pixels, while the set with fewer pixels is considered a normal set of pixels.

[0089] Because during industrial operations, industrial video network data can cause shadows and light spots in the image of the workpiece to be inspected during the shooting process, a pixel distinction threshold q (assuming q = 47) is first needed to distinguish the gray values ​​of the pixels. Secondly, the pixel density in the defect area is higher than that in other areas, so the first set of pixels and the second set of pixels are compared. The set with more pixels is considered as suspected pixels, and the set with fewer pixels is considered as normal pixels. This can better distinguish the areas of the image of the workpiece to be inspected, greatly shorten the inspection time, and directly detect the key areas of the image of the workpiece to be inspected.

[0090] S23: Cluster the normal pixels using the mean offset algorithm to obtain multiple normal pixel regions; cluster the suspected pixels using the mean offset algorithm to obtain multiple suspected regions;

[0091] It should be noted that the mean-shift algorithm searches for the density center of suspected pixels in the grayscale image of the workpiece to be inspected. Using a sliding window method, it continuously moves the density center towards the direction of the highest suspected pixel density. When multiple density-concentrated regions exist in the grayscale image of the workpiece to be inspected, multiple clustering results (i.e., multiple suspected regions) will be obtained. Since the pixel density in defective regions is inherently higher than in other regions (i.e., regions with normal pixels), when the image of the workpiece to be inspected is an irregularly curved workpiece, before clustering normal pixels and suspected pixels, please refer to [the relevant documentation / reference needed]. Figure 3 Therefore, each clustering result of mean shift is a density concentration region (i.e., a suspected region). When the image of the workpiece to be detected is an irregular curved workpiece, after clustering normal pixels and suspected pixels, see [the relevant documentation]. Figure 4 This allows for a good distinction between areas in the grayscale image of the workpiece to be inspected that are likely to contain suspected defect areas and areas that are likely to contain normal areas.

[0092] Therefore, based on the clustering results of mean shift, the suspected pixels in the grayscale image of the workpiece to be inspected can be clustered into multiple suspected regions, and each suspected region may contain suspected defect areas.

[0093] S24: Optimize the suspected region by deleting isolated suspected pixels at the edges; calculate the local grayscale standard deviation of the optimized suspected region; calculate the area of ​​the suspected region based on the suspected pixels in the suspected region, and calculate different suspected region areas based on suspected regions of different shapes;

[0094] The defect probability is calculated by combining the local grayscale standard deviation and the area of ​​the suspected region using a weighted method.

[0095] The formula for the defect probability is:

[0096]

[0097] s k The probability of a defect in the k-th suspected region;

[0098] H k Let be the weighted local grayscale standard deviation of the k-th suspected region;

[0099] A k Let the area of ​​the k-th suspected region be [the area of ​​the suspected region].

[0100] e is the natural constant;

[0101] The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated, and the workpiece images to be detected in the previous and next frames are collected. The defect probability is further verified by the workpiece images to be detected in the previous and next frames to see if it reaches the final output defect probability.

[0102] It should be noted that optimizing the suspected region can remove noise at the boundary of the suspected region and eliminate some isolated suspected pixels at the edge of the suspected region, avoiding deviations in the calculation of the local grayscale standard deviation and the area of ​​the suspected region. Calculating the local grayscale standard deviation of the suspected region can reveal the degree of variation in the grayscale values ​​of pixels within the suspected region and also reflects the uniformity of the grayscale distribution of pixels in the suspected region. If the image of the workpiece to be inspected may contain defective areas, the local grayscale standard deviation is often large because defects usually cause significant changes in grayscale values, such as shadows and light spots.

[0103] Specifically, the calculation of the local grayscale standard deviation of the optimized suspected region involves the following steps:

[0104] Within the optimized suspected region, a local window is preset, the size of which is w*w (i.e., 3*3 or 4*4); each suspected pixel within the suspected region is traversed, and the standard deviation of the gray values ​​of all suspected pixels within the local window is calculated with each suspected pixel as the center; the standard deviations of all local windows are weighted to obtain the local gray standard deviation of the suspected region.

[0105] The local grayscale standard deviation weighting formula is as follows:

[0106]

[0107] In the formula, H k The weighted local grayscale standard deviation of the k-th suspected region;

[0108] m k Let be the number of suspected pixels in the k-th suspected region;

[0109] Let be the local grayscale standard deviation of the i-th suspected pixel within the k-th suspected region;

[0110] w is the weighting coefficient; e is the natural constant.

[0111] Specifically, see Figure 5 In step S24, the area of ​​the suspected region is calculated based on the suspected pixels of the suspected region, and different suspected region areas are calculated based on different shapes of suspected regions. The specific operation steps are as follows:

[0112] S241: Connect the suspected edge pixels of the suspected region (i.e., suspected edge pixels refer to suspected pixels at the edge of the suspected region) to obtain the outline of the suspected region; use the closing operation to smooth the outline of the suspected region, fill the outline of the suspected region with small holes, and obtain a new outline of the suspected region.

[0113] It should be noted that the outline of the suspected region is obtained, and a closing operation (expansion) is performed on the outline of the suspected region. This expansion operation can transform the irregular shape of the suspected region into a regular shape. Furthermore, after the expansion operation, the area between the suspected pixels of the suspected region is enlarged. By filling the small holes or gaps between the suspected pixels, the area between the suspected pixels is smoothed. Finally, an erosion operation is used to restore the outline of the expanded suspected region, forming a new outline of the suspected region (i.e., the previously filled holes or gaps are not restored after the erosion operation). This ensures that there are no gaps in the outline of the suspected region, improving the accuracy of the area calculation for the suspected region in the subsequent step S242 of forming a closed curve around the suspected region, and also improving the efficiency of the subsequent scanning of the suspected region's area around the closed curve.

[0114] S242: Using Green's theorem, the new contour of the suspected region is transformed into a closed curve on a plane. The suspected region is then enclosed by the closed curve of the suspected region to obtain the basic area of ​​the suspected region.

[0115] The basic area expression for the suspected region is:

[0116]

[0117] In the formula, C is a closed curve on the plane of the suspected region (i.e., the new contour of the suspected region, because the new contour of the suspected region is transformed into a closed curve of the line integral of the plane by Green's theorem, so the closed curve can represent the new contour of the suspected region); ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol.

[0118] D is the suspected region enclosed by the closed curve C;

[0119] x and y represent the coordinates of a point on the closed curve C; dx and dy represent the small changes in x and y, respectively; Pdx and Qdy (i.e., Pdx + Qdy) represent the integrands of the line integral (i.e., P in Pdx is a bivariate function, usually a function of x and y, i.e., P(x, y), and dx represents the small change or increment of x; Q in Qdy is a bivariate function, usually a function of x and y, i.e., Q(x, y), and dy represents the small change or increment of y); P and Q are bivariate functions on the suspected region of D, and the values ​​of P and Q change through the two variables x and y;

[0120] x and y represent the coordinates of a suspected pixel on the closed curve C;

[0121] ∫∫_D represents the double integral (i.e., the surface integral) over the suspected region D; The integrand of a double integral (i.e. It is the partial derivative of Q with respect to x. (where is the partial derivative of P with respect to y); dxdy represents the area elements in the x and y directions;

[0122] In the above formula, Green's theorem is used to analyze the shape of the contour of the suspected region (i.e., closed curve) in the image of the workpiece to be detected using line integrals, which can calculate the contour of the suspected region with different shapes. The area integral is used to calculate the base plane of the suspected region, so that the closed curve is the same as the plane (i.e., the closed curve is the new contour of the suspected region, and the plane of the area integral is the suspected region, so the closed curve is equal to the plane in the formula), thereby calculating the base area of ​​the suspected region in the subsequent process. Green's theorem is used to represent the pixel intensity, gradient, and texture features in the image of the workpiece to be detected (i.e., calculating the area integral). In edge detection, it may represent the gradient of the image of the workpiece to be detected (i.e., calculating the line integral).

[0123] The basic area is calculated using area integrals, and the area enclosed is enclosed using line integrals and surface integrals. The calculation formula is as follows:

[0124] Basic area = ∮_C(xdy-ydx);

[0125] In the formula, ∮_C represents the line integral along the closed curve C;

[0126] x and y represent the coordinates of a point on the closed curve C (as explained in the aforementioned technical content);

[0127] dx and dy represent minute changes in x and y, respectively (as explained in the preceding technical content);

[0128] (xdy-ydx) represents the area swept along the closed curve C by a randomly selected suspected pixel.

[0129] It should be noted that when P = -y and Q = x,

[0130] In the above, It is the partial derivative of Q with respect to x. Since Q = x, therefore... It is the partial derivative of P with respect to y. Since P = -y, therefore... so, At this point, the area integral is used to calculate the basic area; a random suspected pixel is selected and moved along the closed curve C. During the movement, the suspected area is scanned around (that is, the area integral is enclosed by the surface integral using the line integral). Therefore, the line integral is calculated using ∮_C(xdy-ydx).

[0131] xdy represents the area increment formed by a tiny movement in the y direction (i.e., the area increment represents the tiny change in area caused by the tiny movement of the suspected pixel on the closed curve, and also corresponds to the tiny change and increment mentioned above); ydx represents the area increment formed by a tiny movement in the x direction (i.e., the area increment represents the tiny change in area caused by the tiny movement of the suspected pixel on the closed curve, and also corresponds to the tiny change and increment mentioned above).

[0132] Green's theorem described above allows for the rapid calculation of the area of ​​different suspected regions based on their shapes. It also improves the accuracy of area calculation for irregular suspected regions.

[0133] The base area of ​​the suspected region calculated using Green's theorem is taken as the first target area to be determined.

[0134] See Figure 6 Following S242, a secondary verification process is performed on the area of ​​the first target to be determined calculated using Green's theorem. If the verification result is correct, the current area of ​​the first target to be determined is determined to be the area of ​​the second target. The specific operation steps are as follows:

[0135] S2421: The suspected region is decomposed into multiple triangles using a triangulation algorithm;

[0136] S2422: Then calculate the area of ​​each triangle, and finally add up the areas of all triangles to output the area calculation result of the suspected region;

[0137] Triangulation: This method decomposes an irregular shape into multiple triangles. Triangulation algorithms (such as Delaunay triangulation) are used to decompose the shape into triangles. Triangulation is the process of dividing a polygon into several triangles; then the area of ​​each triangle is calculated, and finally the areas of all triangles are added together (i.e., the areas of all triangles are summed to obtain a rough estimate of the total area of ​​the irregular shape), such as... Figure 7 As shown.

[0138] S2423: Compare the calculated area of ​​the suspected area with the first target area to be determined calculated by Green's theorem. If the ratio of the difference between the two to the value of the first target area is less than the standard error ratio threshold G%, the verification result is determined to be correct, and the current first target area to be determined is determined to be the second target area. The above standard error ratio threshold G% is a preset value. It is explained that the triangulation method is a rough calculation method, so it cannot be used as the main calculation method for suspected areas. In this application embodiment, Green's theorem is used as the main calculation method, but the triangulation method is used for auxiliary verification. As long as the verification result is correct, the process continues.

[0139] Example 2

[0140] like Figure 8 As shown, the present invention also provides a video image defect detection and recognition processing system, including: a detection module 10; a calculation module 20; and a confirmation module 30;

[0141] The detection module 10 is used to acquire the image of the workpiece to be detected from industrial video network data, convert the image of the workpiece to be detected to grayscale to obtain a grayscale image of the workpiece to be detected, and calculate the grayscale value of the pixels in the grayscale image of the workpiece to be detected.

[0142] The calculation module 20 is used to distinguish suspected pixels based on the gray value of the pixels, cluster the suspected pixels to obtain suspected regions, and calculate the defect probability based on the suspected regions.

[0143] The confirmation module 30 is used to determine whether there is a defect in the image of the workpiece to be inspected based on the defect probability.

[0144] Example 3

[0145] On the other hand, this third embodiment, based on the video image defect detection and recognition processing method provided in the first embodiment of the invention, also provides a computer storage medium 1140 (hereinafter referred to as the storage medium). Figure 9 The diagram shown is a schematic of a computer storage medium structure framework provided in Embodiment 3 of the present invention, which includes:

[0146] Memory 1130 is used to store computer programs;

[0147] Communication interface 1120 is used to connect memory 1140 and processor 1110;

[0148] The processor 1110 is used to execute a computer program to implement a video image defect detection and recognition processing method disclosed in Embodiment 1 of any combination of the above embodiments.

[0149] In summary, the video image defect detection and identification method and system proposed in this invention acquires an image of the workpiece to be inspected from industrial video network data. The method calculates the grayscale value of each pixel in the image, using the grayscale value to identify shadow and light spot areas. The pixels are then sorted by grayscale value, and a pixel distinction threshold is set to differentiate the sorted pixels, resulting in a first set and a second set. The number of pixels within each set is determined, and the set with more pixels is identified as a suspected set. This allows for the effective identification of shadow and light spot areas in the workpiece image captured by the industrial video network data, based on the higher pixel density in defective areas compared to other areas. This enables rapid identification of normal pixels and potentially defective pixels.

[0150] Furthermore, suspected pixels are clustered to obtain suspected regions. Based on the suspected regions, it can be determined that there may be defects in the region. The local grayscale standard deviation and the area of ​​the suspected regions are calculated. The defect probability is obtained through the local grayscale standard deviation and the area of ​​the suspected regions. The presence of defects in the workpiece image is determined based on the level of the defect probability.

[0151] Furthermore, the contour of the suspected region is calculated using suspected pixels at the edges. A closing operation is then performed on the contour of the suspected region: first, dilation and filling, then erosion to restore the contour, resulting in a new contour without gaps. Green's theorem is then used to convert the new contour into a closed curve that encloses the suspected region. Based on selected suspected pixels on the closed curve, a circumferential scan is performed on the suspected region to obtain its basic area. This basic area is then converted to actual unit area (i.e., suspected region area) based on the resolution of the workpiece image. This allows for further confirmation of the basic area of ​​the suspected region, thereby improving its accuracy. Moreover, the area can be calculated for suspected regions of different shapes, solving the problem of detecting defects such as shadows and light spots in the workpiece image.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting and identifying defects in video images, characterized in that, The following steps are included: The process involves acquiring an image of the workpiece to be inspected from industrial video network data, converting the image to grayscale to obtain a grayscale image of the workpiece, and calculating the grayscale value of each pixel in the grayscale image of the workpiece. Suspected pixels are distinguished based on their grayscale values, and these suspected pixels are clustered to obtain suspected regions; the probability of defects is then calculated based on these suspected regions. Determine whether the image of the workpiece to be inspected has defects based on the defect probability; Suspected pixels are distinguished based on their grayscale values, and these suspected pixels are clustered to obtain suspected regions. The probability of defects is then calculated based on these suspected regions. The specific steps are as follows: Based on the gray values ​​of the pixels in the grayscale image of the workpiece to be detected, they are sorted in order from low to high gray values; a preset pixel distinction threshold q is used to determine the gray value boundary of the pixels, thereby obtaining the first set of pixels and the second set of pixels. Determine the difference between the number of pixels in the first pixel set and the number of pixels in the second pixel set; If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, then the first pixel set is determined to be a suspected pixel set, and the second pixel set is determined to be a normal pixel set. If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, then the second pixel set is determined to be a suspected pixel set, and the first pixel set is determined to be a normal pixel set. The normal pixels are clustered using the mean offset algorithm to obtain multiple normal pixel regions; The suspected pixels are clustered using the mean offset algorithm to obtain multiple suspected regions; The suspected region is optimized by deleting isolated suspected pixels at their edges; Calculate the local grayscale standard deviation of the optimized suspected region; The area of ​​the suspected region is calculated based on the suspected pixels of the suspected region, and different suspected region areas are calculated based on suspected regions of different shapes. The defect probability is calculated by combining the local grayscale standard deviation and the area of ​​the suspected region using a weighted method. The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated, and then the workpiece images to be detected in the previous and next frames are collected. The defect probability is further verified by the workpiece images to be detected in the previous and next frames to see if it reaches the final output defect probability.

2. The video image defect detection and recognition processing method according to claim 1, characterized in that, The formula for the defect probability is: s k The probability of a defect in the k-th suspected region; H k Let be the weighted local grayscale standard deviation of the k-th suspected region; A k ' is the area of ​​the k-th suspected region; e is a natural constant.

3. The video image defect detection and recognition processing method according to claim 2, characterized in that, The specific steps for calculating the local grayscale standard deviation of the optimized suspected region are as follows: Within the optimized suspected region, a local window is preset with a size of w*w; each suspected pixel within the suspected region is traversed, and the standard deviation of the gray values ​​of all suspected pixels within the local window is calculated with each suspected pixel as the center; the standard deviations of all local windows are weighted to obtain the local gray standard deviation of the suspected region. The local grayscale standard deviation weighting formula is as follows: In the formula, H k The weighted local grayscale standard deviation of the k-th suspected region; m k Let be the number of suspected pixels in the k-th suspected region; Let be the local grayscale standard deviation of the i-th suspected pixel within the k-th suspected region; w is the weighting coefficient; e is the natural constant.

4. The video image defect detection and recognition processing method according to claim 3, characterized in that, The area of ​​the suspected region is calculated based on the suspected pixels in the suspected region, and different suspected region areas are calculated based on different shapes of suspected regions. The specific operation steps are as follows: Connect the suspected pixels at the edges of the suspected region to obtain the outline of the suspected region; use the closing operation to smooth the outline of the suspected region, and fill the outline of the suspected region with small holes to obtain a new outline of the suspected region. Green's theorem is used to transform the new contour of the suspected region into a closed curve in a plane. The suspected region is then enclosed by the closed curve of the suspected region to obtain the basic area of ​​the suspected region. The base area of ​​the calculated suspected area is used as the first target area to be determined; The area of ​​the first target to be determined calculated by Green's theorem is subjected to a second verification process. If the verification result is correct, the area of ​​the first target to be determined is determined to be the area of ​​the second target.

5. The video image defect detection and recognition processing method according to claim 4, characterized in that, The basic area expression for the suspected region is: In the formula, C is a closed curve on the plane of the suspected region; ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol; D is the suspected region enclosed by the closed curve C; Pdx and Qdy are represented as the integrands of the line integral; P and Q are bivariate functions on the suspected region of D, and the values ​​of P and Q vary through the two variables x and y. x and y represent the coordinates of a point on the closed curve C; ∫∫_D denotes the double integral over the suspected region D; It is represented as the integrand of a double integral; dxdy represents the area elements in the x and y directions; When P = -y and Q = x Enclosing the surface using line integrals, the calculation formula is as follows: Basic area = ∮_C(xdy-ydx); In the formula, ∮_C represents the line integral along the closed curve C; x and y represent the coordinates of a point on the closed curve C; dx and dy represent small changes in x and y, respectively; (xdy-ydx) represents the area swept along the closed curve C by a randomly selected suspected pixel.

6. The video image defect detection and recognition processing method according to claim 5, characterized in that, The area of ​​the first target to be determined calculated using Green's theorem is subjected to a secondary verification process. If the verification result is correct, the area of ​​the first target to be determined is determined to be the area of ​​the second target. Specifically, this includes: The suspected region is decomposed into multiple triangles using a triangulation algorithm; Then calculate the area of ​​each triangle, and finally add up the areas of all the triangles to output the area calculation result of the suspected region. The area calculation result of the suspected area is compared with the area of ​​the first target to be determined calculated by Green's theorem. If the ratio of the difference between the two to the value of the first target area is less than the standard error ratio threshold G%, the verification result is determined to be correct, and the current area of ​​the first target to be determined is determined to be the area of ​​the second target.

7. A video image defect detection and recognition processing system, comprising: Module to be tested; Calculation module; Confirmation module; The detection module is used to acquire the image of the workpiece to be detected from industrial video network data, convert the image of the workpiece to be detected to grayscale to obtain a grayscale image of the workpiece to be detected, and calculate the grayscale value of the pixels in the grayscale image of the workpiece to be detected. The calculation module is used to distinguish suspected pixels based on the grayscale value of the pixels, cluster the suspected pixels to obtain suspected regions, calculate the defect probability based on the suspected regions, and sort the pixels of the grayscale image of the workpiece to be inspected in ascending order of grayscale value. A preset pixel discrimination threshold q is used to determine the grayscale value boundary of pixels, resulting in a first pixel set and a second pixel set. The number of pixels in the first pixel set is compared to the number of pixels in the second pixel set. If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, the first pixel set is determined to be a suspected pixel set, and the second pixel set is determined to be a normal pixel set. If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, the second pixel set is determined to be a suspected pixel set, and the first pixel set is determined to be a normal pixel set. The normal pixels are clustered using a mean shift algorithm to obtain multiple normal pixel regions. The suspected pixels are also clustered using the mean shift algorithm to obtain multiple suspected regions. The suspected region is optimized by deleting isolated suspected pixels at their edges; Calculate the local grayscale standard deviation of the optimized suspected region; calculate the area of ​​the suspected region based on the suspected pixels of the suspected region, and calculate different suspected region areas based on different shapes of suspected regions; combine the local grayscale standard deviation and the suspected region area, and calculate the defect probability using a weighted method; based on the calculated defect probability of the workpiece image to be detected in the current frame of the industrial video network data, collect the workpiece images to be detected in the frames before and after the current frame of the workpiece image to be detected, and further verify whether the defect probability reaches the final output defect probability using the workpiece images to be detected in the frames before and after the current frame; The confirmation module is used to determine whether the image of the workpiece to be inspected has defects based on the defect probability.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the video image defect detection and recognition processing method according to any one of claims 1-6.

Images