Video image defect detection and recognition processing method and system

Abstract

The invention discloses a video image defect detection and identification processing method and system, and the method comprises the following operation steps: obtaining a to-be-detected workpiece image of industrial video network data, carrying out the graying of the to-be-detected workpiece image, and obtaining a to-be-detected workpiece gray image; calculating gray values of pixel points in the gray image of the to-be-detected workpiece; distinguishing suspected pixel points according to the gray values of the pixel points, and clustering the suspected pixel points to obtain suspected areas; calculating the defect probability according to the suspected area; and judging whether the to-be-detected workpiece image has defects or not according to the defect probability.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and system for detecting and identifying video image defects. Background Art

[0002] Product quality inspection is an important part of industrial production. With the development of industrial automation and intelligence, automatic defect detection technology based on machine vision has been widely used. However, there are still some challenges in practical application;

[0003] Video images acquired in industrial environments often have interference such as shadows and light spots, which affect the detection accuracy; different types of defects show different characteristics in the image, which are difficult to detect with a single method; simple threshold methods are prone to false detection or missed detection, and more complex algorithms are needed to improve detection accuracy; detection on the production line requires a fast response, and the efficiency of the algorithm is crucial.

[0004] To solve these problems, researchers have proposed a variety of improvement methods. Image preprocessing improves image quality through grayscale, filtering and other operations; then, grayscale statistical features, texture features, etc. are calculated to distinguish defects from normal areas; studies have found that predefined features (such as grayscale statistics, texture features) may not be able to fully capture the characteristics of all types of defects, and defects of different sizes may require different processing strategies. Traditional methods are difficult to handle multi-scale shape problems at the same time, and it is difficult to quickly detect the regional shapes of shadows and light spots generated by high-resolution images. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes an intelligent defect detection and recognition processing method and system based on video images. The method can detect defects for shadows and light spots of different shapes, and realizes rapid detection of video images.

[0006] The purpose of the present invention is to provide a method and system for detecting and identifying video image defects, which solves the above-mentioned technical problems pointed out in the prior art.

[0007] The present invention provides a video image defect detection and recognition processing method, comprising the following steps:

[0008] Acquire an image of a workpiece to be detected from industrial video network data, grayscale the image of the workpiece to be detected, and obtain a grayscale image of the workpiece to be detected; calculate the grayscale values ​​of pixels in the grayscale image of the workpiece to be detected;

[0009] Distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas; calculate the defect probability according to the suspected areas;

[0010] It is determined whether the image of the workpiece to be inspected has defects according to the defect probability.

[0011] Preferably, suspected pixels are distinguished according to the grayscale values ​​of the pixels, and the suspected pixels are clustered to obtain suspected areas; the defect probability is calculated according to the suspected areas, and the specific operation steps are as follows:

[0012] According to the grayscale values ​​of the pixels of the grayscale image of the workpiece to be detected, the pixels are sorted in descending order of grayscale values; a pixel distinction threshold q is preset, and the grayscale value boundary of the pixels is determined according to the preset pixel distinction threshold q to obtain a first pixel set and a second pixel set;

[0013] Determine the difference between the number of pixels in the first pixel set and the number of pixels in the second pixel set;

[0014] If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, the first pixel set is determined to be suspected pixels, and the second pixel set is determined to be normal pixels;

[0015] If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, the second pixel set is determined to be suspected pixels, and the first pixel set is determined to be normal pixels;

[0016] Clustering the normal pixels using a mean shift algorithm to obtain a plurality of normal pixel regions; clustering the suspected pixels using a mean shift algorithm to obtain a plurality of suspected regions;

[0017] Performing optimization processing on the suspected area by deleting suspected pixel points with isolated edges; calculating the local grayscale standard deviation of the optimized suspected area; calculating the area of ​​the suspected area according to the suspected pixel points of the suspected area, and calculating the areas of different suspected areas according to suspected areas of different shapes;

[0018] The defect probability is calculated by combining the local grayscale standard deviation and the suspected area using a weighted method;

[0019] The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated to collect the workpiece images to be detected in the previous and next frames of the current frame, and the workpiece images to be detected in the previous and next frames are used to further verify whether the defect probability reaches the final output defect probability.

[0020] Preferably, the formula for the defect probability is:

[0021]

[0022] s k is the defect probability of the kth suspected area;

[0023] H k is the weighted local grayscale standard deviation of the kth suspected area;

[0024] A k ' is the suspected area of ​​the kth suspected area;

[0025] e is a natural constant.

[0026] Preferably, the local grayscale standard deviation of the post-optimization suspected area is calculated, and the specific operation steps are as follows:

[0027] In the optimized suspected area, a local window is preset, and the size of the local window is w*w; each suspected pixel point in the suspected area is traversed, and the standard deviation of the grayscale values ​​of all suspected pixels in the local window is calculated with each suspected pixel point as the center; the standard deviations of all local windows are weighted to obtain the local grayscale standard deviation of the suspected area;

[0028] The weighted formula of the local grayscale standard deviation is:

[0029]

[0030] In the formula, H k is the weighted local grayscale standard deviation of the kth suspected area;

[0031] m k is the number of suspected pixels in the kth suspected area;

[0032] is the local grayscale standard deviation of the i-th suspected pixel in the k-th suspected area;

[0033] w is the weighting coefficient; e is a natural constant.

[0034] Preferably, the area of ​​the suspected region is calculated according to the suspected pixels of the suspected region, and the areas of different suspected regions are calculated according to suspected regions of different shapes. The specific operation steps are as follows:

[0035] Connecting the edge suspected pixel points of the suspected area to obtain the outline of the suspected area; smoothing the outline of the suspected area by using a closing operation, filling small holes in the outline of the suspected area to obtain a new outline of the suspected area;

[0036] The new contour of the suspected area is converted into a closed curve of a plane by using Green's theorem, and the suspected area is surrounded by the closed curve of the suspected area to obtain the basic area of ​​the suspected area.

[0037] Preferably, the basic area expression of the suspected area is:

[0038]

[0039] Where C is a closed curve on the plane of the suspected area; ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol;

[0040] D is the suspected area surrounded by C;

[0041] Pdx and Qdy are expressed as the integrand of the line integral; P and Q are binary functions on the suspected region of D, and the values ​​of P and Q vary through the two variables x and y;

[0042] x and y represent the coordinates of a point on the closed curve C;

[0043] ∫∫_D represents the double integral over the suspected region D; is represented as the integrand of a double integral; dxdy represents the area elements in the x and y directions;

[0044] When P = -y and Q = x,

[0045] Use line integral to enclose the surface integral, and the calculation formula is:

[0046] Base area = ∮_C(xdy-ydx);

[0047] Where ∮_C represents the line integral along the closed curve C;

[0048] x and y represent the coordinates of a point on the closed curve C;

[0049] dx and dy represent small changes in x and y, respectively;

[0050] (xdy-ydx) represents the area swept by a random suspected pixel along the closed curve C.

[0051] Preferably, the first target area to be determined calculated by the Green's theorem is subjected to a secondary verification process. If the verification process result is correct, the current first target area to be determined is determined to be the second target area, which specifically includes:

[0052] Decomposing the suspected region of irregular shape into a plurality of triangles by using a triangulation algorithm for the suspected region;

[0053] Then calculate the area of ​​each triangle, and finally add up the areas of all triangles to output the area calculation result value of the suspected area;

[0054] The area calculation result of the suspected area is compared with the first target area to be determined calculated by Green's theorem. If the ratio of the difference between the two and the value of the first target area is less than the standard error ratio threshold G%, the verification processing result is judged to be correct, and the current first target area to be determined is judged to be the second target area.

[0055] Accordingly, the present invention also provides a video image defect detection and identification processing system, comprising: a module to be detected; a calculation module; a confirmation module;

[0056] The module to be detected is used to obtain the image of the workpiece to be detected from the industrial video network data, grayscale the image of the workpiece to be detected, and obtain a grayscale image of the workpiece to be detected; and calculate the grayscale values ​​of the pixels in the grayscale image of the workpiece to be detected;

[0057] The calculation module is used to distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas, and calculate the defect probability according to the suspected areas;

[0058] The confirmation module is used to determine whether the image of the workpiece to be inspected has defects according to the defect probability.

[0059] Correspondingly, the present invention also provides a storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of a video image defect detection and identification processing method described in any one of the above are implemented.

[0060] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0061] From the analysis of the above-mentioned video image defect detection and identification processing method provided by the present invention, it can be known that in specific applications, the image of the workpiece to be detected of the industrial video network data is obtained, the grayscale value of the pixel of the workpiece image to be detected is grayed and the grayscale value of the pixel is calculated, and the shadow and light spot area of ​​the workpiece image to be detected can be judged by the grayscale value of the pixel; the grayscale value of the pixel is sorted, and the pixel distinction threshold is set to distinguish the sorted pixel points by boundaries to obtain a first pixel point set and a second pixel point set, and the number of pixels inside the first pixel point set and the second pixel point set is judged, and the pixel point set with more pixels is judged as a suspected pixel point. In this way, according to the characteristic that the density of pixels in the defective area is higher than that in other areas, the shadow area and light spot area appearing on the image of the workpiece to be detected in the shooting process of the industrial video network data can be effectively judged, and normal pixels and suspected pixels that may have defects can be quickly identified;

[0062] Furthermore, the suspected pixels are clustered to obtain suspected areas, and the suspected areas can be used to determine whether there may be defects in the area; the local grayscale standard deviation and the area of ​​the suspected area are calculated based on the suspected area, and the defect probability is obtained through the local grayscale standard deviation and the area of ​​the suspected area, and whether there is a defect problem in the image of the workpiece to be inspected is determined based on the defect probability;

[0063] Furthermore, the contour of the suspected area is calculated through the suspected pixel points on the edge, and the contour of the suspected area is closed by first expanding and filling, and then corroding to restore the contour of the suspected area, so as to obtain a new contour of the suspected area, so that the contour of the suspected area has no gaps; in this way, the new contour of the suspected area is converted into a closed curve by using Green's theorem to surround the suspected area, and the suspected area is scanned around for one circle according to the selected suspected pixel points on the closed curve to obtain the basic area of ​​the suspected area; then, the basic area of ​​the suspected area is converted into an actual unit area (i.e., the area of ​​the suspected area) according to the resolution of the image of the workpiece to be detected, so that the basic area of ​​the suspected area can be confirmed again, thereby improving the accuracy of the area of ​​the suspected area; and the area can be calculated for suspected areas of different shapes, thereby solving the problem of detection defects of shadows and light spots in the image of the workpiece to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a video image defect detection and identification processing method in Embodiment 1;

[0065] Figure 2 A flowchart of calculating defect probability of a video image defect detection and identification processing method in Embodiment 1;

[0066] Figure 3 A schematic diagram of a video image defect detection and recognition processing method before clustering normal pixels and suspected pixels in Embodiment 1;

[0067] Figure 4 A schematic diagram of a video image defect detection and identification processing method in Embodiment 1 after clustering of normal pixels and suspected pixels;

[0068] Figure 5 A flow chart of calculating the suspected region area of ​​a video image defect detection and identification processing method in Embodiment 1;

[0069] Figure 6 This is a flow chart of verifying the suspected area of ​​a video image defect detection and identification processing method in Example 1;

[0070] Figure 7A schematic diagram of a triangle constructed from irregular edges of a three-dimensional suspected area in a video image defect detection and recognition processing method in Example 1;

[0071] Figure 8 This is a flow chart of a video image defect detection and recognition processing system in Embodiment 2;

[0072] Fig. 9 The present invention is a schematic diagram of the structure of a storage medium to which the above-mentioned video image defect detection, identification and processing method is applied.

[0073] Reference numerals: module to be detected 10 ; calculation module 20 ; confirmation module 30 ; memory 1130 ; communication interface 1120 ; processor 1110 ; computer storage medium 1140 . DETAILED DESCRIPTION

[0074] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0076] Embodiment 1

[0077] See also Figure 1 The first embodiment of the present invention provides a method for detecting and identifying video image defects, including the following steps:

[0078] S10: Acquire an image of a workpiece to be detected from industrial video network data, grayscale the image of the workpiece to be detected, and obtain a grayscale image of the workpiece to be detected; calculate the grayscale values ​​of pixels in the grayscale image of the workpiece to be detected;

[0079] S20: distinguishing suspected pixels according to the grayscale values ​​of the pixels, clustering the suspected pixels to obtain suspected areas; and calculating the defect probability according to the suspected areas;

[0080] It should be noted that when obtaining the image of the workpiece to be detected from the industrial video network data, the image of the workpiece to be detected is detected to determine whether there are defects; however, due to the dim light environment in the industrial workshop, the clear area that can be captured in the image of the workpiece to be detected can be quickly detected, while the image of the workpiece to be detected captured by the video recording equipment produces shadows, light spots and other areas, which will have a serious impact on the defect detection of the image of the workpiece to be detected. Therefore, the grayscale value of the pixel point is used to distinguish the suspected area of ​​the shadow and light spot, and the defect probability of the suspected area is calculated; the defect probability is used to determine whether the image of the workpiece to be detected has defects; and the suspected defects that may exist in the shadow and light spot areas can be expressed by the suspected pixels, and the suspected areas of the shadow and light spot are used to determine whether the image of the workpiece to be detected has defects;

[0081] S30: Determine whether the image of the workpiece to be inspected has defects according to the defect probability.

[0082] It should be noted that when the defect probability is obtained, a probability threshold is set, and whether the image of the workpiece to be detected has defects is judged by comparing the defect probability with the probability threshold; when the defect probability is greater than or equal to the probability threshold, the image of the workpiece to be detected has defects; when the defect probability is less than the probability threshold, the image of the workpiece to be detected has no defects;

[0083] Specifically, see Figure 2 In step S20, the suspected pixels are distinguished according to the grayscale values ​​of the pixels, and the suspected pixels are clustered to obtain suspected areas; the defect probability is calculated according to the suspected areas, and the specific operation steps are as follows:

[0084] S21: sorting the pixels of the grayscale image of the workpiece to be detected in descending order according to the grayscale values; presetting a pixel distinction threshold q, determining the grayscale value boundary of the pixels according to the preset pixel distinction threshold q, and obtaining a first pixel set and a second pixel set (i.e., a pixel set refers to all pixels in the entire set, so the pixel set is a set);

[0085] S22: Determine the difference between the number of pixels in the first pixel point set and the number of pixels in the second pixel point set;

[0086] If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, the first pixel set is determined to be suspected pixels, and the second pixel set is determined to be normal pixels;

[0087] If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, the second pixel set is determined to be suspected pixels, and the first pixel set is determined to be normal pixels;

[0088] It should be noted that, for example, the gray values ​​of the pixels are sorted from 1, 2, 3, ..., 100, and a pixel distinction threshold q is set (assuming q=47), and a distinction boundary is made for the gray values ​​of the pixels (i.e., the pixel distinction threshold q (i.e., 47) is the distinction boundary), the gray values ​​of the pixels from 1 to 47 are the first pixel set, and the gray values ​​of the pixels from 48 to 100 are the second pixel set; and it is determined whether the number of pixels in the first pixel set or the second pixel set is larger, and the pixel set with more pixels is determined as suspected pixels, and the pixel set with less pixels is determined as normal pixels;

[0089] Because, in the process of industrial operation, the industrial video network data will cause some shadow areas and light spot areas to appear in the image of the workpiece to be detected during the shooting process, so firstly, a pixel point distinction threshold q (assuming q = 47) is required to distinguish the grayscale value of the pixel point. Secondly, the density of the pixel points in the defect area is higher than that in other areas, so the first pixel point set and the second pixel point set are judged. The pixel point set with more pixels is used as suspected pixels, and the pixel point set with less pixels is used as normal pixels. In this way, the area of ​​the workpiece image to be detected can be better distinguished, the detection time can be greatly shortened, and the key area of ​​the workpiece image to be detected can be directly detected;

[0090] S23: clustering the normal pixels using a mean shift algorithm to obtain a plurality of normal pixel regions; clustering the suspected pixels using a mean shift algorithm to obtain a plurality of suspected regions;

[0091] It should be noted that the mean shift algorithm will find the density center of the suspected pixel points in the grayscale image of the workpiece to be detected, and use the sliding window method to continuously move the density center toward the direction of the maximum suspected pixel density. When there are multiple densely concentrated areas in the grayscale image of the workpiece to be detected, multiple clustering results (i.e., multiple suspected areas) will be obtained; since the density of the pixels in the defect area is higher than that in other areas (i.e., the area of ​​normal pixels), when the workpiece image to be detected is an irregular curve workpiece, before the normal pixels and the suspected pixels are clustered, refer to Figure 3 , so each clustering result of mean shift is a density concentration area (i.e., suspected area). When the workpiece image to be detected is an irregular curve workpiece, after the normal pixels and the suspected pixels are clustered, see Figure 4 , which can well distinguish which areas in the grayscale image of the workpiece to be inspected may contain suspected defect areas with a high probability, and which areas may contain possible normal areas;

[0092] Therefore, the suspected pixels in the grayscale image of the workpiece to be inspected can be clustered into multiple suspected areas according to the clustering result of mean shift, and each suspected area may have a suspected defect area problem;

[0093] S24: performing optimization processing on the suspected area by deleting suspected pixel points with isolated edges; calculating the local grayscale standard deviation of the optimized suspected area; calculating the area of ​​the suspected area according to the suspected pixel points of the suspected area, and calculating the areas of different suspected areas according to suspected areas of different shapes;

[0094] The defect probability is calculated by combining the local grayscale standard deviation and the suspected area using a weighted method;

[0095] The formula for the defect probability is:

[0096]

[0097] s k is the defect probability of the kth suspected area;

[0098] H k is the weighted local grayscale standard deviation of the kth suspected area;

[0099] A k is the suspected area of ​​the kth suspected area;

[0100] e is a natural constant;

[0101] The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated to collect the workpiece images to be detected in the previous and next frames of the workpiece image to be detected in the current frame, and the defect probability is further verified by the workpiece images to be detected in the previous and next frames to see whether it reaches the final output defect probability;

[0102] It should be noted that by optimizing the suspected area, the noise at the boundary of the suspected area can be removed, and some isolated suspected pixels at the edge of the suspected area that may exist can be removed, so as to avoid deviations in the calculation of the local grayscale standard deviation of the suspected area and the calculation of the area of ​​the suspected area; by calculating the local grayscale standard deviation of the suspected area, the degree of change of the grayscale value of the pixels in the suspected area can be understood; at the same time, it also reflects the uniformity of the grayscale distribution of the pixels in the suspected area; if there may be defective areas in the image of the workpiece to be detected, since the defects usually cause significant shadows and light spots in the grayscale value, the local grayscale standard deviation is often large;

[0103] Specifically, the local grayscale standard deviation of the post-optimization suspected area is calculated, and the specific operation steps are as follows:

[0104] In the optimized suspected area, a local window is preset, and the size of the local window is w*w (that is, 3*3, 4*4); each suspected pixel point in the suspected area is traversed, and the standard deviation of the grayscale values ​​of all suspected pixels in the local window is calculated with each suspected pixel point as the center; the standard deviations of all local windows are weighted to obtain the local grayscale standard deviation of the suspected area;

[0105] The weighted formula of the local grayscale standard deviation is:

[0106]

[0107] In the formula, H k is the weighted local grayscale standard deviation of the kth suspected area;

[0108] m k is the number of suspected pixels in the kth suspected area;

[0109] is the local grayscale standard deviation of the i-th suspected pixel in the k-th suspected area;

[0110] w is the weighting coefficient; e is a natural constant.

[0111] Specifically, see Figure 5 In step S24, the area of ​​the suspected region is calculated according to the suspected pixel points of the suspected region, and the areas of different suspected regions are calculated according to the suspected regions of different shapes. The specific operation steps are as follows:

[0112] S241: Connecting edge suspected pixel points of the suspected area (i.e. edge suspected pixel points refer to suspected pixel points at the edge of the suspected area) to obtain the outline of the suspected area; smoothing the outline of the suspected area by using a closing operation, filling small holes in the outline of the suspected area, and obtaining a new outline of the suspected area;

[0113] It should be noted that the contour of the suspected area is obtained, and the contour of the suspected area is expanded by a closed operation. The shape of the irregular suspected area can be changed into the shape of a regular suspected area by expanding the contour of the suspected area. After the contour of the suspected area is expanded, the area between the suspected pixels of the contour of the suspected area is enlarged, and the area between the suspected pixels is smoothed by filling the small holes or gaps in the area between the suspected pixels. The contour of the expanded suspected area is restored by an erosion operation to form a new contour of the new suspected area (that is, the filled small holes or gaps will not be restored after the erosion operation). In this way, no gaps are generated in the contour of the suspected area, and the accuracy of the calculation of the area of ​​the suspected area by forming a closed curve to surround the suspected area in the subsequent step S242 is improved, and the efficiency of the subsequent scanning of the area of ​​the suspected area by surrounding the closed curve can also be improved.

[0114] S242: using Green's theorem to convert the new contour of the suspected region into a closed curve on a plane, and using the closed curve of the suspected region to surround the suspected region to obtain a basic area of ​​the suspected region;

[0115] The basic area expression of the suspected area is:

[0116]

[0117] Wherein, C is a closed curve on the plane of the suspected region (i.e., the new contour of the suspected region, because the new contour of the suspected region is converted into a closed curve of the line integral of the plane by Green's theorem, so the closed curve can represent the new contour of the suspected region); ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol;

[0118] D is the suspected area surrounded by the closed curve C;

[0119] x and y represent the coordinates of a point on the closed curve C; dx and dy represent small changes in x and y, respectively; Pdx and Qdy (i.e., Pdx+Qdy) are expressed as the integrand of the line integral (i.e., P in Pdx is a binary function, usually a function of x and y, i.e., P(x, y), dx represents a small change or increment of x; Q in Qdy is a binary function, usually a function of x and y, i.e., Q(x, y), dy represents a small change or increment of y); P and Q are binary functions on the suspected area of ​​D, and the values ​​of P and Q change through the two variables x and y;

[0120] x and y represent the coordinates of the suspected pixel points on the closed curve C;

[0121] ∫∫_D represents the double integral (i.e., surface integral) over the suspected region D; The integrand expressed as a double integral (i.e. is the partial derivative of Q with respect to x, is the partial derivative of P with respect to y); dxdy represents the area elements in the x and y directions;

[0122] In the above formula, the shape of the contour of the suspected area (i.e., closed curve) in the image of the workpiece to be detected is analyzed by line integral through Green's theorem, and the contours of suspected areas of different shapes can be calculated; and the surface integral can be used to calculate the basic plane of the suspected area, so that the closed curve and the plane are the same (i.e., the closed curve is the new contour of the suspected area, and the plane of the surface integral is the suspected area, so the closed curve is equal to the plane in the formula), so as to calculate the basic area of ​​the suspected area in the subsequent process; through Green's theorem, the integrand in the image of the workpiece to be detected is expressed as pixel intensity, gradient, and texture features (i.e., calculating the surface integral), and in edge detection, it may represent the gradient of the image of the workpiece to be detected (i.e., calculating the line integral);

[0123] Use surface integral to calculate the base area, and use line integral to enclose the surface integral. The calculation formula is:

[0124] Base area = ∮_C(xdy-ydx);

[0125] Where ∮_C represents the line integral along the closed curve C;

[0126] x and y represent the coordinates of a point on the closed curve C (as explained in the above technical content);

[0127] dx and dy represent small changes in x and y, respectively (as explained in the previous technical content);

[0128] (xdy-ydx) represents the area swept by a random suspected pixel along the closed curve C;

[0129] It should be noted that when P = -y and Q = x,

[0130] In the above, is the partial derivative of Q with respect to x. Since Q = x, is the partial derivative of P with respect to y. Since P = -y, so, At this time, the surface integral is used to calculate the basic area; a random suspected pixel point is selected to move along the closed curve C, and the suspected area is scanned for a circle during the movement (that is, the line integral is used to surround the surface integral), so ∮_C(xdy-ydx) is used to calculate the line integral;

[0131] xdy represents the area increment formed by a small movement in the y direction (i.e., the area increment represents a small change in the area caused by a small movement of the suspected pixel on the closed curve, and also corresponds to the small change and increment mentioned above); ydx represents the area increment formed by a small movement in the x direction (i.e., the area increment represents a small change in the area caused by a small movement of the suspected pixel on the closed curve, and also corresponds to the small change and increment mentioned above);

[0132] The Green's theorem mentioned above can be used to quickly calculate the area of ​​different suspected regions. At the same time, the area calculation accuracy of irregular suspected regions can also be improved.

[0133] The basic area of ​​the suspected area calculated by the above Green's theorem is used as the first target area to be determined.

[0134] See also Figure 6 After S242, the method further includes performing a secondary verification process on the first target area to be determined calculated by the Green's theorem. If the verification process result is correct, the first target area to be determined is determined to be the second target area. The specific operation steps are as follows:

[0135] S2421: Decomposing the irregular-shaped suspected region into a plurality of triangles by using a triangulation algorithm for the suspected region;

[0136] S2422: Then calculate the area of ​​each triangle, and finally add up the areas of all triangles to output the area calculation result value of the suspected area;

[0137] Triangulation: This method breaks down irregular shapes into triangles. A triangulation algorithm, such as Delaunay triangulation, is used to break down the shape into triangles. Triangulation is the process of dividing a polygon into a number of triangles; the area of ​​each triangle is then calculated and the areas of all triangles are added together (i.e., the areas of all triangles are added together to give a rough estimate of the total area of ​​the irregular shape), such as Figure 7 shown.

[0138] S2423: Compare the area calculation result of the suspected area with the first target area to be determined calculated by Green's theorem. If the ratio of the difference between the two and the value of the first target area is less than the standard error ratio threshold G%, the verification processing result is judged to be correct, and the current first target area to be determined is judged to be the second target area; the above-mentioned standard error ratio threshold G% is a preset setting value, which explains that the triangulation method is a rough calculation method, so it cannot be used as the main calculation method for the suspected area. The embodiment of the present application adopts Green's theorem as the main calculation method, but the triangulation method is used for auxiliary verification. As long as the verification processing result is correct, it will continue to be executed.

[0139] Embodiment 2

[0140] like Figure 8 As shown, the present invention also provides a video image defect detection and identification processing system, comprising: a module to be detected 10; a calculation module 20; a confirmation module 30;

[0141] The module to be detected 10 is used to obtain the image of the workpiece to be detected from the industrial video network data, grayscale the image of the workpiece to be detected to obtain a grayscale image of the workpiece to be detected; and calculate the grayscale values ​​of the pixels in the grayscale image of the workpiece to be detected;

[0142] The calculation module 20 is used to distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas, and calculate the defect probability according to the suspected areas;

[0143] The confirmation module 30 is used to determine whether the image of the workpiece to be inspected has defects according to the defect probability.

[0144] Embodiment 3

[0145] On the other hand, this third embodiment is based on the video image defect detection and identification processing method provided in the first embodiment of the invention, and further provides a computer storage medium 1140 (hereinafter referred to as storage medium). Fig. 9 FIG. 1 is a schematic diagram of a computer storage medium structure framework provided in Embodiment 3 of the present invention, which includes:

[0146] A memory 1130, used for storing computer programs;

[0147] The communication interface 1120 is used to connect the memory 1140 to the processor 1110;

[0148] The processor 1110 is used to execute a computer program to implement a video image defect detection and identification processing method involved in the first embodiment disclosed in combination with any of the above-mentioned implementation modes.

[0149] In summary, a method and system for detecting and identifying video image defects proposed in an example of the present invention obtains an image of a workpiece to be detected from industrial video network data, grayscales the image of the workpiece to be detected and calculates the grayscale value of the pixel points, and the shadow and light spot areas of the image of the workpiece to be detected can be determined by the grayscale value of the pixel points; the grayscale values ​​of the pixels are sorted, and a pixel point distinction threshold is set to distinguish the sorted pixels by boundaries to obtain a first pixel point set and a second pixel point set, and the number of pixels within the first pixel point set and the second pixel point set is determined, and a pixel point set with more pixels is determined as a suspected pixel point. In this way, the shadow area and light spot area that appear on the image of the workpiece to be detected in the shooting process of the industrial video network data can be effectively determined according to the characteristic that the density of pixels in the defective area is higher than that in other areas, and normal pixels and suspected pixels that may have defects can be quickly identified;

[0150] Furthermore, the suspected pixels are clustered to obtain suspected areas, and the suspected areas can be used to determine whether there may be defects in the area; the local grayscale standard deviation and the area of ​​the suspected area are calculated based on the suspected area, and the defect probability is obtained through the local grayscale standard deviation and the area of ​​the suspected area, and whether there is a defect problem in the image of the workpiece to be inspected is determined based on the defect probability;

[0151] Furthermore, the contour of the suspected area is calculated through the suspected pixel points on the edge, and the contour of the suspected area is closed by first expanding and filling, and then corroding to restore the contour of the suspected area, so as to obtain a new contour of the suspected area, so that the contour of the suspected area has no gaps; in this way, the new contour of the suspected area is converted into a closed curve by using Green's theorem to surround the suspected area, and the suspected area is scanned around for one circle according to the selected suspected pixel points on the closed curve to obtain the basic area of ​​the suspected area; then, the basic area of ​​the suspected area is converted into an actual unit area (i.e., the area of ​​the suspected area) according to the resolution of the image of the workpiece to be detected, so that the basic area of ​​the suspected area can be confirmed again, thereby improving the accuracy of the area of ​​the suspected area; and the area can be calculated for suspected areas of different shapes, thereby solving the problem of detection defects of shadows and light spots in the image of the workpiece to be detected.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video image defect detection and recognition processing method, characterized in that: The steps are as follows: Acquire an image of a workpiece to be detected from industrial video network data, grayscale the image of the workpiece to be detected, and obtain a grayscale image of the workpiece to be detected; calculate the grayscale values ​​of pixels in the grayscale image of the workpiece to be detected; Distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas; calculate the defect probability according to the suspected areas; It is determined whether the image of the workpiece to be inspected has defects according to the defect probability.

2. The method for detecting and identifying video image defects according to claim 1, characterized in that: Distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas; calculate the defect probability according to the suspected areas, and the specific operation steps are as follows: According to the grayscale values ​​of the pixels of the grayscale image of the workpiece to be detected, the pixels are sorted in descending order of grayscale values; a pixel distinction threshold q is preset, and the grayscale value boundary of the pixels is determined according to the preset pixel distinction threshold q to obtain a first pixel set and a second pixel set; Determine the difference between the number of pixels in the first pixel set and the number of pixels in the second pixel set; If the number of pixels in the first pixel set is greater than the number of pixels in the second pixel set, the first pixel set is determined to be suspected pixels, and the second pixel set is determined to be normal pixels; If the number of pixels in the second pixel set is greater than the number of pixels in the first pixel set, the second pixel set is determined to be suspected pixels, and the first pixel set is determined to be normal pixels; Clustering the normal pixel points using a mean shift algorithm to obtain a plurality of normal pixel point regions; Clustering the suspected pixels using a mean shift algorithm to obtain a plurality of suspected regions; Performing optimization processing on the suspected area by deleting suspected pixel points with isolated edges; Calculate the local grayscale standard deviation of the optimized suspected area; Calculating the area of ​​the suspected region according to the suspected pixels of the suspected region, and calculating the areas of different suspected regions according to the suspected regions of different shapes; The defect probability is calculated by combining the local grayscale standard deviation and the suspected area using a weighted method; The defect probability of the workpiece image to be detected in the current frame of the industrial video network data is calculated to collect the workpiece images to be detected in the previous and next frames of the current frame, and the workpiece images to be detected in the previous and next frames are used to further verify whether the defect probability reaches the final output defect probability.

3. The method for detecting and identifying video image defects according to claim 2, characterized in that: The formula for the defect probability is: s k is the defect probability of the kth suspected area; H k is the weighted local grayscale standard deviation of the kth suspected area; A k ' is the suspected area of ​​the kth suspected area; e is a natural constant.

4. The method for detecting and identifying video image defects according to claim 3, characterized in that: The specific operation steps of calculating the local grayscale standard deviation of the post-optimization suspected area are as follows: In the optimized suspected area, a local window is preset, and the size of the local window is w*w; each suspected pixel point in the suspected area is traversed, and the standard deviation of the grayscale values ​​of all suspected pixels in the local window is calculated with each suspected pixel point as the center; the standard deviations of all local windows are weighted to obtain the local grayscale standard deviation of the suspected area; The weighted formula of the local grayscale standard deviation is: In the formula, H k is the weighted local grayscale standard deviation of the kth suspected area; m k is the number of suspected pixels in the kth suspected area; is the local grayscale standard deviation of the i-th suspected pixel in the k-th suspected area; w is the weighting coefficient; e is a natural constant.

5. The method for detecting and identifying video image defects according to claim 4, characterized in that: The area of ​​the suspected region is calculated according to the suspected pixel points of the suspected region, and the areas of different suspected regions are calculated according to the suspected regions of different shapes. The specific operation steps are as follows: Connecting the edge suspected pixel points of the suspected area to obtain the outline of the suspected area; smoothing the outline of the suspected area by using a closing operation, filling small holes in the outline of the suspected area to obtain a new outline of the suspected area; The new contour of the suspected area is converted into a closed curve of a plane by using Green's theorem, and the suspected area is surrounded by the closed curve of the suspected area to obtain the basic area of ​​the suspected area; The calculated base area of ​​the suspected area is used as the first target area to be determined; The first target area to be determined calculated by the Green's theorem is subjected to a secondary verification process. If the verification process result is correct, the first target area to be determined is determined to be the second target area.

6. A video image defect detection and recognition processing method according to claim 5, characterized in that: The basic area expression of the suspected area is: Where C is a closed curve on the plane of the suspected area; ∮_C represents the line integral along the closed curve C, and ∮ represents the line integral symbol; D is the suspected area surrounded by the closed curve C; Pdx and Qdy are expressed as the integrand of the line integral; P and Q are binary functions on the suspected region of D, and the values ​​of P and Q vary through the two variables x and y; x and y represent the coordinates of a point on the closed curve C; ∫∫_D represents the double integral over the suspected region D; is represented as the integrand of a double integral; dxdy represents the area elements in the x and y directions; When P = -y and Q = x, Use line integral to enclose the surface integral, and the calculation formula is: Base area = ∮_C(xdy-ydx); Where ∮_C represents the line integral along the closed curve C; x and y represent the coordinates of a point on the closed curve C; dx and dy represent small changes in x and y, respectively; (xdy-ydx) represents the area swept by a random suspected pixel along the closed curve C.

7. The method for detecting and identifying video image defects according to claim 6, characterized in that: The first target area to be determined calculated by the Green's theorem is subjected to a secondary verification process. If the verification process result is correct, the first target area to be determined is determined to be the second target area, which specifically includes: Decomposing the suspected region of irregular shape into a plurality of triangles by using a triangulation algorithm for the suspected region; Then calculate the area of ​​each triangle, and finally add up the areas of all triangles to output the area calculation result value of the suspected area; The area calculation result of the suspected area is compared with the first target area to be determined calculated by Green's theorem. If the ratio of the difference between the two and the value of the first target area is less than the standard error ratio threshold G%, the verification processing result is judged to be correct, and the current first target area to be determined is judged to be the second target area.

8. A video image defect detection, recognition and processing system, comprising: Module to be tested; Computation module; Confirmation module; The module to be detected is used to obtain the image of the workpiece to be detected from the industrial video network data, grayscale the image of the workpiece to be detected, and obtain a grayscale image of the workpiece to be detected; and calculate the grayscale values ​​of the pixels in the grayscale image of the workpiece to be detected; The calculation module is used to distinguish suspected pixels according to the grayscale values ​​of the pixels, cluster the suspected pixels to obtain suspected areas, and calculate the defect probability according to the suspected areas; The confirmation module is used to determine whether the image of the workpiece to be inspected has defects according to the defect probability.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the video image defect detection and identification processing method described in any one of claims 1 to 7 are implemented.

Images