Product detection method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a product detection method and device, electronic equipment and a readable storage medium, which are applied to the technical field of image processing, and the method comprises the steps: extracting a plurality of product edge feature points from an image of a circular product, carrying out the fitting of the plurality of product edge feature points to obtain an ellipse, and carrying out the building of a standard circle through the ellipse, then, a perspective transformation matrix is constructed by using an ellipse and a standard circle, perspective transformation is carried out on the multiple product edge feature points and images respectively, non-standard circles presented by qualified circular products in the images due to camera shooting angle or product position inclination are corrected, complete visual angle correction is achieved, and the product quality is improved. And finally, based on the plurality of transformation feature points obtained by perspective transformation and the target image obtained by perspective transformation, carrying out qualification detection on the circular product, so that the over-detection rate can be reduced, and the detection efficiency and precision of the circular product can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and particularly relates to a product detection method, a product detection device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Currently, circular products are widely used in industries such as electronics, photovoltaic, military, and medical. During the production process, some defects often occur in products, and circular products are no exception. If defective products are not detected in time and are applied to the next process, non-defective materials in the next process will be scrapped together, resulting in unnecessary waste.

[0003] Currently, the qualification of circular products can be detected based on the pictures of circular products taken by a camera. However, due to reasons such as the camera shooting angle or the inclination of the product placement position, qualified circular products often appear as non-standard circles in the pictures, resulting in over-inspection. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a product detection method, a product detection device, an electronic device, and a computer-readable storage medium, which can correct the non-standard circles presented by qualified circular products in pictures caused by the camera shooting angle or product position inclination based on perspective transformation, so as to perform product detection after complete perspective correction, reduce the over-inspection rate, and improve the detection efficiency and accuracy of circular products.

[0005] To solve the above technical problems, the present application provides a product detection method, including:

[0006] Extract multiple product edge feature points from the image of the circular product;

[0007] Fit an ellipse based on the multiple product edge feature points, and construct a standard circle according to the ellipse;

[0008] Construct a perspective transformation matrix based on the ellipse and the standard circle;

[0009] Perform perspective transformation on the multiple product edge feature points using the perspective transformation matrix to obtain multiple transformed feature points; perform perspective transformation on the image using the perspective transformation matrix to obtain a target image;

[0010] Perform qualification detection on the circular product based on the multiple transformed feature points and the target image.

[0011] Optionally, extracting multiple product edge feature points from the image of the circular product includes:

[0012] After obtaining the image, perform noise reduction processing on the image;

[0013] The multiple product edge feature points are extracted from the image after noise reduction processing.

[0014] Optionally, a standard circle is constructed based on the ellipse, including:

[0015] A radius value is calculated based on the major axis and minor axis of the ellipse;

[0016] With the center coordinates of the ellipse as the center and the radius value, the standard circle is constructed.

[0017] Optionally, a perspective transformation matrix is constructed based on the ellipse and the standard circle, including:

[0018] Based on the ellipse and the standard circle, a translation matrix, a rotation matrix, a shear matrix, and a scaling matrix are constructed;

[0019] The product of the translation matrix, the rotation matrix, the shear matrix, and the scaling matrix is used as the perspective transformation matrix.

[0020] Optionally, the multiple product edge feature points are perspectively transformed using the perspective transformation matrix, including:

[0021] The multiple product edge feature points are respectively multiplied by the perspective transformation matrix to perspectively transform the multiple product edge feature points.

[0022] Optionally, the image is perspectively transformed using the perspective transformation matrix, including:

[0023] Each pixel point in the image is multiplied by the perspective transformation matrix to perspectively transform the image.

[0024] Optionally, based on the multiple transformed feature points and the target image, a qualification detection is performed on the circular product, including:

[0025] Based on the multiple transformed feature points, at least one defect on the circular product is determined;

[0026] At least one target area where the at least one defect is located is determined;

[0027] In at least one target area in the target image, the characteristic parameters of the defects in the corresponding target area are calculated;

[0028] Based on the characteristic parameters and the number of defects, a qualification detection is performed on the circular product.

[0029] Optionally, based on the multiple transformed feature points, at least one defect on the circular product is determined, including:

[0030] Calculate the distances between the multiple transformed feature points and the circumference of the standard circle respectively;

[0031] Take the transformed feature points corresponding to the distances greater than the set value as target points;

[0032] Classify the target points, and take at least one group of the classified target points as at least one defect.

[0033] Optionally, perform the following steps for each defect:

[0034] Enlarge the defect area enclosed by a group of target points corresponding to any one defect and the circumference of the standard circle;

[0035] Take the enlarged defect area as a target area.

[0036] Optionally, calculate the characteristic parameters of the defects in at least one target area in the target image, including:

[0037] Perform binarization processing on at least one target area in the target image;

[0038] Calculate the side length and area of the defects in the corresponding target area after binarization processing as the characteristic parameters of the defects.

[0039] Optionally, perform qualification detection on the circular product based on the characteristic parameters and the number of defects, including:

[0040] If the characteristic parameters of any one defect exceed the qualified parameter range or the number of defects exceeds the quantity threshold, determine that the circular product is unqualified; otherwise, determine that the circular product is qualified.

[0041] This application also provides a product detection device, including:

[0042] An extraction module, configured to extract multiple product edge feature points from an image of a circular product;

[0043] A fitting module, configured to fit an ellipse based on the multiple product edge feature points and construct a standard circle according to the ellipse;

[0044] A construction module, configured to construct a perspective transformation matrix based on the ellipse and the standard circle;

[0045] A transformation module, configured to perform perspective transformation on the multiple product edge feature points by using the perspective transformation matrix to obtain multiple transformed feature points; perform perspective transformation on the image by using the perspective transformation matrix to obtain a target image;

[0046] A detection module, configured to perform qualification detection on the circular product based on the multiple transformed feature points and the target image.

[0047] Optionally, the extraction module is specifically configured to:

[0048] After obtaining the image, perform noise reduction processing on the image;

[0049] Extract the multiple product edge feature points from the noise-reduced image.

[0050] Optionally, the fitting module is specifically configured to:

[0051] Calculate a radius value based on the major axis and minor axis of the ellipse;

[0052] With the center coordinates of the ellipse as the center and using the radius value, construct the standard circle.

[0053] Optionally, the construction module is specifically configured to:

[0054] Based on the ellipse and the standard circle, construct a translation matrix, a rotation matrix, a shear matrix, and a scaling matrix;

[0055] Take the product of the translation matrix, the rotation matrix, the shear matrix, and the scaling matrix as the perspective transformation matrix.

[0056] Optionally, the transformation module is specifically configured to:

[0057] Multiply each of the multiple product edge feature points by the perspective transformation matrix to perform perspective transformation on the multiple product edge feature points.

[0058] Optionally, the transformation module is specifically configured to:

[0059] Multiply each pixel point in the image by the perspective transformation matrix to perform perspective transformation on the image.

[0060] Optionally, the detection module includes:

[0061] A first determination unit for determining at least one defect on the circular product based on the multiple transformed feature points;

[0062] A second determination unit for determining at least one target area where the at least one defect is located;

[0063] A calculation unit for calculating characteristic parameters of the defect in the corresponding target area within at least one target area in the target image;

[0064] A detection unit for performing qualification detection on the circular product based on the characteristic parameters and the number of defects.

[0065] Optionally, the first determination unit is specifically configured to:

[0066] Calculate the distances between the multiple transformed feature points and the circumference of the standard circle respectively;

[0067] Take the transformed feature points corresponding to the distances greater than the set value as target points;

[0068] Classify the target points, and take at least one set of classified target points as at least one defect.

[0069] Optionally, the second determination unit is specifically configured to perform the following steps for each defect:

[0070] Enlarge the defect area enclosed by a set of target points corresponding to any defect and the circumference of the standard circle;

[0071] Take the enlarged defect area as a target area.

[0072] Optionally, the calculation unit is specifically configured to:

[0073] Perform binarization processing on at least one target area in the target image;

[0074] Calculate the side length and area of the defect in the corresponding target area after binarization processing as the characteristic parameters of the defect.

[0075] Optionally, the detection unit is specifically configured to:

[0076] If the characteristic parameters of any defect exceed the qualified parameter range or the number of defects exceeds the quantity threshold, determine that the circular product is unqualified; otherwise, determine that the circular product is qualified.

[0077] This application also provides an electronic device, including a memory and a processor, where:

[0078] The memory is used to store a computer program;

[0079] The processor is used to execute the computer program to implement the above product detection method.

[0080] This application also provides a computer-readable storage medium for storing a computer program, where the computer program, when executed by a processor, implements the above product detection method.

[0081] The product detection method provided by this application, after extracting multiple product edge feature points from the image of a circular product, fits an ellipse based on the multiple product edge feature points, constructs a standard circle relying on the ellipse, and then uses the perspective transformation matrix constructed by the ellipse and the standard circle to perform perspective transformation on the multiple product edge feature points and the image respectively, so as to correct the non-standard circles presented by qualified circular products in the picture due to the camera shooting angle or the inclination of the product position, realizing complete perspective correction. Finally, based on the multiple transformed feature points obtained by perspective transformation and the target image obtained by perspective transformation, the qualification detection of circular products is carried out, which can reduce the passing rate and improve the detection efficiency and accuracy of circular products.

[0082] In addition, this application also provides a product detection device, an electronic device and a computer-readable storage medium, which also have the above beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the drawings required to be used in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only the embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0084] Figure 1 It is a schematic structural diagram of an electronic device provided by an embodiment of this application;

[0085] Figure 2 It is a flowchart of a product detection method provided by an embodiment of this application;

[0086] Figure 3 It is a comparison diagram of a deformed image and a non-deformed image provided by an embodiment of this application;

[0087] Figure 4 It is another comparison diagram of a deformed image and a non-deformed image provided by an embodiment of this application;

[0088] Figure 5 It is a schematic diagram of a product defect provided by an embodiment of this application;

[0089] Figure 6 It is another flowchart of a product detection method provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0090] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0091] Each step in this application can be executed by a specified electronic device, and the form of the specified electronic device is not limited. For example, it can be a general computing device such as a computer or a server. Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of an electronic device provided by an embodiment of this application. The electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of a multimedia component 103, an information input / output (I / O) interface 104, and a communication component 105.

[0092] Among them, the processor 101 is used to control the overall operation of the electronic device 100 to complete all or part of the steps in the above product detection method; the memory 102 is used to store various types of data to support the operation of the electronic device 100. These data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0093] The multimedia component 103 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used for outputting and / or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 102 or sent through the communication component 105. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 104 provides an interface between the processor 101 and other interface modules, and the other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them. Accordingly, the communication component 105 may include: a Wi-Fi component, a Bluetooth component, an NFC component.

[0094] The electronic device 100 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for performing the product detection method proposed in this application.

[0095] In a specific embodiment, the above-mentioned electronic device may include a material loading component, a detection component, and a control component. Among them, the control component may be the above-mentioned processor 101, and the detection component may be connected to other components such as other control components through the above-mentioned I / O interface 104. The material loading component is used to load materials waiting to be tested for products, such as circular products. The control component may specifically be composed of a motion control card, an industrial control computer, and a display. The industrial control computer can, on the one hand, control the detection component to move to the detection position of the material through the motion control card. Specifically, the material is placed on the material loading component, and then the control component moves the material to the detection position. The detection component takes an on-line image through a camera therein, and then performs calculation and analysis based on the product detection method of the present application to obtain a product detection result, and finally returns the product detection result to the control component. The control component can further visually display the product detection result, and when necessary, for example, if the detection result is not within the tolerance range of the material, the control component can also give a reminder and warning.

[0096] Please refer to Figure 2 , Figure 2 which is a flowchart of a product detection method provided by an embodiment of the present application. The method includes:

[0097] S201. Extract multiple product edge feature points from the image of the circular product.

[0098] The product edge feature points are generally the points with the highest contrast at the edge. It should be noted that the circular product can be placed on the material loading component. After the circular product is moved to the detection position by the control component connected to the material loading component, the detection component controls the camera therein to take an on-line image, so as to obtain an image of the circular product. Then, the product detection method of this embodiment is run for calculation and analysis to obtain a product detection result.

[0099] Generally, in order to improve the image detection effect, the image can be preprocessed when extracting the product edge feature points. For example: perform median filtering, mean filtering, or Gaussian filtering on the image to eliminate image noise and reduce the interference of image noise. Therefore, in one embodiment, extracting multiple product edge feature points from the image of the circular product includes: after obtaining the image, performing noise reduction processing on the image; extracting multiple product edge feature points from the image after noise reduction processing. The way to extract the product edge feature points can be: using conventional edge detection operators, such as canny operator or sobel operator, etc., to extract edge points to obtain multiple product edge feature points. In one example, n product edge feature points can be recorded as a set P{p1, p2, p3... pn}.

[0100] S202. Fit an ellipse based on multiple product edge feature points and construct a standard circle according to the ellipse.

[0101] According to the standard ellipse formula Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0, fitting multiple edge feature points of the product can obtain an ellipse, and then the center coordinates, major axis and minor axis of the ellipse can be calculated. Next, a standard circle is constructed based on the center coordinates, major axis and minor axis of the ellipse. In one implementation, constructing a standard circle based on the ellipse includes: calculating a radius value according to the major axis and minor axis of the ellipse; using the center coordinates of the ellipse as the center of the circle and constructing a standard circle with the radius value. Among them, the radius value of the standard circle can be r = (a + b) / 2, where a and b represent the lengths of the two semi-axes of the ellipse.

[0102] S203. Construct a perspective transformation matrix based on the ellipse and the standard circle.

[0103] Theoretically, the image of the circular product captured by the camera is Figure 3 A in, but due to the influence of the shooting angle or the inclination of the product placement, the actual image of the circular product captured by the camera is Figure 3 B in. Please refer to Figure 4 , theoretically, the plane where the camera lens is located needs to be parallel to the plane where the material product is located. When the two are parallel, the image of the circular product captured by the camera can be Figure 3 A in, and the corresponding side view and top view are as Figure 4 A in, and the plane where the camera and the material product are located are parallel to each other; but since the product may be inclined, then the surface of the camera and the surface of the product are not parallel. When the two are not parallel, the image of the circular product captured by the camera can be Figure 3 B in, and the corresponding side view is as Figure 4 B in, and the plane where the camera and the product are located are not parallel. To solve this problem, in this embodiment, perspective transformation can be used to convert the Figure 3 and Figure 4 the non-parallel lines shown in into parallel lines, which can achieve complete perspective correction and has higher correction accuracy; it can also correct the ellipse presented by the qualified circular product in the picture due to the camera shooting angle or the inclination of the product position into a standard circle. In one implementation, constructing a perspective transformation matrix based on the ellipse and the standard circle includes: constructing a translation matrix, a rotation matrix, a shear matrix and a scaling matrix based on the ellipse and the standard circle; taking the product of the translation matrix, the rotation matrix, the shear matrix and the scaling matrix as the perspective transformation matrix. The perspective transformation matrix can be: , and the four matrices multiplied in sequence in the formula are: translation matrix, rotation matrix, shear matrix and scaling matrix.

[0104] S304. Use the perspective transformation matrix to perform perspective transformation on multiple product edge feature points to obtain multiple transformed feature points; use the perspective transformation matrix to perform perspective transformation on the image to obtain the target image.

[0105] In this embodiment, performing perspective transformation on multiple product edge feature points and the image respectively using the perspective transformation matrix means: assigning the perspective transformation matrix to each product edge feature point and the image of the circular product. The specific assignment process is: multiplying each product edge feature point by the perspective transformation matrix respectively, and multiplying each pixel in the circular product image by the perspective transformation matrix respectively. Therefore, in one implementation, performing perspective transformation on multiple product edge feature points using the perspective transformation matrix includes: multiplying multiple product edge feature points by the perspective transformation matrix respectively to perform perspective transformation on multiple product edge feature points. In one implementation, performing perspective transformation on the image using the perspective transformation matrix includes: multiplying each pixel point in the image by the perspective transformation matrix to perform perspective transformation on the image.

[0106] S205. Perform qualification detection on the circular product based on multiple transformed feature points and the target image.

[0107] Since there may be more than one defect on the circular product shown in the circular product image, in order to achieve more accurate product compliance detection, this embodiment first locates these defects and then further detects the size of each defect. In one implementation, performing qualification detection on the circular product based on multiple transformed feature points and the target image includes: determining at least one defect on the circular product based on the multiple transformed feature points obtained by perspective transformation, thereby determining the total number of defects; determining at least one target area where at least one defect is located to locate each defect; calculating the characteristic parameters of the defects in the corresponding target area within at least one target area in the target image to detect the size of each defect; performing qualification detection on the circular product based on the characteristic parameters and the number of defects.

[0108] Among them, determining at least one defect on the circular product based on the multiple transformed feature points obtained by perspective transformation includes: calculating the distances between the multiple transformed feature points obtained by perspective transformation and the circumference of the standard circle respectively; taking the transformed feature points corresponding to the distances greater than the set value as target points; classifying the target points, and taking at least one set of target points obtained by classification as at least one defect; the set value is taken according to experience. For example, if the side length of most defects is 5 millimeters, then the set value can be taken as 5 millimeters. As Figure 5As shown, the transformed feature points corresponding to distances greater than the set value are: points not on the circumference of the standard circle, and these points appear as protrusions or depressions on the circumference, and the protrusions or depressions are the product defects. Classifying the target points can classify and determine at least one set of target points, and one set of target points represents one defect, from which the number of defects on the circumference can be determined. For example Figure 5 , if there is one protrusion and one depression on the circumference, then for this depression, the target points enclosing the defect can be offset by a preset distance in the direction of the center of the standard circle; the enlarged concave region formed by the offset set of target points and the edge of the standard circle is used as a target region; for this protrusion, the target points enclosing the defect can be offset by a preset distance in the opposite direction of the center of the standard circle; the enlarged convex region formed by the offset set of target points and the edge of the standard circle is used as a target region. It can be seen that the offset directions for protrusions and depressions are different, but both are to enlarge the region where the defect is located to facilitate measuring the defect size. For example Figure 5 , the regions enclosed by the offset target points are larger, and this region also encloses the originally possibly missed defect region, and can more accurately determine the characteristic parameters such as the shape and size of the defect.

[0109] Therefore, in one implementation, the following steps are performed for each defect: expanding the defect region enclosed by a set of target points corresponding to any defect and the circumference of the standard circle; using the expanded defect region as a target region. Among them, expanding the defect region enclosed by a set of target points corresponding to any defect and the circumference of the standard circle includes: for concave defects, offsetting the target points enclosing the defect by a preset distance in the direction of the center of the standard circle; using the concave region formed by the offset set of target points and the edge of the standard circle as a target region; for convex defects, offsetting the target points enclosing the defect by a preset distance in the opposite direction of the center of the standard circle; using the convex region formed by the offset set of target points and the edge of the standard circle as a target region.

[0110] In this embodiment, in order to calculate the defect size characteristic parameters, the image area where the defect is located is first binarized. In one implementation, calculating the characteristic parameters of the defect in at least one target region in the target image includes: binarizing at least one target region in the target image; calculating the side length and area of the defect in the corresponding target region after binarization as the characteristic parameters of the defect. The defect may be a regular shape or an irregular shape, and there are generally multiple side lengths.

[0111] In one implementation, the qualification detection of circular products is based on characteristic parameters and the number of defects, including: if the characteristic parameters of any defect exceed the qualified parameter range or the number of defects exceeds the quantity threshold, it is determined that the circular product is unqualified; otherwise, it is determined that the circular product is qualified. In one example, when there are too many defects on a product, regardless of the size of the defects, the product can be directly regarded as an unqualified product; when there are not many defects on the product, then it is determined whether the product is qualified by judging whether the defects on it exceed the qualified parameter range. Generally speaking, as long as there is one defect on a product that exceeds the qualified parameter range, the product can be regarded as an unqualified product.

[0112] It can be seen that in this embodiment, after extracting multiple product edge feature points from the image of the circular product, an ellipse is fitted based on the multiple product edge feature points, and a standard circle is constructed relying on the ellipse. Then, using the perspective transformation matrix constructed by the ellipse and the standard circle, the perspective transformation is performed on the multiple product edge feature points and the image respectively, so as to correct the non-standard circle presented by the qualified circular product in the picture due to the camera shooting angle or the inclination of the product position, realizing complete perspective correction. Finally, based on the multiple transformed feature points obtained by the perspective transformation and the target image obtained by the perspective transformation, the qualification detection of the circular product is carried out, which can reduce the passing rate and improve the detection efficiency and accuracy of the circular product.

[0113] Please refer to Figure 6 , another product detection process includes:

[0114] 1. Perform image preprocessing on the image to be detected to eliminate noise interference.

[0115] In this embodiment, the image preprocessing method can be: median filtering, mean filtering or Gaussian filtering, etc., to reduce the interference of noise.

[0116] 2. Extract the region of interest (the region where the material is located) from the preprocessed image, and perform edge extraction within this region to obtain edge feature points, that is, product edge feature points.

[0117] Among them, the edge extraction operation can use conventional edge detection operators, such as canny operator, sobel operator, etc., to obtain the edge feature point set of the circular product, defined as P{p1, p2, p3... pn}. Specifically, the gradient can be calculated by calculating the difference of at least two pixels horizontally or vertically in the image, then the gradient value of the edge is obtained, and the maximum gradient value is used as the feature edge.

[0118] 3. Perform ellipse fitting processing on the above-mentioned edge feature points, and calculate a standard circle according to the fitted ellipse to obtain the parameters of the standard ideal circle.

[0119] Among them, according to the ellipse standard formula Ax2 +Bxy + Cy 2 +Dx + Ey + F = 0. Fitting multiple edge feature points of the product can obtain an ellipse. After operations such as rotation, the following ellipse formula can be constructed: , where and represent the horizontal and vertical coordinates of the center of the ellipse, and a and b represent the lengths of the two semi - axes of the ellipse. The next step is to construct a standard circle based on the center coordinates, major axis, and minor axis of the ellipse: , r = (a + b) / 2.

[0120] 4. Calculate the mapping matrix (i.e., the perspective transformation matrix) through the above - mentioned ellipse and ideal circle, and transform the edge points extracted in step 2 according to the mapping matrix and transform the image according to the mapping matrix.

[0121] Based on the difference between the ellipse and the standard circle, a translation matrix, a rotation matrix, a shear matrix, and a scaling matrix are constructed; the product of the translation matrix, the rotation matrix, the shear matrix, and the scaling matrix is used as the perspective transformation matrix M. Perform perspective transformation on each point in P, and the point set Pt after perspective transformation is Pt = M×P. Similarly, perform perspective transformation on the image to eliminate the angle between the product plane and the camera plane caused by the offset or tilt of the material incoming position, that is, eliminate the difference between the fitted standard circle and the actual circular product, thereby reducing false detection and missed detection.

[0122] 5. Calculate the distance from the above - mentioned feature points to the standard circle, filter out the feature points with distance values less than the set value, and then cluster the remaining feature points into different groups.

[0123] In this step, calculate the distance D from Pt to the center of the standard circle, retain the points with distances greater than the set value to obtain Pc; then cluster Pc, and divide Pc into groups of point sets.

[0124] 6. Offset the feature points of each of the above - mentioned groups along the direction of the line connecting to the center of the circle, form an enclosed area with the circle, process this area, extract the exact defect size, and then perform judgment and screening to obtain accurate product defect results.

[0125] For each group, offset the points in the current group along the line connecting the points to the standard center of the circle so that the offset range can fully cover the defect corresponding to this group. The offset distance is tentatively set to 5 pixels, and the specific offset value can be set according to the actual situation and experience; connect the offset points in sequence, and these points and the circumference construct a polygon area, which is the area where the defect is located.

[0126] In the target image obtained by perspective transformation, binarize the aforementioned determined polygonal region, accurately locate the defect positions therein, calculate parameters such as the length, width, and area of the defects, and perform screening based on these parameters. For example: Parameters with small values are not considered defects and are excluded.

[0127] In this embodiment, after photographing the product, edge extraction is performed on the edges of the product in the picture to obtain N edge points; an ellipse is fitted based on the N edge points and the ellipse formula; a standard circle is constructed with reference to the axis lengths of the ellipse; a perspective transformation matrix M is constructed based on the circle and the ellipse; the N edge points are perspectively transformed using the perspective transformation matrix M to obtain N new points; the original picture is perspectively transformed using the perspective transformation matrix M to obtain a new picture; the distances from the N new points to the center of the standard circle are calculated respectively, and the points with distances greater than the set value are retained; the retained points are classified, and one category obtained is a defect location, and multiple categories indicate multiple defect locations; then for each defect location, the retained points are moved a little towards the far end of the defect, and the moved points are connected so that the connected polygonal region completely covers the defect; in the new picture, the polygonal region is binarized to locate the defects in the region, determine parameters such as the length, width, and area of the defects, and judge based on the parameters of the defects. Those that do not meet the set conditions are not considered defects. For example: If the defect parameters are too small, the product is considered qualified to avoid over-inspection.

[0128] It can be seen that in this embodiment, after extracting the edge feature points, an ellipse is fitted to these feature points, then a standard circle is calculated based on the fitted ellipse, and the mapping relationship from the ellipse to the standard circle is calculated. Finally, this mapping relationship is brought into the image to achieve the projection transformation of the image, which can effectively correct the deformation of the material in the image, effectively solve the problem that the defects are too small to be missed due to the non-parallelism between the material and the camera's field of view plane, and can also solve the problem of over-inspection caused by inaccurate edge fitting, and can improve the product detection efficiency and detection accuracy.

[0129] Next, the product detection device provided by the embodiments of the present application will be introduced. The product detection device described below can be mutually referred to with the product detection method described above.

[0130] The embodiments of the present application provide a product detection device, including:

[0131] An extraction module for extracting a plurality of product edge feature points from the image of the circular product;

[0132] A fitting module for fitting an ellipse based on a plurality of product edge feature points and constructing a standard circle according to the ellipse;

[0133] A construction module for constructing a perspective transformation matrix based on the ellipse and the standard circle;

[0134] A transformation module, configured to perform perspective transformation on multiple product edge feature points by using a perspective transformation matrix to obtain multiple transformed feature points; and perform perspective transformation on an image by using the perspective transformation matrix to obtain a target image.

[0135] A detection module, configured to perform qualification detection on circular products based on multiple transformed feature points and the target image.

[0136] In one implementation, the extraction module is specifically configured to:

[0137] After obtaining the image, perform noise reduction processing on the image;

[0138] Extract multiple product edge feature points from the image after noise reduction processing.

[0139] In one implementation, the fitting module is specifically configured to:

[0140] Calculate a radius value according to the major axis and minor axis of an ellipse;

[0141] Construct a standard circle with the center coordinates of the ellipse as the center and the radius value.

[0142] In one implementation, the construction module is specifically configured to:

[0143] Based on the ellipse and the standard circle, construct a translation matrix, a rotation matrix, a shear matrix, and a scaling matrix;

[0144] Take the product of the translation matrix, the rotation matrix, the shear matrix, and the scaling matrix as the perspective transformation matrix.

[0145] In one implementation, the transformation module is specifically configured to:

[0146] Multiply each of the multiple product edge feature points by the perspective transformation matrix to perform perspective transformation on the multiple product edge feature points.

[0147] In one implementation, the transformation module is specifically configured to:

[0148] Multiply each pixel point in the image by the perspective transformation matrix to perform perspective transformation on the image.

[0149] In one implementation, the detection module includes:

[0150] A first determination unit, configured to determine at least one defect on the circular product based on multiple transformed feature points;

[0151] A second determination unit, configured to determine at least one target area where at least one defect is located;

[0152] A calculation unit, configured to calculate feature parameters of the defect in the corresponding target area within at least one target area in the target image.

[0153] A detection unit for performing qualification detection on circular products based on feature parameters and the number of defects.

[0154] In one implementation, the first determination unit is specifically configured to:

[0155] Calculate the distances between multiple transformed feature points and the circumference of the standard circle respectively;

[0156] Take the transformed feature points corresponding to the distances greater than the set value as target points;

[0157] Classify the target points, and take at least one group of classified target points as at least one defect.

[0158] In one implementation, the second determination unit is specifically configured to perform the following steps for each defect:

[0159] Expand the defect area enclosed by a group of target points corresponding to any one defect and the circumference of the standard circle;

[0160] Take the expanded defect area as a target area.

[0161] In one implementation, the calculation unit is specifically configured to:

[0162] Perform binarization processing on at least one target area in the target image;

[0163] Calculate the side length and area of the defect in the corresponding target area after binarization processing as the feature parameters of the defect.

[0164] In one implementation, the detection unit is specifically configured to:

[0165] If the feature parameters of any one defect exceed the qualified parameter range or the number of defects exceeds the quantity threshold, determine that the circular product is unqualified; otherwise, determine that the circular product is qualified.

[0166] It can be seen that this embodiment can correct the non-standard circles presented by qualified circular products in the picture due to the camera shooting angle or the inclination of the product position, achieve complete perspective correction, and perform qualification detection on circular products based on multiple transformed feature points after perspective transformation and the target image after perspective transformation, which can reduce the over-inspection rate and improve the detection efficiency and accuracy of circular products.

[0167] Next, the computer-readable storage medium provided by the embodiments of the present application will be introduced. The computer-readable storage medium described below can be correspondingly referred to the product detection method described above.

[0168] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned product detection method are implemented.

[0169] The computer-readable storage medium may include: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0170] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method part.

[0171] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this document can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0172] The steps of the methods or algorithms described in combination with the embodiments disclosed in this document can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), an internal memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium well-known in the technical field.

[0173] Finally, it should also be noted that in this document, relationships such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "including", "comprising", or any other variant are intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device.

[0174] In this text, specific examples are used to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A product detection method, characterized in that, Including: Extracting multiple product edge feature points from an image of a circular product; Fitting an ellipse based on the multiple product edge feature points, and constructing a standard circle according to the ellipse; Constructing a perspective transformation matrix based on the ellipse and the standard circle; Performing perspective transformation on the multiple product edge feature points by using the perspective transformation matrix to obtain multiple transformed feature points; Performing perspective transformation on the image by using the perspective transformation matrix to obtain a target image; Performing qualification detection on the circular product based on the multiple transformed feature points and the target image.

2. The method according to claim 1, characterized in that, Extracting multiple product edge feature points from an image of a circular product, including: After obtaining the image, performing noise reduction processing on the image; Extracting the multiple product edge feature points from the image after noise reduction processing.

3. The method according to claim 1, wherein Constructing a standard circle according to the ellipse, including: Calculating a radius value according to the major axis and minor axis of the ellipse; Constructing the standard circle with the center coordinates of the ellipse as the center and the radius value.

4. The method according to claim 1, characterized in that, Constructing a perspective transformation matrix based on the ellipse and the standard circle, including: Based on the ellipse and the standard circle, constructing a translation matrix, a rotation matrix, a shear matrix and a scaling matrix; Taking the product of the translation matrix, the rotation matrix, the shear matrix and the scaling matrix as the perspective transformation matrix.

5. The method according to claim 1, characterized in that Performing perspective transformation on the multiple product edge feature points by using the perspective transformation matrix, including: Multiplying the multiple product edge feature points by the perspective transformation matrix respectively to perform perspective transformation on the multiple product edge feature points.

6. The method according to claim 1, wherein Performing perspective transformation on the image by using the perspective transformation matrix, including: Multiplying each pixel point in the image by the perspective transformation matrix to perform perspective transformation on the image.

7. The method according to any one of claims 1 to 6, characterized in that, Performing qualification detection on the circular product based on the multiple transformed feature points and the target image, including: Determining at least one defect on the circular product based on the multiple transformed feature points; Determining at least one target area where the at least one defect is located; Calculating characteristic parameters of the defect in the corresponding target area within at least one target area in the target image; Performing qualification detection on the circular product based on the characteristic parameters and the number of defects.

8. The method according to claim 7, wherein Determining at least one defect on the circular product based on the multiple transformed feature points, including: Calculating the distances between the multiple transformed feature points and the circumference of the standard circle respectively; Taking the transformed feature points corresponding to the distances greater than a set value as target points; Classifying the target points, and taking at least one set of classified target points as at least one defect.

9. The method according to claim 8, wherein Performing the following steps for each defect: Expanding the defect area surrounded by a set of target points corresponding to any defect and the circumference of the standard circle; Taking the expanded defect area as a target area.

10. The method according to claim 7, wherein Calculating characteristic parameters of the defect in the corresponding target area within at least one target area in the target image, including: Performing binarization processing on at least one target area in the target image; Calculating the side length and area of the defect in the corresponding target area after binarization processing as the characteristic parameters of the defect.

11. The method according to claim 7, wherein Perform qualification detection on the circular product based on the characteristic parameters and the number of defects, including: If the characteristic parameters of any defect exceed the qualified parameter range or the number of defects exceeds the quantity threshold, determine that the circular product is unqualified; otherwise, determine that the circular product is qualified.

12. A product detection device, characterized in that, Including: An extraction module, configured to extract multiple product edge feature points from the image of the circular product; A fitting module, configured to fit an ellipse based on the multiple product edge feature points and construct a standard circle according to the ellipse; A construction module, configured to construct a perspective transformation matrix based on the ellipse and the standard circle; A transformation module, configured to perform perspective transformation on the multiple product edge feature points by using the perspective transformation matrix to obtain multiple transformed feature points; perform perspective transformation on the image by using the perspective transformation matrix to obtain a target image; A detection module, configured to perform qualification detection on the circular product based on the multiple transformed feature points and the target image.

13. An electronic device, characterized in that, Including a memory and a processor, wherein: The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, For storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images