A method for recognizing surface state of a precision stamping die based on machine vision

Abstract

The present invention discloses a method for identifying the surface state of a precision stamping die based on machine vision, which relates to the technical field of image recognition. A method for identifying the surface state of a precision stamping die based on machine vision includes collecting pictures of the upper and lower surfaces of the die before stamping and establishing a standard data set, setting the region of interest, collecting pictures of the upper and lower surfaces of the die during the stamping process in real time, matching the region of interest, generating real-time comparison picture data, comparing the real-time comparison picture data with the picture data of the standard data set and generating a comparison value, setting a threshold, and after setting, judging the difference between the comparison value and the set threshold in real time, judging the surface state of the stamping die, and updating the standard data set. The present invention reduces the dependence on manual labor in stamping processing, reduces the labor intensity of workers, can judge the surface state of the die before the die presses the material, is not prone to the situation of die damage, and improves the speed and accuracy of judging the surface state of the die.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method for recognizing the surface condition of precision stamping dies based on machine vision. Background Technology

[0002] High-performance precision stamped parts with free and irregular contours have become one of the core components that are indispensable in the industrial field. During precision stamping, due to friction on the die surface and lubricating oil, parts or scrap may remain on the lower surface of the die, which will cause stress concentration when stamping metal raw materials next time, resulting in die damage and seriously affecting processing efficiency and product quality.

[0003] Currently, foreign matter residue on the mold surface is mainly avoided through two methods: manual inspection and sensor inspection. Manual inspection involves analyzing and judging the state of the processing area by human eyes during the feeding process and manually removing residual material. Sensor inspection mainly involves measuring the changes in hydraulic cylinder pressure and the displacement of the fine blanking slide to determine the stamping state.

[0004] However, manual inspection increases the workload of workers and has lower judgment speed and accuracy. Sensor detection sometimes has a lag and cannot make judgments before the mold presses on the material, which can easily lead to mold damage. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based method for recognizing the surface condition of precision stamping dies, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for surface condition recognition of precision stamping dies based on machine vision, comprising the following steps:

[0007] Step 1: Collect images of the upper and lower surfaces of the die before stamping and establish a standard dataset;

[0008] Step 2, set the region of interest;

[0009] Step 3: Collect images of the upper and lower surfaces of the die during the stamping process in real time, match the region of interest, and generate real-time comparison image data;

[0010] Step 4: Compare the real-time comparison image data with the standard dataset image data and generate comparison values;

[0011] Step 5: Set a threshold value. After setting the threshold value, compare the difference between the comparison value and the set threshold value in real time to determine the surface condition of the stamping die.

[0012] Step 6: Update the standard dataset.

[0013] Preferably, both steps 1 and 3 use an industrial camera to acquire images including the upper and lower surfaces of the mold.

[0014] Preferably, step 2 further includes manually removing background interference.

[0015] Preferably, step 3 expands the selection of image data for real-time comparison based on the range of the region of interest.

[0016] Preferably, the specific process of step 4 is as follows:

[0017] Step 4.1: Use the pyrDown() function in OpenCV to compress the acquired images and templates. The number of compression layers can be adjusted according to the pixels of the acquired images.

[0018] Step 4.2: Calculate the normalized cross-correlation matching coefficients between each point within the compressed real-time comparison image data range that can contain the size of the region of interest and the compressed standard dataset image, and record the location where the maximum value occurs;

[0019] Normalized cross-correlation matching coefficient

[0020]

[0021] The standard dataset contains g images of size m×n, while the real-time comparison images contain S images of size M×N. Since the standard dataset images are smaller than the real-time comparison images, we use S. x,y This represents a sub-block in S with (x,y) as its top-left corner and the same size as g;

[0022] Based on the location where the maximum value occurs, calculate the normalized cross-correlation matching coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image near that location, and record the maximum value as the current normalized cross-correlation matching comparison value.

[0023] Step 4.3: Calculate the structural similarity coefficient between each point within the compressed real-time comparison image data range that can contain the size of the region of interest and the compressed standard dataset image, and record the location where the maximum value occurs;

[0024] Structural similarity coefficient

[0025] SSIM(x, y) = [l(x, y)] α [c(x, y)] β [s(x, y)] γ

[0026]

[0027]

[0028]

[0029] l(x,y) is the brightness contrast function, c(x,y) is the contrast function, and s(x,y) is the structure contrast function; α, β, γ > 0, used to adjust the importance of these three modules;

[0030] C1=(k1L) 2 C2 = (k2L) 2 C3 = (k3L) 2 This is a coefficient used to maintain algorithm stability; L is the dynamic range of pixel values.

[0031] Based on the location where the maximum value occurs, the structural similarity coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image is calculated near that location, and the maximum value is recorded as the structural similarity matching comparison value.

[0032] Preferably, the specific process of step 5 is as follows: place common minimum-sized foreign objects on the mold, record the normalized cross-correlation matching comparison value and structural similarity matching comparison value after placing the foreign objects, and set a threshold based on the comparison value;

[0033] After setting, the difference between the comparison value and the set threshold is judged in real time. When the comparison value exceeds the set threshold, it is determined that there are foreign objects on the surface of the stamping die.

[0034] Preferably, the specific process of step 6 is as follows: set the dataset update frequency. When the number of stampings reaches the set number or the set time, add the stamping die images that are collected in real time and judged to be free of foreign objects on the surface of the stamping die to the standard dataset, and update the standard dataset. When the update of the standard dataset exceeds the set number of datasets, delete the earliest image data in the current dataset.

[0035] In actual processing, the system determines whether to continue stamping based on the condition of the die surface before stamping.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This machine vision-based method for recognizing the surface condition of precision stamping dies solves the problem of needing manual supervision of dies, reduces the dependence of stamping processing on manual labor, reduces the labor intensity of workers, and enables the determination of the surface condition of the die before it presses onto the material, which reduces the risk of die damage and improves the speed and accuracy of die surface condition determination. Attached Figure Description

[0038] Figure 1 A schematic diagram of the standard dataset for the upper and lower surfaces of a structural mold;

[0039] Figure 2This is a schematic diagram illustrating real-time image data comparison after setting the region of interest.

[0040] Figure 3 Real-time comparison image data for when the region of interest is set and there are foreign objects on the lower surface of the mold;

[0041] Figure 4 This is a flowchart of the machine vision-based method for recognizing the surface condition of precision stamping dies according to the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] It should be noted that in the description of this invention, the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0044] Furthermore, it should be understood that, for ease of description, the dimensions of the various components shown in the accompanying drawings are not drawn to actual scale; for example, the thickness or width of some layers may be exaggerated relative to other layers.

[0045] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined or described in one figure, it will not need to be discussed or described in detail in the description of the subsequent figures.

[0046] like Figure 4 As shown, the present invention provides a technical solution: a method for surface condition recognition of precision stamping dies based on machine vision, comprising the following steps:

[0047] Step 1: Collect images of the upper and lower surfaces of the die before stamping and establish a standard dataset, such as... Figure 1 As shown, the specific process is as follows:

[0048] An industrial camera was used to acquire images of the upper and lower surfaces of the mold, and a standard dataset was established.

[0049] Step 2, set the region of interest, the specific process is as follows:

[0050] The upper and lower surfaces of the mold are manually selected as regions of interest to eliminate background interference.

[0051] Step 3: Real-time acquisition of images of the upper and lower surfaces of the die during the stamping process, matching regions of interest, and generating real-time comparison image data, such as... Figure 2 As shown, this is a real-time comparison of image data of the mold surface under normal conditions. Figure 3 The image shown is a real-time comparison image of a foreign object on the mold surface. Because the mold inevitably vibrates during the stamping process, the real-time acquired images will have misalignment errors with the standard dataset images, so correction is required. The specific process is as follows:

[0052] Industrial cameras are used to capture real-time images of the upper and lower surfaces of the mold, and the range of the area of ​​interest is expanded to select images for real-time comparison.

[0053] Step 4: Compare the real-time comparison image data with the standard dataset image data and generate comparison values. The specific process is as follows:

[0054] Step 4.1: Use the pyrDown() function in OpenCV to compress the acquired images and templates. The number of compression layers can be adjusted according to the pixels of the acquired images.

[0055] The pyrDown() function filters an image and then samples it (downsampling is achieved by discarding even-numbered rows and columns).

[0056] Step 4.2: Calculate the normalized cross-correlation matching coefficients between each point within the compressed real-time comparison image data range that contains the region of interest and the compressed standard dataset image, and record the location of the maximum value. It should be noted that because the image resolution is low after compression, using the compressed real-time comparison image data and the compressed standard dataset image to calculate the normalized matching coefficients allows the system to quickly find the location of the maximum value (i.e., the accurate location of the compressed real-time comparison image). It should be understood that "system" refers to the entire comparison method (i.e., the method described in this patent), which in actual use is a software program.

[0057] Normalized cross-correlation matching coefficient

[0058]

[0059] The standard dataset images are g, with a size of m×n, and the real-time comparison images are S, with a size of M×N. The standard dataset images are smaller than the real-time comparison images. A sub-block of size m×n is randomly selected from the real-time comparison images, and its top-left corner is located at coordinates (x, y) in image S, denoted as S0. (x,y)That is, in S, there is a sub-block with the same size as g, whose top-left corner is (x,y). It should be noted that the x-coordinate range is 0≤x≤Mm, and the y-coordinate range is 0≤y≤Nn. M and N are the number of rows and columns of the real-time comparison image, and m and n are the number of rows and columns of the standard dataset image. It should be known that (i,j) is the coordinate of the sub-block in the standard dataset image.

[0060] Based on the location of the maximum value, the normalized cross-correlation matching coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image is calculated in the vicinity of that location, with a range of [-1, 1]. The maximum value is recorded as the current normalized cross-correlation matching comparison value. It should be noted that since compressed images have low resolution and uncompressed images have high resolution, the maximum value of the normalized cross-correlation matching coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image should be used as the comparison value, as this comparison value is more accurate (the larger the normalized cross-correlation matching coefficient, the higher the similarity between the standard dataset image and the real-time comparison image).

[0061] Step 4.3: Calculate the structural similarity coefficient between each point within the compressed real-time comparison image data range that can contain the size of the region of interest and the compressed standard dataset image. The range of the structural similarity coefficient is [0,1], and record the position where the maximum value occurs.

[0062] Structural similarity coefficient

[0063] SSIM(x, y) = [l(x, y)] α [c(x, y)] β [s(x, y)] γ

[0064]

[0065]

[0066]

[0067] l(x,y) is the brightness contrast function, c(x,y) is the contrast function, and s(x,y) is the structure contrast function; α, β, γ>0 are used to adjust the importance of these three modules, and α=1, β=2, and γ=3 are taken; it should be noted that the greater the influence of the brightness contrast function, the smaller the value of α.

[0068] C1=(k1L) 2 C2 = (k2L) 2 C3 = (k3L) 2 These are coefficients used to maintain algorithm stability. L is the dynamic range of pixel values, taken as L = 255, k1 = 0.01, k2 = 0.02, k3 = 0.03; where μx μ is the average value of x. y σ is the average value of y. x σ is the variance of x. y σ is the variance of y. xy It is the covariance of x and y.

[0069] Based on the location where the maximum value occurs, the structural similarity coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image is calculated near that location (the larger the structural similarity coefficient, the higher the similarity between the standard dataset image and the real-time comparison image), and the maximum value is recorded as the structural similarity matching comparison value.

[0070] Examples based on the above methods are provided below:

[0071] The standard dataset image is g, with a size of m×n, and the real-time comparison image is S, with a size of M×N. The standard dataset image (g) and the real-time comparison image (S) are compressed. The standard dataset image (g') is compressed to a size of (m / z)×(n / z), and the real-time comparison image (S') is compressed to a size of (M / z)×(N / z). The compression rate is z. The normalized cross-correlation matching coefficient and structural similarity coefficient of the compressed real-time comparison image (S') and the compressed standard dataset image (g') are calculated. The maximum values ​​are selected and recorded as the normalized cross-correlation matching comparison value and the structural similarity matching comparison value of the compressed image, and their positions are recorded as (Zx, Zy).

[0072] Then, the compressed image positions are converted to the corresponding positions (Zx*z, Zy*z) of the uncompressed image, denoted as (Px, Py).

[0073] (Px-z, Py-z) (Px-z+1, Py-z) (Px-z+2, Py-z) … (Px+z, Py-z) (Px-z, Py-z+1) (Px-z+1, Py-z+1) (Px-z+2, Py-z+1) … (Px+z, Py-z+1) (Px-z, Py-z+2) (Px-z+1, Py-z+2) (Px-z+2, Py-z+2) … (Px+z, Py-z+2) … … … … … (Px-z, Py+z) (Px-z+1, Py+z) (Px-z+2, Py+z) … (Px+z, Py+z)

[0074] Within the range (Px-z, Py-z) to (Px+z, Py+z) (x and y are traversed separately), the normalized cross-correlation matching coefficient and structural similarity coefficient of the uncompressed real-time comparison image and the uncompressed standard set image are calculated, and the maximum value is selected and recorded as the normalized cross-correlation matching comparison value and the structural similarity matching comparison value.

[0075] Step 5: Set a threshold value. After setting the threshold, compare the difference between the comparison value and the set threshold value in real time to determine the surface condition of the stamping die. The specific process is as follows:

[0076] Place common minimum-sized foreign objects on the mold, record the normalized cross-correlation matching comparison value and structural similarity matching comparison value after placing the foreign objects, and set a threshold based on the comparison value;

[0077] After setting, the difference between the comparison value and the set threshold is judged in real time. When the comparison value exceeds the set threshold, it is determined that there are foreign objects on the surface of the stamping die.

[0078] Specifically, common minimum-sized foreign objects are placed on the mold. The normalized cross-correlation matching comparison value 'a' and the structural similarity matching comparison value 'b' are recorded after the foreign object is placed. Therefore, the normalized cross-correlation threshold is manually set to 'a+5%' (5% is not a fixed value; staff can change the percentage based on the actual situation), and the structural similarity threshold is set to 'b+5%' (5% is not a fixed value; staff can change the percentage based on the actual situation). When the normalized cross-correlation comparison value of the real-time comparison is greater than 'a+5%' and the structural similarity threshold is greater than 'b+5%', it indicates that the foreign object does not exist; otherwise, it is considered that there is a foreign object on the mold surface.

[0079] Step 6, update the standard dataset, the specific process is as follows:

[0080] Set the dataset update frequency. When the number of stampings reaches the set number or the set time, add the real-time collected images of the die before stamping, which are judged to be free of foreign objects on the die surface, to the standard dataset and update the standard dataset. When the update of the standard dataset exceeds the set number of datasets, delete the oldest image data in the current dataset. During actual processing, decide whether to continue stamping based on the state of the die surface before stamping as judged by the system.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for surface condition recognition of precision stamping dies based on machine vision, characterized in that: Includes the following steps: Step 1: Collect images of the upper and lower surfaces of the die before stamping and establish a standard dataset; Step 2, set the region of interest; Step 3: Collect images of the upper and lower surfaces of the die during the stamping process in real time, match the region of interest, and generate real-time comparison image data; Step 4: Compare the real-time comparison image data with the standard dataset image data and generate comparison values; Step 5: Set a threshold value. After setting the threshold value, compare the difference between the comparison value and the set threshold value in real time to determine the surface condition of the stamping die. Step 6, update the standard dataset; The specific process of step 4 is as follows: Step 4.1: Use the pyrDown() function in OpenCV to compress the acquired images and templates. The number of compression layers can be adjusted according to the pixels of the acquired images. Step 4.2: Calculate the normalized cross-correlation matching coefficients between each point within the compressed real-time comparison image data range that can contain the size of the region of interest and the compressed standard dataset image, and record the location where the maximum value occurs; because the resolution of the compressed image is low, using the compressed real-time comparison image data and the compressed standard dataset image to calculate the normalized matching coefficients can enable the system to quickly find the location where the maximum value occurs. Normalized cross-correlation matching coefficient ; The standard dataset picture is g, with a size of m x n, and the real-time comparison picture image is S, with a size of M x N. The standard dataset picture size is smaller than the real-time comparison picture image. S x,y represents a sub-block in S with the same size as g, with a left upper corner point of (x, y). Based on the location of the maximum value, the normalized cross-correlation matching coefficients of the uncompressed real-time comparison image and the uncompressed standard dataset image are calculated near that location, and the maximum value is recorded as the current normalized cross-correlation matching comparison value. Since compressed images have low resolution and uncompressed images have high resolution, the maximum value of the normalized cross-correlation matching coefficients of the uncompressed real-time comparison image and the uncompressed standard dataset image should be used as the comparison value. The comparison value is more accurate, and the larger the normalized cross-correlation matching coefficient, the higher the similarity between the standard dataset image and the real-time comparison image. Step 4.3: Calculate the structural similarity coefficient between each point within the compressed real-time comparison image data range that can contain the size of the region of interest and the compressed standard dataset image, and record the location where the maximum value occurs; Structural similarity coefficient ; ; ; ; l(x,y) is the brightness contrast function, c(x,y) is the contrast function, and s(x,y) is the structure contrast function; α, β, γ > 0, used to adjust the importance of these three modules; C1=(k1L) 2 C2=(k2L) 2 C3 = (k3L) 2 The coefficients are used to maintain algorithm stability, where L is the dynamic range of pixel values; and μ is the value of μ. x μ is the average value of x. y σ is the average value of y. x σ is the variance of x. y σ is the variance of y. xy It is the covariance of x and y; Based on the location where the maximum value occurs, the structural similarity coefficient between the uncompressed real-time comparison image and the uncompressed standard dataset image is calculated near that location, and the maximum value is recorded as the structural similarity matching comparison value.

2. The method for surface condition recognition of precision stamping dies based on machine vision according to claim 1, characterized in that: Both steps 1 and 3 involve using an industrial camera to capture images of the upper and lower surfaces of the mold.

3. The method for surface condition recognition of precision stamping dies based on machine vision according to claim 1, characterized in that: Step 2 also includes manually removing background interference.

4. The method for surface condition recognition of precision stamping dies based on machine vision according to claim 1, characterized in that: Step 3 expands the selection of image data for real-time comparison based on the range of the region of interest.

5. The method for surface condition recognition of precision stamping dies based on machine vision according to claim 1, characterized in that: The specific process of step 5 is as follows: place common minimum-sized foreign objects on the mold, record the normalized cross-correlation matching comparison value and structural similarity matching comparison value after placing the foreign objects, and set a threshold based on the comparison value; After setting, the difference between the comparison value and the set threshold is judged in real time. When the comparison value exceeds the set threshold, it is determined that there are foreign objects on the surface of the stamping die.

6. The method for surface condition recognition of precision stamping dies based on machine vision according to claim 1, characterized in that: The specific process of step 6 is as follows: Set the dataset update frequency. When the number of stampings reaches the set number or the set time, add the stamping mold images that are collected in real time and judged to be free of foreign objects on the surface of the stamping mold to the standard dataset, update the standard dataset, and delete the earliest image data in the current dataset when the update of the standard dataset exceeds the set number of datasets. In actual processing, the system determines whether to continue stamping based on the condition of the die surface before stamping.

Images