Die cavity anomaly detection method based on machine vision

Abstract

The invention relates to the field of mold detection, in particular to a mold cavity anomaly detection method based on machine vision. The method comprises the steps of obtaining a preprocessed mold cavity surface grey-scale map, inputting the mold cavity surface grey-scale map into a preset neural network, outputting a probability value, and obtaining a mold cavity type according to the probability value; constructing a target grey-scale map according to the type of the die cavity, extracting an ROI region of the target grey-scale map, carrying out the grid division of the ROI region according to the probability value, obtaining a plurality of sub-regions, calculating the gradient consistency of the ROI region, and completing the abnormality detection of the die cavity. According to the technical scheme, the accuracy of abnormal detection of the die cavity can be improved, and the production quality of products is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of mold detection, and more specifically, to a method for detecting mold cavity abnormalities based on machine vision. Background Art

[0002] In the manufacturing industry, especially in the plastic, rubber, metal processing and other industries, mold cavity filling is a crucial link. The mixing of foreign matter may not only cause defects on the product surface and structural damage, but also cause more serious safety problems, thus affecting the safety and reliability of the product. Therefore, the detection of foreign matter in the mold cavity has become a key step to ensure product quality. Traditional detection methods mainly rely on manual visual inspection and some basic automation equipment. These methods are not only inefficient and have limited accuracy, but also have poor adaptability and are difficult to meet the needs of modern manufacturing for high efficiency and high accuracy.

[0003] In terms of mold cavity anomaly detection, the application of machine vision inspection technology is also increasing. This technology can detect foreign matter in mold parts, part defects, poor part demolding, and insert misalignment. It has the characteristics of high accuracy, high efficiency and the ability to achieve real-time detection.

[0004] The existing Chinese patent application document with publication number CN108288274A discloses a mold detection method, device and electronic device, the method comprising: matching a background image; transforming, normalizing and cropping the mold image to be detected and the matching background image based on a matching matrix of the mold image to be detected and the matching background image, respectively, to obtain a first mold image and a first background image; obtaining a first differential image between the first mold image and the first background image, and performing threshold segmentation and area screening of connected regions on the first differential image to obtain a first region image; determining whether the pixel values ​​of the pixels in the first region image are all 0; and outputting information on the presence of foreign matter in the mold.

[0005] However, due to the material problem of the mold and the unevenness of the mold cavity, reflections will occur during the image processing process, and the reflections will cover some abnormalities, resulting in inaccurate mold cavity abnormality detection results. Summary of the invention

[0006] In order to solve the problem of inaccurate mold cavity anomaly detection results, the present invention proposes a mold cavity anomaly detection method based on machine vision.

[0007] The present invention discloses a method for detecting mold cavity anomalies based on machine vision, comprising: obtaining a preprocessed grayscale image of the mold cavity surface, inputting the grayscale image of the mold cavity surface into a preset neural network, outputting a probability value, the probability value including a normal probability, a residual probability, a defect probability and a reflection probability, and obtaining a mold cavity type according to the probability value; constructing a target grayscale image according to the mold cavity type, extracting a ROI area of ​​the target grayscale image, meshing the ROI area according to the probability value to obtain a plurality of sub-areas, calculating the gradient consistency of the ROI area, and completing mold cavity anomaly detection; wherein obtaining a plurality of sub-areas comprises: obtaining an initial mesh size of meshing; calculating an updated mesh size, the mesh size including a mesh length and a mesh width, and the updated mesh size satisfies the relationship: , represents the updated grid size, represents the probability of reflection, Represents the initial grid size; the ROI area is divided into multiple sub-areas according to the updated grid size.

[0008] By preprocessing the grayscale image of the mold cavity surface and using a neural network to output different probability values, the mold cavity type can be accurately identified and a target grayscale image can be constructed. These grayscale images are images with a high probability of reflection, that is, images with a high probability of containing anomalies. By extracting the ROI area of ​​the target grayscale image and dynamically adjusting the grid size according to the reflection probability, the ROI area can be divided more finely, thereby improving the consistency and accuracy of detection.

[0009] Preferably, the neural network is a convolutional neural network, which extracts features from a grayscale image of the cavity surface to obtain image features, and classifies the image features to obtain probability values.

[0010] Preferably, the training process of the neural network includes: taking a grayscale image of the cavity surface of the mold in history as input information, and taking the true value of the cavity type of the mold as a label to obtain a set of training data; inputting the training data into the neural network to obtain an output result; calculating the loss value of the neural network through the output result and the label, and the loss function uses cross entropy loss, back-propagating the error signal according to the loss value, updating the network parameters of the neural network to reduce the loss value; iteratively updating the network parameters of the neural network, and when the neural network reaches the set maximum number of training times or the loss value is less than the set loss value, stopping the update to obtain a trained neural network.

[0011] Preferably, the target grayscale image is obtained including: the cavity types are normal cavity, residual cavity, defective cavity and reflective cavity; when any one of the probability values ​​is greater than 0.5, the corresponding cavity type is obtained; the cavity surface grayscale image corresponding to the mold with cavity reflection and the cavity surface grayscale image corresponding to the mold with probability values ​​not greater than 0.5 are taken together as the target grayscale image.

[0012] The probability value output by the neural network can be used to obtain the specific mold cavity type, and the mold cavity with mold reflectivity can be found more quickly from the already determined mold cavity types. The mold cavity that cannot be determined by the probability value can be used as the target grayscale image, and the mold cavity that needs special attention or processing can be quickly identified, so that the production parameters can be adjusted or maintenance can be carried out in time to reduce the generation of defective products.

[0013] Preferably, the method of obtaining the target grayscale image also includes: for any mold cavity surface grayscale image, respectively calculating the Euclidean distance between the normal probability and the residual probability, the Euclidean distance between the normal probability and the defect probability, and the Euclidean distance between the normal probability and the reflection probability, adding the three Euclidean distances and mapping the result through negative correlation as the degree of difficulty in classification; taking the mold cavity surface grayscale image whose degree of difficulty in classification is greater than a preset threshold as the target grayscale image.

[0014] By adding these distances to obtain the degree of difficulty in classification, the degree of difference between the mold cavity image and the center point of each category can be quantified. It reflects the degree of deviation between the mold cavity image features and the typical features of each category. The larger the calculation result of the Euclidean distance, the greater the difference between the features of the mold cavity image and the features of the normal state, which usually means that there is some abnormality in the mold cavity.

[0015] Preferably, the gradient consistency includes: for any sub-region, weighted summing of the gradient directions according to the gradient value of each pixel in the sub-region to obtain the total gradient direction of the sub-region; traversing to obtain the total gradient direction of each sub-region in the ROI region, constructing the gradient co-occurrence matrix of the ROI region, and taking the modulus of the gradient co-occurrence matrix as the gradient consistency.

[0016] Gradient consistency effectively quantifies the uniformity of the cavity surface. Gradient consistency is achieved by analyzing the gradient value of each pixel in the sub-region and performing a weighted summation of the gradient direction, thereby being able to capture subtle changes in the cavity surface and improve the sensitivity and accuracy of detection. In addition, by constructing a gradient co-occurrence matrix and calculating its modulus as a measure of gradient consistency, the ability to identify abnormalities on the cavity surface is further enhanced.

[0017] Preferably, the gradient consistency further includes: for any sub-region, calculating the gradient direction of each pixel in the sub-region, drawing a gradient histogram to obtain the main gradient direction and the sub-gradient direction of the sub-region; constructing the main gradient co-occurrence matrix and the sub-gradient co-occurrence matrix of the ROI region respectively; calculating the main gradient eigenvector according to the main gradient co-occurrence matrix, and calculating the sub-gradient eigenvector according to the sub-gradient co-occurrence matrix; calculating the gradient consistency to satisfy the relationship: ,in, represents gradient consistency, represents the main gradient eigenvector, represents the transpose of the subgradient eigenvector, represents the magnitude of the main gradient eigenvector, Represents the magnitude of the subgradient eigenvector.

[0018] The directional information of local changes in the image is used, which reflects the microstructure and potential abnormal features of the cavity surface. The main gradient direction usually corresponds to the direction with the most significant changes in the image, that is, the main texture direction in the sub-region of the cavity, while the secondary gradient direction provides complementary information about the direction of change, that is, the less important texture direction in the sub-region of the cavity.

[0019] Preferably, the gradient consistency also includes: for any sub-region, calculating the gradient direction of each pixel in the sub-region, drawing a gradient histogram to obtain the main gradient direction and the sub-gradient direction of the sub-region; constructing the main gradient co-occurrence matrix and the sub-gradient co-occurrence matrix of the ROI region respectively; calculating the main gradient eigenvector according to the main gradient co-occurrence matrix, and calculating the sub-gradient eigenvector according to the sub-gradient co-occurrence matrix; and taking the similarity between the main gradient eigenvector and the sub-gradient eigenvector as the gradient consistency.

[0020] Beneficial effects of the present invention: 1. The present invention pre-processes the grayscale image of the mold cavity surface and inputs it into a convolutional neural network. The network can output probability values ​​reflecting different states. These probability values ​​comprehensively reflect the current state of the mold cavity. This method can automatically extract features from image data and classify them, thereby reducing manual intervention and improving detection efficiency and accuracy.

[0021] 2. The present invention significantly improves the accuracy and reliability of mold cavity anomaly detection by comprehensively utilizing gradient direction information, gradient consistency calculation, and grid size adjusted by reflection probability. This method can not only identify small changes in the mold cavity surface, but also adapt to different detection conditions, such as the influence of reflection and shadow, thus providing a powerful tool for mold cavity quality control.

[0022] 3. The present invention significantly improves the accuracy and reliability of mold cavity anomaly detection through automated feature extraction and quantitative anomaly detection. It can not only identify small changes in the mold cavity surface, but also adapt to different detection conditions, such as the influence of reflections and shadows. By timely detecting abnormalities on the mold cavity surface, potential quality problems can be avoided, product consistency and reliability can be improved, production costs can be reduced, and production efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 It is a flow chart of a method for detecting mold cavity anomalies based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0026] The present invention provides a method for detecting mold cavity abnormalities based on machine vision. Figure 1 As shown, the mold cavity abnormality detection method based on machine vision includes steps S1 and S2, which are described in detail below.

[0027] S1, obtaining a preprocessed grayscale image of the mold cavity surface, inputting the grayscale image of the mold cavity surface into a preset neural network, outputting a probability value, and obtaining the mold cavity type according to the probability value.

[0028] In one embodiment, an industrial camera is used to capture an RGB (Red Green Blue) image of a mold cavity, and the RGB image of the mold cavity is grayed to obtain a grayscale image. The grayscale image can simplify image data, reduce computational complexity, and retain sufficient information for subsequent feature extraction and analysis.

[0029] Preprocessing includes: first, converting the RGB image into a grayscale image through grayscale processing, which can reduce the complexity and computational complexity of image processing. Then, histogram equalization technology can be used to enhance the contrast of the image, highlight the details, and make the texture and structure of the cavity surface clearer. In addition, low-pass filtering methods such as mean filtering, median filtering and Gaussian filtering can be used to remove image noise, smooth image edges, and reduce the high-frequency part of image details, thereby suppressing noise and bright spots.

[0030] Morphological operations, such as erosion, dilation, opening, and closing, can further improve the image quality, remove small noise points or impulse noise, and enhance the edge information of objects in the image. These operations help to more accurately identify and locate abnormalities on the cavity surface in subsequent steps.

[0031] The grayscale image of the mold cavity surface is input into a preset neural network, wherein the neural network is a convolutional neural network. The convolutional neural network extracts features of the grayscale image of the mold cavity surface to obtain image features, and classifies the image features to obtain probability values.

[0032] It should be noted that convolutional neural networks are CNNs (Convolutional Neural Networks), which include input layers, convolution layers, activation layers, pooling layers, and fully connected layers (or output layers). The input layer is used to receive the grayscale image of the cavity surface; the convolution layer is a layer containing multiple convolution operations, each of which uses a different convolution kernel (or filter) to extract the feature map of the cavity surface grayscale image; the activation layer usually follows the convolution layer, introducing nonlinearity so that the neural network can learn complex features; the pooling layer is used to reduce the spatial size of the feature map, reduce the amount of calculation, and extract important features; the fully connected layer is after the convolution layer and the pooling layer, mapping the extracted features to the final output, that is, the probability value.

[0033] The training process of the neural network includes: taking the grayscale image of the cavity surface of the mold in history as input information, and taking the true value of the cavity type of the mold as a label to obtain a set of training data; inputting the training data into the neural network to obtain the output result; calculating the loss value of the neural network through the output result and the label, and the loss function uses the cross entropy loss, back-propagating the error signal according to the loss value, updating the network parameters of the neural network to make the loss value smaller; iteratively updating the network parameters of the neural network, and when the neural network reaches the set maximum number of training times or the loss value is less than the set loss value, stop updating to obtain a trained neural network.

[0034] Exemplarily, when the neural network training times reaches 200 or the loss value is less than 0.0001, the update is stopped.

[0035] The output of the neural network is a probability value. When any of the probability values ​​is greater than the probability threshold of the network, the cavity corresponding to the grayscale image of the cavity surface is the corresponding cavity type. The cavity types are normal cavity, residual cavity, defective cavity and reflective cavity.

[0036] For example, if the input normal probability, residual probability, defect probability and reflection probability are 0.3, 0.6, 0.05 and 0.05 respectively, and the probability threshold of the network is generally set to 0.5, then the cavity corresponding to the grayscale image of the cavity surface is the cavity residue.

[0037] S2, constructs a target grayscale image according to the cavity type, extracts the ROI area of ​​the target grayscale image, meshes the ROI area according to the probability value to obtain multiple sub-areas, calculates the gradient consistency of the ROI area, and completes the cavity anomaly detection.

[0038] It should be noted that in the mold manufacturing process, image anomalies caused by reflections on the mold cavity surface are often encountered. Such reflections are usually just pseudo-abnormalities caused by light reflection and will not have a substantial impact on product quality. However, if other types of abnormalities, such as residues and defects, happen to be hidden in these reflective areas, these original reflective areas will become real abnormal areas. At this time, a high-precision detection system is needed to distinguish and identify these real abnormalities mixed in the reflections to ensure that the quality of the mold cavity products is not damaged.

[0039] In one embodiment, the mold cavity type obtained according to step S1 further includes a case where all probability values ​​are not greater than a probability threshold of the network.

[0040] The cavity surface grayscale image corresponding to the mold with cavity reflection and the cavity surface grayscale image corresponding to the mold with probability values ​​not greater than 0.5 are taken as the target grayscale image.

[0041] It should be noted that when the probability values ​​are all no greater than 0.5, it indicates that the residual probability and defect probability are close to the reflection probability, and the grayscale image of the cavity surface is difficult to classify. In this case, further analysis of the grayscale image of the cavity surface is required.

[0042] Extracting the ROI area of ​​the target grayscale image includes: if the position of the ROI in the image is known, it can be directly extracted using a rectangular box; for irregular ROI areas, masks can be used to extract them; for ROIs of complex shapes, contour detection can be used to obtain them; by converting the image to the HSV color space, the inRange function can be used to create a mask to extract the ROI area of ​​a specific color.

[0043] Get the initial grid size of the grid division; calculate the updated grid size, which includes the length and width of the grid. The updated grid size satisfies the relationship: , represents the updated grid size, represents the probability of reflection, Indicates the initial grid size.

[0044] The ROI region is divided according to the updated grid size to obtain multiple sub-regions.

[0045] It should be noted that the main research object is the grayscale image of the cavity surface with a high probability of reflection. The higher the probability of reflection, the greater the possibility of containing other anomalies, and the smaller the grid size required to divide the ROI area into grids.

[0046] For any sub-region, the gradient direction is weightedly summed according to the gradient value of each pixel in the sub-region to obtain the total gradient direction of the sub-region.

[0047] The total gradient direction of each sub-region in the ROI region is obtained by traversing, the gradient co-occurrence matrix of the ROI region is constructed, and the modulus of the gradient co-occurrence matrix is ​​used as the gradient consistency.

[0048] It should be noted that on the surface of the mold cavity, foreign matter or defects often break the original gradient consistency, causing significant changes in the gradient direction and modulus value of the local area. High gradient consistency means that the grayscale change direction between pixels in the area is relatively consistent, which usually indicates that the edge and texture features of the area are clearer. In mold cavity inspection, clear edges and textures help to identify foreign matter or defects more accurately. And gradient consistency can also reflect the flatness of the mold cavity surface. If the gradient consistency of an area is low, it may mean that the surface of the area is uneven or has bumps and bumps, which may be caused by manufacturing defects in the mold cavity itself or wear during use.

[0049] Compare the gradient consistency with the preset gradient threshold. When the gradient consistency is less than the preset gradient threshold, generate and send an alarm to remind the staff to inspect the mold cavity.

[0050] In one embodiment, obtaining the target grayscale image also includes: for any mold cavity surface grayscale image, respectively calculating the Euclidean distance between the normal probability and the residual probability, the Euclidean distance between the normal probability and the defect probability, and the Euclidean distance between the normal probability and the reflection probability, adding the three Euclidean distances and mapping the result through negative correlation as the degree of difficulty in classification; taking the mold cavity surface grayscale image whose degree of difficulty in classification is greater than a preset threshold as the target grayscale image.

[0051] In one embodiment, gradient consistency also includes: for any sub-region, calculating the gradient direction of each pixel in the sub-region, drawing a gradient histogram to obtain the main gradient direction and sub-gradient direction of the sub-region; constructing the main gradient co-occurrence matrix and sub-gradient co-occurrence matrix of the ROI region respectively; calculating the main gradient eigenvector according to the main gradient co-occurrence matrix, and calculating the sub-gradient eigenvector according to the sub-gradient co-occurrence matrix.

[0052] Calculate the gradient consistency and satisfy the relationship: ,in, represents gradient consistency, represents the main gradient eigenvector, represents the transpose of the subgradient eigenvector, represents the magnitude of the main gradient eigenvector, Represents the magnitude of the subgradient eigenvector.

[0053] In one embodiment, gradient consistency also includes: for any sub-region, calculating the gradient direction of each pixel in the sub-region, drawing a gradient histogram to obtain the main gradient direction and sub-gradient direction of the sub-region; constructing the main gradient co-occurrence matrix and sub-gradient co-occurrence matrix of the ROI region respectively; calculating the main gradient eigenvector according to the main gradient co-occurrence matrix, and calculating the sub-gradient eigenvector according to the sub-gradient co-occurrence matrix; and taking the similarity between the main gradient eigenvector and the sub-gradient eigenvector as the gradient consistency.

[0054] Preferably, the similarity may use cosine similarity.

[0055] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0056] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting mold cavity abnormalities based on machine vision, characterized in that: include: Obtain a preprocessed grayscale image of the mold cavity surface, input the grayscale image of the mold cavity surface into a preset neural network, output a probability value, the probability value includes a normal probability, a residual probability, a defect probability and a reflection probability, and obtain the mold cavity type according to the probability value; Construct a target grayscale image according to the mold cavity type, extract the ROI area of ​​the target grayscale image, divide the ROI area into multiple sub-areas according to the probability value, calculate the gradient consistency of the ROI area, and complete the mold cavity abnormality detection; Among them, obtaining multiple sub-areas includes: Get the initial grid size for grid division; Calculate the updated grid size, which includes the length and width of the grid. The updated grid size satisfies the relationship: , represents the updated grid size, represents the probability of reflection, represents the initial grid size; Divide the ROI area according to the updated grid size to obtain multiple sub-areas; The target grayscale image comprises: The cavity types are normal cavity, residual cavity, defective cavity and reflective cavity; When any of the probability values ​​is greater than 0.5, the corresponding cavity type is obtained; The cavity surface grayscale image corresponding to the mold with cavity reflection and the cavity surface grayscale image corresponding to the mold with probability values ​​not greater than 0.5 are taken as the target grayscale image.

2. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: The neural network is a convolutional neural network, which extracts features from the grayscale image of the cavity surface to obtain image features, and classifies the image features to obtain probability values.

3. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: The training process of the neural network includes: The grayscale image of the mold cavity surface in the history is used as input information, and the true value of the mold cavity type is used as a label to obtain a set of training data; Input the training data into the neural network and get the output result; The loss value of the neural network is calculated by outputting the results and labels. The loss function uses cross entropy loss. The error signal is back-propagated according to the loss value, and the network parameters of the neural network are updated to reduce the loss value. Iteratively update the network parameters of the neural network. When the neural network reaches the set maximum number of training times or the loss value is less than the set loss value, stop updating and obtain a trained neural network.

4. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: Acquiring the target grayscale image further includes: For any cavity surface grayscale image, the Euclidean distance between normal probability and residual probability, the Euclidean distance between normal probability and defect probability, and the Euclidean distance between normal probability and reflection probability are calculated respectively. The three Euclidean distances are added together and the result of negative correlation mapping is used as the degree of difficulty in classification. The grayscale image of the cavity surface with a degree of difficulty in classification greater than a preset threshold is taken as the target grayscale image.

5. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: The gradient consistency includes: For any sub-region, the gradient direction is weighted summed according to the gradient value of each pixel in the sub-region to obtain the total gradient direction of the sub-region; The total gradient direction of each sub-region in the ROI region is obtained by traversing, the gradient co-occurrence matrix of the ROI region is constructed, and the modulus of the gradient co-occurrence matrix is ​​used as the gradient consistency.

6. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: The gradient consistency also includes: For any sub-region, calculate the gradient direction of each pixel in the sub-region and draw a gradient histogram to obtain the main gradient direction and secondary gradient direction of the sub-region; Construct the main gradient co-occurrence matrix and sub-gradient co-occurrence matrix of the ROI region respectively; The main gradient eigenvector is calculated according to the main gradient co-occurrence matrix, and the sub-gradient eigenvector is calculated according to the sub-gradient co-occurrence matrix; Calculate the gradient consistency and satisfy the relationship: ,in, represents gradient consistency, represents the main gradient eigenvector, represents the transpose of the subgradient eigenvector, represents the magnitude of the main gradient eigenvector, Represents the magnitude of the subgradient eigenvector.

7. The method for detecting mold cavity abnormality based on machine vision according to claim 1, characterized in that: The gradient consistency also includes: For any sub-region, calculate the gradient direction of each pixel in the sub-region and draw a gradient histogram to obtain the main gradient direction and secondary gradient direction of the sub-region; Construct the main gradient co-occurrence matrix and sub-gradient co-occurrence matrix of the ROI region respectively; The main gradient eigenvector is calculated according to the main gradient co-occurrence matrix, and the sub-gradient eigenvector is calculated according to the sub-gradient co-occurrence matrix; The similarity between the main gradient eigenvector and the sub-gradient eigenvector is taken as the gradient consistency.

Images