A sand mold defect detection method and system based on machine vision

Abstract

The present invention discloses a sand mold defect detection method and system based on machine vision, comprising the following steps: obtaining a current sand mold image to be inspected, determining the sand mold model of the current sand mold image and a standard sand mold image; aligning the two sand mold images based on the center point of the pores; obtaining a sequence of image blocks of the current sand mold image and the standard sand mold image; calculating the texture features of each image block; comparing the similarity between the texture features of image blocks at corresponding positions in the current sand mold image and the standard sand mold image to determine whether the current image block has a defect; and based on all defective image blocks, in combination with a defect segmentation model, determining the type, size, and location of each defect in the sand mold to be inspected. The present invention can achieve accurate and refined detection of all sand mold defect types, significantly reducing the scrap rate of cast blank products.

Description

Technical Field

[0001] The present invention relates to the technical field of sand mold defect detection, and in particular to a sand mold defect detection method and system based on machine vision. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] A sand mold is a casting cavity made of raw sand, binder, and other auxiliary materials during the casting production process. Sand casting refers to the method of producing castings in a sand mold. Sand casting is the most widely used casting method in actual production. It is suitable for the production of castings of various shapes, sizes, batches, and various common alloys. Accurately detecting defects in the sand molds used in casting production can greatly reduce product scrap rates.

[0004] Common sand mold defects include mold peeling, sand sticking to the mold plate, and sand block dropouts. During the sand mold production process, due to the complex structure and large area of ​​the sand mold, it is difficult to accurately and quickly identify all defects, especially small ones, using the human eye alone. Furthermore, traditional manual visual inspection methods are prone to false positives, missed detections, and low efficiency. It is also difficult to verify the accuracy of various sand mold components, resulting in a high scrap rate for the rough products obtained from sand casting. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a sand mold defect detection method and system based on machine vision, which detects sand mold defects in real time through image acquisition and image processing technology to improve the qualification rate of sand mold production.

[0006] In some embodiments, the following technical solutions are adopted:

[0007] A sand mold defect detection method based on machine vision, comprising:

[0008] Obtain a current sand mold image to be detected, determine the sand mold model of the current sand mold image through a template matching method, and obtain a standard sand mold image of the sand mold model;

[0009] Detect the strip-shaped pores and circular pores of the current sand mold image and the standard sand mold image respectively, and calculate the pore center points; align the two sand mold images based on the pore center points;

[0010] The two aligned sand mold images are divided into multiple block regions of set sizes. The block regions of the current sand mold image correspond one-to-one with the block regions of the standard sand mold image, thereby obtaining an image block sequence of the current sand mold image and the standard sand mold image;

[0011] Calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at the corresponding positions in the current sand mold image and the standard sand mold image to determine whether the current image block has defects;

[0012] Based on all the image blocks with defects and combined with the defect segmentation model, the type, size and location of each defect in the sand mold to be inspected are obtained.

[0013] In other embodiments, the following technical solutions are adopted:

[0014] A sand mold defect detection method based on machine vision, comprising:

[0015] An image acquisition module is used to acquire the current sand mold image to be detected, determine the sand mold model of the current sand mold image through a template matching method, and acquire a standard sand mold image of the sand mold model;

[0016] The image alignment module is used to detect the strip-shaped pores and circular pores in the current sand mold image and the standard sand mold image respectively, and calculate the pore center points; and align the two sand mold images based on the pore center points;

[0017] The defect recognition module is used to divide the two aligned sand mold images into multiple block areas of a set size. The block areas of the current sand mold image correspond one-to-one with the block areas of the standard sand mold image, thereby obtaining an image block sequence of the current sand mold image and the standard sand mold image; calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at corresponding positions in the current sand mold image and the standard sand mold image, and determine whether the current image block has a defect;

[0018] The defect segmentation module is used to obtain the type, size and location of each defect in the sand mold to be detected based on all defective image blocks combined with the defect segmentation model.

[0019] In other embodiments, the following technical solutions are adopted:

[0020] A terminal device includes a processor and a memory, wherein the processor is used to implement various instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the above-mentioned sand mold defect detection method based on machine vision.

[0021] In other embodiments, the following technical solutions are adopted:

[0022] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the above-mentioned sand mold defect detection method based on machine vision.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) The present invention aligns the current sand mold image with the standard sand mold image, segments the image blocks, and determines whether each image block has defects. The defective image blocks are then passed through a neural network model to achieve accurate and refined detection of all sand mold defect types. This can save labor, reduce costs, improve detection accuracy, ensure product quality, improve production efficiency, and significantly reduce the scrap rate of casting blank products. Defect data is also saved in real time for the factory to promptly improve the production process.

[0025] Other features and advantages of additional aspects of the present invention will be given in part in the following description and in part will become obvious from the following description or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the structure of a sand mold defect detection device based on machine vision in an embodiment of the present invention;

[0027] Figure 2 Flowchart of a sand mold defect detection method based on machine vision in an embodiment of the present invention;

[0028] Figure 3 This is a flow chart of the pore detection algorithm in an embodiment of the present invention;

[0029] Figure 4 Schematic diagram of pore detection results of a sand mold image in an embodiment of the present invention;

[0030] Figure 5 Flowchart of the sand mold texture feature extraction algorithm in an embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of a defect segmentation model in an embodiment of the present invention;

[0032] Figure 7 Schematic diagram of a sand mold image with a mold removal defect in an embodiment of the present invention;

[0033] Figure 8 Schematic diagram of a sand mold image with a defect of sand sticking to the mold plate in an embodiment of the present invention;

[0034] Figure 9 Schematic diagram of a sand mold image with defects of falling small sand pieces in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0037] Example 1

[0038] In one or more embodiments, a sand mold defect detection method based on machine vision is disclosed. Figure 1 The machine vision-based sand mold defect detection device shown in the figure includes an industrial camera, an industrial lens, an industrial light source, a dark box, and an industrial computer. The bottom of the dark box is provided with a feed port and a discharge port according to the conveying direction of the sand mold production line. The production line extends through the feed port and the discharge port of the dark box to the outside of the dark box, and the sand mold to be tested is placed on the production line. The industrial light source is fixed to the top of the dark box. The industrial light source is configured as a surface light source, and the direction of the light beam is perpendicular to the surface of the sand mold. The industrial camera is set below the industrial light source, and the industrial camera is fixed to the top of the dark box. The industrial lens is assembled with the industrial camera, and the industrial lens is perpendicular to the surface of the sand mold.

[0039] After capturing images, the industrial camera transmits the digitized images to the industrial computer via Gigabit Ethernet. The industrial computer inputs the obtained images into the sand mold defect detection system and outputs the detection results.

[0040] Based on the above device, this embodiment discloses a sand mold defect detection method based on machine vision, combined with Figure 2 , including the following steps:

[0041] Step (1): obtaining a current sand mold image to be detected, determining the sand mold model of the current sand mold image by a template matching method, and obtaining a standard sand mold image of the sand mold model;

[0042] After acquiring the current sand mold image, the industrial computer uses a template matching algorithm to match it with standard sand molds of different models in the database. The sand mold model with the greatest similarity is used as the model of the current sand mold image, and a standard sand mold image is obtained for detecting defects in the current sand mold. In this embodiment, the standard sand mold images of different models are set as normal sand mold images without defects.

[0043] Step (2): Detect the strip pores and circular pores of the current sand mold image and the standard sand mold image respectively, and calculate the pore center points; align the two sand mold images based on the pore center points;

[0044] The pore detection algorithm in this embodiment is a connected domain clustering algorithm based on multi-threshold segmentation. A series of thresholds are used to segment the sand mold image to obtain a series of binary images. The strip and circular connected areas are obtained using the restriction conditions. Then, the connected domains in different binary images are clustered to obtain the strip and circular pores of the sand mold. The algorithm flow chart is shown in FIG. Figure 3 As shown, the specific process includes the following:

[0045] ① For the current sand mold image or the standard sand mold image, a series of continuous thresholds are used to segment the sand mold image to obtain a series of binary images;

[0046] ② Extract all connected regions consisting of zero pixels in each binary image and calculate the center and area of ​​the connected regions;

[0047] ③ By limiting the aspect ratio and area of ​​the minimum circumscribed rectangle of the connected region, the long strip connected region is screened out; by limiting the roundness and area of ​​the connected region, the circular connected region is screened out;

[0048] ④ According to statistics, the grayscale value of pore pixels in sand mold images is the smallest. Therefore, a single pore will produce overlapping connected areas in a series of binary images, while connected areas that are not pores will produce only a small amount of overlapping connected areas. To remove connected areas that are not pores and improve pore detection accuracy, we cluster the long strip connected areas and circular connected areas whose center coordinate spacing is less than the set first threshold and whose area difference is less than the set second threshold. Clusters with fewer than the set third threshold are then removed. The largest connected area in each cluster is then selected, representing the strip pores and circular pores in the current sand mold image or the standard sand mold image.

[0049] The sand mold image pore detection results are as follows: Figure 4 As shown. The offset distance between the corresponding pore center points of the current sand mold image and the standard sand mold image is then calculated, and the two sand mold images are aligned vertically and horizontally based on the offset distance. For example, in the image coordinate system, the coordinates of the same pore center point in the current sand mold image and the standard sand mold image are (241, 150) and (238, 155), respectively. This indicates that the current sand mold image is offset 3 pixels to the right and 5 pixels upward relative to the standard sand mold image. Therefore, the two sand mold images can be aligned by first translating the current sand mold image to the left by 3 pixels and then translating it downward by 5 pixels.

[0050] Step (3): Divide the two aligned sand mold images into multiple block areas of set sizes. The block areas of the current sand mold image correspond one-to-one with the block areas of the standard sand mold image, and their positions and sizes are exactly the same, thereby obtaining an image block sequence of the current sand mold image and the standard sand mold image; calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at corresponding positions in the current sand mold image and the standard sand mold image, and determine whether the current image block has defects;

[0051] Specifically, the two sand mold images are divided into several image blocks to obtain the image block sequence of the standard sand mold {G1, G2, G3...Gn} and the image block sequence of the current sand mold {P1, P2, P3...Pn};

[0052] In this embodiment, the size of the two sand mold images is 2048×2450, and the two sand mold images are divided into 10×10 image blocks of size 204×245 to obtain a standard sand mold image block sequence.

[0053] {G1, G2, ..., Gi, ...G100}, and the image block sequence of the current sand mold

[0054] {P1, P2, …, Pi, …P100}; among them, the size, dimension and position of Gi and Pi are completely corresponding.

[0055] The texture feature extraction algorithm is used to calculate the texture features of each image block respectively, and the texture feature similarity between the standard sand mold image block Gi and the image block Pi corresponding to the current sand mold is calculated. If the texture feature similarity is less than the threshold T, it is considered that the image block Pi of the current sand mold has defects;

[0056] In this embodiment, the LBP feature extraction algorithm is used to calculate the texture feature vector representing each image block. The LBP feature extraction algorithm flow chart is as follows: Figure 5 As shown, the specific process includes the following:

[0057] ① Perform Gaussian filtering on the image block and divide it into multiple square areas of set size;

[0058] ② Calculate the LBP value of each pixel in each square area, obtain the LBP distribution histogram of each square area, and normalize the histogram;

[0059] ③ Connect the LBP distribution histograms of all square areas of the image block into a vector, which is the LBP texture feature vector of the image block.

[0060] The texture feature similarity of two image blocks is calculated based on the Euclidean distance between the corresponding texture feature vectors, as shown in formula (1);

[0061]

[0062] The larger the Euclidean distance is, the smaller the similarity is. If the Euclidean distance is greater than a set fourth threshold, it is considered that the current image block has a defect.

[0063] Step (4): Based on all the image blocks with defects and combined with the defect segmentation model, the type, size and location of each defect in the sand mold to be detected are obtained.

[0064] Adjacent image blocks with defects are merged to obtain a sequence of defective image blocks {F1, F2, F3…Fn} in the current sand mold image. The defective image block F is input into the defect segmentation model, which outputs the type of each pixel in the image block, including non-defective, mold peeling, sand sticking to the mold plate, and sand block falling. Then, the pixels belonging to the same defect type are counted to obtain the defect type, size, and location contained in the current image block.

[0065] In this embodiment, the defect segmentation model uses a multi-layer deep convolutional network, such as Mask RCNN, HRNet, etc. The network structure of Mask RCNN is as follows: Figure 6 As shown in the figure, the training data includes local images of sand mold defects and manually labeled data of the defect type of each pixel. The model is trained using a stochastic gradient descent optimization algorithm. Finally, the defect type of each pixel can be output, and the defect type, size, and location contained in the current image block are finally obtained.

[0066] In this embodiment, it is possible to inspect larger sand molds. The length, width, and depth of the sand mold to be inspected in this embodiment are approximately 1.4m × 1.1m × 0.4m. This embodiment method supports the detection of all defect types, including mold removal, cracks, missing materials, sand sticking to the mold plate, and small sand blocks falling off. Figure 7-Figure 9 Schematic diagrams of sand mold images showing mold removal defects, sand sticking to the mold plate defects, and small sand drop defects are provided. This method can detect and segment defects as small as 10mm x 10mm. It also supports user-defined rejection criteria, allowing users to manually set rejection criteria based on defect location and size.

[0067] As a specific implementation method, the method of this embodiment is applied to a vehicle having a length, width and depth of about 1.4 m×

[0068] The surface of a 1.1m×0.4m sand mold is inspected for mold peeling, material shortage, sand sticking to the mold plate, and small sand blocks. The sand mold color is close to black-brown. A 5-megapixel black-and-white industrial camera is used, with an industrial lens with a focal length of 8.06mm, a field of view of 816mm×682mm, an exposure time of 6.5ms, and an image resolution of 2048×2450. The image block size in step S4 is 204×245, and the texture overlap threshold is 0.8.

[0069] The measured length of the sand mold surface defect is 300mm, the width is 280mm, and the area is 111,111mm. 2 The length of the sand sticking defect on the template is 100mm, the width is 110mm, and the area is 11,000mm. 2 ; The defect of small sand pieces falling is 50mm in length, 50mm in width and 2,500mm in area. 2 .

[0070] Example 2

[0071] In one or more embodiments, a machine vision-based sand mold defect detection system is disclosed, comprising:

[0072] An image acquisition module is used to acquire the current sand mold image to be detected, determine the sand mold model of the current sand mold image through a template matching method, and acquire a standard sand mold image of the sand mold model;

[0073] The image alignment module is used to detect the strip-shaped pores and circular pores in the current sand mold image and the standard sand mold image respectively, and calculate the pore center points; the two sand mold images are aligned based on the pore center points;

[0074] The defect recognition module is used to obtain the image block sequence of the current sand mold image and the standard sand mold image, calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at corresponding positions in the current sand mold image and the standard sand mold image, and determine whether the current image block has defects;

[0075] The defect classification module is used to obtain the type, size and location of each defect in the sand mold to be inspected based on all defective image blocks combined with the defect classification model.

[0076] It should be noted that the specific implementation of each of the above modules has been described in detail in Example 1 and will not be described in detail here.

[0077] Example 3

[0078] In one or more embodiments, a terminal device is disclosed, including a server. The server includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the machine vision-based sand mold defect detection method of Example 1 is implemented. For the sake of brevity, this description is omitted here.

[0079] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0080] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0081] During implementation, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software.

[0082] Example 4

[0083] In one or more embodiments, a computer-readable storage medium is disclosed, in which a plurality of instructions are stored. The instructions are suitable for being loaded by a processor of a terminal device and executing the sand mold defect detection method based on machine vision described in Example 1.

[0084] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A sand mold defect detection method based on machine vision, characterized in that: include: Obtain a current sand mold image to be detected, determine the sand mold model of the current sand mold image through a template matching method, and obtain a standard sand mold image of the sand mold model; Detect the strip-shaped pores and circular pores of the current sand mold image and the standard sand mold image respectively, and calculate the pore center points; Align the two sand mold images based on the pore center point; The two aligned sand mold images are divided into multiple block regions of set sizes. The block regions of the current sand mold image correspond one-to-one with the block regions of the standard sand mold image, thereby obtaining an image block sequence of the current sand mold image and the standard sand mold image; Calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at the corresponding positions in the current sand mold image and the standard sand mold image to determine whether the current image block has defects; Based on all defective image blocks and combined with the defect segmentation model, the type, size and location of each defect in the sand mold to be inspected are obtained; Detect the strip-shaped pores and circular pores of the current sand mold image and the standard sand mold image respectively, including: For the current sand mold image or the standard sand mold image, a series of continuous thresholds are used to segment the sand mold image to obtain a series of binary images; Extract all connected regions consisting of zero pixels in each binary image and calculate the center and area of ​​the connected regions; By limiting the aspect ratio and area of ​​the minimum circumscribed rectangle of the connected region, the long strip connected region is screened out; by limiting the roundness and area of ​​the connected region, the circular connected region is screened out; Cluster the long strip connected regions and circular connected regions whose center coordinate spacing is less than the set first threshold and whose area difference is less than the set second threshold respectively, and extract the connected region with the largest area in each cluster, which is the strip pore and circular pore of the current sand mold image or the standard sand mold image; Calculate the texture features of each image block, including: Perform Gaussian filtering on the image block and divide it into multiple square areas of set size; Calculate the LBP value of each pixel in each square area, obtain the LBP distribution histogram of each square area, and normalize the histogram; The LBP distribution histograms of all square areas of the image block are connected into a vector, which is the LBP texture feature vector of the image block.

2. A sand mold defect detection method based on machine vision according to claim 1, characterized in that: The sand mold model of the current sand mold image is determined by the template matching method, specifically including: The sand mold image to be detected is matched with the standard sand mold images of different models in the database for similarity, and the model of the standard sand mold image with the highest similarity to the current sand mold image is used as the sand mold model of the current sand mold image.

3. The sand mold defect detection method based on machine vision according to claim 1, characterized in that: Align the two sand mold images based on the pore center point, including: Calculate the offset distance between the center points of the pores of the current sand mold image and the standard sand mold image, and align the two sand mold images vertically and horizontally according to the offset distance.

4. The sand mold defect detection method based on machine vision according to claim 1, characterized in that: Compare the similarity between the texture features of the image blocks at the corresponding positions in the current sand mold image and the standard sand mold image, specifically including: The Euclidean distance between the texture features of the two image blocks is calculated. If the Euclidean distance is greater than a set fourth threshold, it is considered that the current image block has a defect.

5. The sand mold defect detection method based on machine vision according to claim 1, characterized in that: Based on all defective image blocks and combined with the defect segmentation model, the type, size, and location of each defect in the sand mold to be inspected are obtained, including: Merge adjacent image blocks with defects at the same time to obtain a sequence of image blocks with defects in the current sand mold image. Input the defective image block into the defect segmentation model, output the type of each pixel in the image block, count the pixels belonging to the same defect type, and obtain the defect type, size and location contained in the current image block; The defect segmentation model uses a multi-layer deep convolutional network.

6. A sand mold defect detection system based on machine vision, using a sand mold defect detection method based on machine vision according to any one of claims 1 to 5, characterized in that: include: An image acquisition module is used to acquire the current sand mold image to be detected, determine the sand mold model of the current sand mold image through a template matching method, and acquire a standard sand mold image of the sand mold model; The image alignment module is used to detect the strip-shaped pores and circular pores in the current sand mold image and the standard sand mold image respectively, and calculate the center points of the pores; Align the two sand mold images based on the pore center point; The defect recognition module is used to divide the two aligned sand mold images into multiple block areas of a set size. The block areas of the current sand mold image correspond one-to-one with the block areas of the standard sand mold image, thereby obtaining an image block sequence of the current sand mold image and the standard sand mold image; calculate the texture features of each image block; compare the similarity between the texture features of the image blocks at corresponding positions in the current sand mold image and the standard sand mold image, and determine whether the current image block has a defect; The defect segmentation module is used to obtain the type, size and location of each defect in the sand mold to be detected based on all defective image blocks combined with the defect segmentation model.

7. A terminal device comprising a processor and a memory, wherein the processor is used to implement various instructions; the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the sand mold defect detection method based on machine vision according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the sand mold defect detection method based on machine vision according to any one of claims 1 to 5.

Images