Intelligent identification and statistical method for die-casting pre-crystallization defect structure

Abstract

The invention discloses a die-casting pre-crystallization defect structure intelligent identification and statistical method, which comprises the following steps: S1, taking a metallographic structure picture of a die-casting alloy through an optical microscope to form a metallographic picture data set, and carrying out an image pretreatment step; s2, labeling the metallographic pictures to form an image-label data set; s3, performing data enhancement processing on the image data set to increase the number of training samples; s4, training the data set by adopting an improved SegNet model, and evaluating the prediction accuracy of the model; s5, identifying and predicting a pre-crystallization region in a new input image by adopting the model trained in the step S4; and calculating the area percentage content by counting the proportion of the number of the pixel points in the pre-crystallization area. According to the method, the die-casting pre-crystallization defect structure can be identified and subjected to content statistics efficiently, accurately and intelligently, and an important basis is provided for intelligent control and process parameter optimization decision-making of the pre-crystallization defect structure.

Description

Technical Field

[0001] The invention relates to the technical field of die casting, and in particular to an intelligent identification and statistical method for die casting pre-crystallization defective structure. Background Art

[0002] With the development of my country's new energy vehicle industry, large integrated die-cast aluminum and magnesium alloy parts have been widely used in the trial production and actual application of large body parts such as subframes, rear floors, battery housings, instrument panel brackets, and rear hatchback doors due to their high degree of integration, low assembly cost, and good lightweight effect. However, the large integrated die-casting process has the characteristics of many parameters, many physical fields, and large time-varying disturbances, which makes it difficult to accurately control the defects in the castings and reduce the defective rate.

[0003] In large integrated die castings, pores, cold shuts, shrinkage cavities, cracks, and pre-crystallization structures are common types of defects, which will have a certain degree of negative impact on the performance of the castings. Among them, the pre-crystallization structure is a structure with coarser grains than the ordinary die-casting structure. The reason for its generation is that when the molten metal is injected into the pressure chamber of the cold chamber die-casting machine, the molten metal contacts the pressure chamber wall with a lower temperature and pre-crystallizes into some small-grain crystals. These small-grain crystals continue to grow during the subsequent injection process. Since its generation time is earlier than that of normal die-casting structures, its size is coarser. And because there is a certain special interaction between the pre-crystallized particles and the molten metal during filling and solidification, shrinkage cavities, cracks and other defects are often enriched around them, which has a more serious negative impact on the performance of the die-casting. Therefore, the content of the pre-crystallization structure is an important criterion for judging whether the quality of the die-casting is qualified and whether the die-casting process parameters need to be further optimized. If you want to establish an accurate prediction of the pre-crystallization structure, you first need to establish a quantitative mapping relationship between the pre-crystallization structure and the die-casting process. However, the current observation and statistics of die-casting pre-crystallization structures mainly use naked eye observation and subjective judgment methods for gold image images, which have low accuracy, strong subjectivity, low efficiency, and cannot form quantitative statistical information or effective data sets. They are less inspiring and instructive for the study of the formation laws of pre-crystallization of castings and the formulation of process parameter optimization strategies. As a result, there is still insufficient research on the formation laws of pre-crystallization defect structures and a lack of effective means to control pre-crystallization defect structures.

[0004] In summary, the deficiencies of the existing background technology can be summarized as follows: (1) There is a lack of intelligent, batch-based automatic identification system for die-casting pre-crystallization defective tissue; (2) There is a lack of automatic quantitative statistical method for the content of die-casting pre-crystallization defective tissue.

[0005] With the development of machine learning and artificial intelligence technology, deep learning technology has been widely used in image recognition tasks in various industries. Deep learning technology based on convolutional neural networks can effectively realize image recognition, detection and segmentation tasks, and is an effective means to realize intelligent recognition and automatic statistics of die-casting pre-crystallization structure. Summary of the invention

[0006] Based on the defects and shortcomings of the above-mentioned status quo, the present invention aims to solve the problems of strong subjectivity, low efficiency, low accuracy and difficulty in quantitative analysis in the observation statistics of pre-crystallization structure of die-cast aluminum alloys and magnesium alloys, and provide a method for intelligent identification and statistics of die-casting pre-crystallization defective structure, set the pre-crystallization structure image pre-processing standards, establish a deep learning model for intelligent identification of pre-crystallization structure, and design an automatic statistical algorithm for the pre-crystallization structure content.

[0007] To achieve the above objectives, the present invention provides a die-casting pre-crystallization defect structure intelligent identification and statistical method, which comprises the following steps:

[0008] S1: Take metallographic photos of the die-cast alloy using an optical microscope to form a metallographic image dataset and perform image pre-processing steps;

[0009] S2: Annotate metallographic images to form an “image-label” data set;

[0010] S3: Perform data augmentation on the image dataset to increase the number of training samples;

[0011] S4: Use the improved SegNet model to train the dataset and evaluate the model prediction accuracy;

[0012] S5: using the model trained in step S4 to identify and predict the pre-crystallization area in the new input image; calculating the area percentage content by counting the percentage of the number of pixels in the pre-crystallization area.

[0013] A further improvement of the present invention is that in step S1, the image pre-processing step includes: converting the original RGB image into a grayscale image, increasing the image contrast, and reconverting it into an RGB image to form an RGB image containing only grayscale color features. In step S1, each effective metallographic image must be guaranteed to contain a pre-crystallized tissue with an area percentage of 5%-70%, and to contain abundant "normal tissue-pre-crystallized tissue" boundaries. During the pre-processing process, the image contrast is increased by 3 to 5 times to form a grayscale image with high contrast. Compared with the original image, the pre-processed metallographic image can obtain better training effects in the image training stage.

[0014] A further improvement of the present invention is that the labeling method in step S2 includes: using the boundary line between the pre-crystallized tissue and the normal tissue area as a standard to divide different areas of the image, and using red, blue and green to represent the pre-crystallized tissue, normal tissue and the area outside the sample, respectively, to form a label image composed of three primary colors. The label image and the original image have the same file name to establish a one-to-one correspondence, and are saved in different folders.

[0015] A further improvement of the present invention is that the data enhancement method in step S3 includes: cutting and then rotating the original image and the corresponding label image, cutting n times and rotating s times, if the number of original metallographic images is N, then the number of sub-images after data enhancement is N·n·(s+1) times;

[0016] The calculation criteria for the sub-image size L of the cutting operation include: for the case where the microscope magnification is M and the pixel size of the original rectangular image is P×Q×3, the pixels of the square sub-image after cutting are L×L×3, then L, M, P, and Q satisfy the following relationship:

[0017] L=k·M·Q (1)

[0018] Where k is a custom coefficient; if the original image pixels are 2580×1944×3, k is 1 / 972, and the pixels of the sub-image after cutting are 1000×1000×3. Studies have shown that sub-images cut according to this size ratio can better reflect the size differences and morphological characteristics of normal tissues and pre-crystallized tissues in each image, and the machine learning effect is better.

[0019] The methods for batch cutting images include: sliding window method, random sampling method; scaling the metallographic image after cutting and rotation operation and the corresponding label image to A×A×3 pixels in proportion, where A <L。

[0020] The further improvement of the present invention is that the improved SegNet model in step S4 includes: using the open source SegNet neural network as the basic model, replacing the convolution operation of the convolution layer with a depth-separable convolution operation to reduce the model size and obtain a faster operation speed. The model input size is A×A×3 three-channel two-dimensional matrix, and the model output size is 3A 2 ×1 three-channel one-dimensional vector; in the process of evaluating the prediction accuracy of the model, four indicators, loss, accuracy, recall and precision, are used to comprehensively evaluate the prediction effect of the model.

[0021] A further improvement of the present invention is that in step S5, in the process of using the model to identify and predict the pre-crystallization region in the new input image, the operation is performed according to the image pre-processing, image cutting, region prediction, and image splicing process, which specifically includes:

[0022] (1) Pre-processing the predicted metallographic structure photograph according to step S1;

[0023] (2) Segment the pre-processed metallographic image according to the size L of the cut sub-image used in step S3, and divide it into z sub-images to be predicted, where z is expressed as:

[0024] z=(int(P / L)+1)(int(Q / L)+1) (2)

[0025] Where int(·) indicates the rounding operation, P, Q, and L are the length and width of the original image and the side length of the sub-image in pixels respectively; the part exceeding the original image size after segmentation is represented in black;

[0026] (3) using the model trained in step S4 to perform image segmentation on each sub-image to be tested, to form a set of segmented sub-images; the segmented region types in the sub-image set include pre-crystallization tissue, normal tissue, and regions outside the sample;

[0027] (4) The sub-images are stitched together according to the corresponding positions during segmentation, and the black parts that exceed the size of the original image are cut off to form a complete image segmentation result; and the image segmentation result is superimposed with the original image. The superposition rule is:

[0028] I F =I0·α+I p β (3)

[0029] Among them I F is the RGB value of the superimposed image, I0 is the RGB value of the original image, and I P is the RGB value of the segmentation result, ɑ and β are decimals between 0 and 1;

[0030] (5) Calculate the area percentage of the pre-crystallization defective tissue region as follows:

[0031]

[0032] Where θ is the percentage of pre-crystallized tissue area, π r is the number of red pixels in the segmentation result, π b is the number of blue pixels in the segmentation result.

[0033] The advantages of the present invention are as follows:

[0034] 1. The present invention adopts a deep learning model based on a semantic segmentation convolutional neural network to train the calibrated die-casting pre-crystallization structure metallographic images, thereby realizing intelligent recognition of pre-crystallization defective tissue areas. The image feature extraction mode of the recognition method is deep separable convolution, which can intelligently extract deep-level image features through big data training, and obtain more and richer image features than traditional naked eye observation or grayscale feature extraction, thereby realizing accurate and automatic recognition of the target area and increasing the objectivity and accuracy of pre-crystallization structure recognition.

[0035] 2. The present invention adopts a deep learning model based on semantic segmentation convolutional neural network to perform intelligent prediction of die-casting pre-crystallization structure, which can effectively reduce the manpower and time costs of metallographic image analysis in die-casting experiments, trial production, and production.

[0036] 3. The present invention can automatically calculate the percentage of pre-crystallized structure content based on the predicted output of the convolutional neural network, with high accuracy and fast calculation speed. It can effectively realize the batch quantitative statistics of the target metallographic structure and provide strong data support for the establishment of the quantitative relationship between die-casting "process parameters-pre-crystallized structure content".

[0037] 4. The statistical algorithm for intelligent identification of pre-crystallized structures used in the present invention can be transferred to the identification of other abnormal metallographic structures or specific metallographic structures in die-cast alloys, and has strong universality and promotion capabilities. In addition, the deep learning model can be continuously iterated and optimized through active learning, reinforcement learning and other methods to improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The figure is a comparison between the original metallographic image of the die-cast aluminum alloy in the embodiment of the present invention and the image after pre-treatment.

[0039] Figure 2 The figure is a comparison between the metallographic image after pre-processing and the result after calibration in the embodiment of the present invention.

[0040] Figure 3 Comparison between the metallographic sub-image after cutting and flipping and the calibration result sub-image in the embodiment.

[0041] Figure 4 Schematic diagram of the SegNet network structure improved by deep separable convolution in an embodiment of the present invention.

[0042] Figure 5 This is the SegNet network training process improved by deep separable convolution in an embodiment of the present invention.

[0043] Figure 6 This is a comparison of the original metallographic image, the pre-processed image, and the pre-crystallization intelligent recognition result in the embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0045] While some exemplary embodiments of the present invention have been described for the purpose of illustration, it should be understood that the present invention may be implemented in other ways not specifically shown in the drawings.

[0046] The embodiment of the present invention takes the metallographic image of the aluminum alloy subframe die casting as an example. The aluminum alloy material involved in the embodiment is JDA2b heat-free aluminum alloy, which contains 33 typical metallographic structures. The magnification of the metallographic photo is M=500x, and the resolution of the image is 2580×1944×3. First, because the settings of the metallographic microscope are different when shooting, the metallographic photos often appear in different colors and different contrasts. For example, Figure 1 The metallographic photograph of (a) is bright green with high contrast. Figure 1 The metallographic photograph in (c) is dark green with low contrast. Figure 1 (c) The hue and contrast of the image will seriously affect the training of the model, resulting in a decrease in the final prediction ability. Therefore, all images are preprocessed, and the Python third-party library function Image.convert is used to convert the RGB three-channel image into a grayscale image and equalize the contrast. Figure 1 The contrast of the golden image in (a) is increased by 3 times. Figure 1 The contrast of the golden image in (c) is increased by 5 times, e.g. Figure 1 Finally, the grayscale image is converted into an RGB three-channel image to meet the input requirements of the convolutional neural network.

[0047] Labelme software was used to calibrate the pre-processed gold image and divide the pre-crystallization tissue area from the normal tissue area. Figure 2 shown. Figure 2 (b) is the tissue calibration result of one of the metallographic images, where the area marked in red is the pre-crystallization defect tissue area, and the area marked in blue is the normal tissue area. Since there is no area outside the sample in this metallographic image, there is no part marked in green.

[0048] According to the sliding window method and formula (1), the original image and the calibrated image of the metallographic image are segmented simultaneously. From the image resolution of 2580×1944×3, we can know that Q=1944. When k is 1 / 972, the side length of the segmented image L=(1 / 972)×500×1944=1000. Therefore, the pixels of the segmented image are 1000×1000×3. The step size of the sliding window is 316 pixels horizontally and 188 pixels vertically. Finally, one original image can be segmented into 5×5=25 sub-images. Each sub-image is rotated 90° three times, and finally 33×25×4=3300 metallographic sub-images and corresponding 3300 calibrated result sub-images are obtained as training sets. Figure 3 As shown, the segmented metallographic sub-image and the calibration result sub-image are both square, and the same area has four different angles.

[0049] The python function tensorflow.image.resize is used to convert the resolution of the metallographic sub-image into a three-channel matrix of 256×256×3 to meet the input requirements of the SegNet network. The tensorflow.image.resize and tensorflow.reshape functions are used to convert the calibration result sub-image into a three-channel column vector of 65536×3 to meet the output requirements of SegNet. The tensorflow.data.Dataset.from_tensor_slices function is used to package the 3300 metallographic sub-images and 3300 calibration result column vectors into a data set for deep learning. The SegNet network structure improved by deep separable convolution is shown in the figure. Figure 4 As shown in the figure, all Convolution layers in the SegNet network are changed to SeparableConvolution layers, and the rest of the network structure remains unchanged. The network hyperparameters are set to have a learning rate of 0.0001, a batch size of 20, a loss function of categorical_crossentropy, an optimizer of Adam, and 10 training rounds. The network training process is as follows Figure 5 As shown in the figure, the model loss function tends to converge at 10 rounds, proving that the network training has been completed and continuing the training will lead to overfitting. The final accuracy, recall and precision of the model training set are 94.78%, 77.96% and 96.59% respectively; the final accuracy, recall and precision of the validation set are 96.78%, 93.49% and 98.34% respectively. It can be seen that the model training effect is good and there is no need to further adjust the hyperparameters.

[0050] The entire metallographic original image is identified by using the "image pre-processing - image cutting - regional prediction - image stitching" process in step S5 described above. The original image, the pre-processed image and the identified output image are shown in FIG. Figure 6 As shown in the figure, it can be seen that the trained deep learning network can effectively segment the pre-crystallization tissue area and output it visually. It also automatically outputs the percentage content of the pre-crystallization tissue area of ​​four pictures, which are 25.28%, 40.71%, 36.4%, and 19.93%, respectively. The prediction time for one picture is about 3.3 seconds, which realizes efficient, accurate, and intelligent pre-crystallization tissue recognition and statistics.

[0051] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An intelligent identification and statistical method for die casting pre-crystallization defect structure, characterized in that The following steps are involved: S1: Take metallographic photos of the die-cast alloy using an optical microscope to form a metallographic image dataset and perform image pre-processing steps; S2: Annotate the metallographic images to form an "image-label" data set; S3: Perform data augmentation on the image dataset to increase the number of training samples; S4: Use the improved SegNet model to train the dataset and evaluate the model prediction accuracy; S5: using the model trained in step S4 to identify and predict the pre-crystallization area in the new input image; calculating the area percentage content by counting the percentage of the number of pixels in the pre-crystallization area.

2. The intelligent identification and statistical method of die casting pre-crystallization defect structure according to claim 1, characterized in that: In step S1, the image pre-processing step includes: converting the original RGB image into a grayscale image, increasing the image contrast, and reconverting it into an RGB image to form an RGB image containing only grayscale color features.

3. The intelligent identification and statistical method of die casting pre-crystallization defect structure according to claim 1, characterized in that: The labeling method in step S2 includes: using the boundary line between the pre-crystallized tissue and the normal tissue area as a standard to divide different areas of the image, and using red, blue and green colors to represent the pre-crystallized tissue, normal tissue and the area outside the sample respectively, forming a label image composed of three primary colors.

4. The method for intelligent identification and statistical analysis of die casting pre-crystallization defective structures according to claim 1, characterized in that: The data enhancement method in step S3 includes: cutting and then rotating the original image and the corresponding label image, cutting n times and rotating s times. If the number of original metallographic images is N, the number of sub-images after data enhancement is N·n·(s+1) times; The calculation criteria for the sub-image size L of the cutting operation include: for the case where the microscope magnification is M and the pixel size of the original rectangular image is P×Q×3, the pixels of the square sub-image after cutting are L×L×3, then L, M, P, and Q satisfy the following relationship: L=k·M·Q Where k is a custom coefficient; The methods for batch cutting images include: sliding window method, random sampling method; scaling the metallographic image after cutting and rotation operation and the corresponding label image to A×A×3 pixels in proportion, where A <L。 5. The intelligent identification and statistical method of die casting pre-crystallization defect structure according to claim 4 is characterized by: The improved SegNet model in step S4 includes: using the open source SegNet neural network as the basic model, replacing the convolution operation of the convolution layer with a depth-wise separable convolution operation, the model input size is A×A×3 three-channel two-dimensional matrix, and the model output size is 3A 2 ×1 three-channel one-dimensional vector; in the process of evaluating the prediction accuracy of the model, four indicators, loss, accuracy, recall rate and precision, are used to comprehensively evaluate the prediction effect of the model.

6. The intelligent identification and statistical method of die casting pre-crystallization defect structure according to claim 5, characterized in that: In step S5, in the process of using the model to identify and predict the pre-crystallization region in the new input image, operations are performed according to the image pre-processing, image cutting, region prediction, and image splicing process, which specifically includes: (1) Pre-processing the predicted metallographic structure photograph according to step S1; (2) Segment the pre-processed metallographic image according to the size L of the cut sub-image used in step S3, and divide it into z sub-images to be predicted, where z is expressed as: z=(int(P / L)+1)(int(Q / L)+1) Where int(·) indicates the rounding operation, P, Q, and L are the length and width of the original image and the side length of the sub-image in pixels respectively; the part exceeding the original image size after segmentation is represented in black; (3) using the model trained in step S4 to perform image segmentation on each sub-image to be tested, to form a set of segmented sub-images; the segmented region types in the sub-image set include pre-crystallization tissue, normal tissue, and regions outside the sample; (4) The sub-images are stitched together according to the corresponding positions during segmentation, and the black parts that exceed the size of the original image are cut off to form a complete image segmentation result; and the image segmentation result is superimposed with the original image. The superposition rule is: I F =I0·α+I p ·b Among them I F is the RGB value of the superimposed image, I0 is the RGB value of the original image, and I P is the RGB value of the segmentation result, ɑ and β are decimals between 0 and 1; (5) Calculate the area percentage of the pre-crystallization defective tissue region as follows: Where θ is the percentage of pre-crystallized tissue area, π r is the number of red pixels in the segmentation result, π b is the number of blue pixels in the segmentation result.

Images