Image recognition method and system for particle size distribution of raw casting sand

Abstract

The present invention discloses an image recognition method and system for the particle size distribution of raw sand for casting, the method comprising: obtaining raw sand particle images and preprocessing them; introducing an instance segmentation branch network into a convolutional neural network model to construct an instance segmentation model; performing sand segmentation on the preprocessed raw sand particle images through a pretrained instance segmentation model; calculating the shape characteristics of the sand particles according to the sand segmentation results, and converting the shape characteristics of the sand particles from pixel size to actual size, calculating the equivalent diameter and shape factor of the sand particles; performing mesh classification according to the equivalent diameter and shape factor of the sand particles, counting the number of sand particles of each mesh, and obtaining the recognition result of the particle size distribution of raw sand for casting. The instance segmentation model constructed by the present invention can realize the instance segmentation of the sticky sand particles, reduce the statistical error caused by the sticking of the sand particles, and perform mesh classification and statistics of the particle size distribution of raw sand for casting according to the shape characteristics of the sand particles, thereby improving the accuracy of the calculation.

Description

Technical Field

[0001] The invention belongs to the technical field of raw sand particle size distribution detection, and in particular relates to an image recognition method and system for the particle size distribution of raw casting sand. Background Art

[0002] In the foundry industry, raw sand is an important component of casting materials. Its quality, particle size distribution and particle shape have a vital impact on the quality and performance of castings. The particle size of raw sand reflects the size and distribution of raw sand particles, and the AFS average fineness directly reflects the average size of raw sand particles. The particle size and distribution of raw sand directly affect the performance of molding sand, such as the air permeability, porosity, and strength of molding sand, which in turn affect the quality of castings. Therefore, in the production and use of raw sand, it is particularly important to select the appropriate raw sand particle size according to different process requirements. Accurately evaluating and controlling the particle size distribution and particle shape of raw sand is a key step in the casting process.

[0003] The traditional raw sand particle size distribution test mainly adopts the screening method. The testing process of the screening method is: weigh 50±1g of the sample, put the sample on the top sieve of the full set of casting test sieves, and then fasten the full set of sieves with the sample on the sand screening machine for screening. The screening time is 12-15min. After the screening is completed, the mass of the residue on each sieve is weighed, and the particle size composition is calculated according to the percentage of the mass of sand on each sieve to the total mass of the sample. The screening method for measuring the particle size of casting raw sand is currently the most widely used method for measuring the particle size of raw sand. However, the screening method is greatly affected by human factors, and the screen is a consumable product. With the increase of the number of measurements, some sieve holes are blocked by raw sand particles, and the size of some sieve holes is seriously deformed, resulting in serious deviations in the measurement results. Therefore, its detection accuracy is affected by factors such as the sieve aperture, vibration intensity, and human factors. The operation is cumbersome and time-consuming.

[0004] With the development of computer vision and image processing technology, the image-based particle size distribution detection method has gradually gained attention. This method uses a camera to collect images of raw sand particles and uses image processing technology to extract the characteristic information of the particles, thereby achieving an accurate assessment of the particle size distribution. The image method has the advantages of non-contact, high efficiency, and good repeatability, and is particularly suitable for rapid detection of a large number of raw sand samples.

[0005] However, the difficulty of the image method lies in the instance segmentation of the adhered sand grains in the image. Figure 1 As shown, due to the adhesion between the original sand particles, traditional image processing algorithms are difficult to accurately distinguish and segment each sand grain. During the segmentation process, it is easy to segment multiple adhered sand grains into single sand grains with larger areas, causing errors in the results.

[0006] Some existing neural network-based object detection algorithms can accurately locate and classify objects in images. However, such object detection algorithms generally only generate bounding boxes of objects and cannot directly segment the particles that are stuck together when processing images of stuck sand grains.

[0007] Therefore, a new image recognition method is needed to achieve instance segmentation of adhered sand particles and obtain accurate particle size distribution of raw casting sand. Summary of the invention

[0008] In view of this, the present invention proposes an image recognition method and system for the particle size distribution of raw casting sand, which is used to solve the problem that the existing image recognition method cannot accurately identify adhering sand particles, resulting in inaccurate assessment of the particle size distribution of raw casting sand.

[0009] In a first aspect, the present invention discloses an image recognition method for the particle size distribution of raw casting sand, the method comprising:

[0010] Obtain the original sand particle image and perform preprocessing;

[0011] Introduce instance segmentation branch network into the convolutional neural network model to build an instance segmentation model;

[0012] The pre-processed original sand particle image is segmented by using the pre-trained instance segmentation model;

[0013] Calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains;

[0014] The sand particles are classified by mesh size according to their equivalent diameter and shape factor, and the number of sand particles of each mesh size is counted to obtain the identification result of the particle size distribution of the original casting sand.

[0015] On the basis of the above technical solution, preferably, the instance segmentation model includes a ResNet module, an FPN module, a head module and an instance segmentation branch network;

[0016] The ResNet module, the FPN module, and the head module are connected in sequence, and the input end of the instance segmentation branch network is connected to the output ends of the FPN module and the head module respectively;

[0017] The ResNet module is used to extract three feature maps of different scales from the input image, and output the three feature maps of different sizes to the FPN module;

[0018] The FPN module is used to perform top-down upsampling and feature fusion on the three feature maps of different scales output by the ResNet module, and output feature layers of multiple scales;

[0019] The head module includes multiple head networks, which are respectively used to perform target recognition and regression on the feature layers of multiple scales output by the FPN module; the head network is a fully convolutional network that introduces multiple CBAM attention mechanisms;

[0020] The instance segmentation branch network is used to extract fixed-size feature blocks from the feature layer of the FPN module through the RoIAlign technology according to the detection results of the head module to perform mask prediction and realize instance segmentation.

[0021] On the basis of the above technical solution, preferably, the instance segmentation branch network includes a feature selection unit and a mask prediction unit connected in sequence;

[0022] The feature selection unit is used to calculate the size of each detection frame according to the regression result of the head module, and select the feature layer corresponding to the FPN module according to the size of each detection frame;

[0023] The mask prediction unit is used to extract a fixed-size feature block from the corresponding feature layer using the RoIAlign technology according to the selection result of the feature selection unit, perform mask prediction on the fixed-size feature block through a Mask branch including multiple convolutional layers, and generate a single-channel binary mask.

[0024] On the basis of the above technical solution, preferably, calculating the size of each detection frame according to the regression result of the head module, and selecting the feature layer corresponding to the FPN module according to the size of each detection frame specifically includes:

[0025] Calculate the size s of each detection box:

[0026]

[0027] Among them, s represents the scale of the detection box, w and h represent the width and height of the detection box respectively;

[0028] Assume that the corresponding feature layers from bottom to top in the FPN module are P3 to P7, and select the feature layer corresponding to the FPN module according to the size s of each detection frame:

[0029] If s<32, select P3 feature layer;

[0030] If 32≤s<64, select P4 feature layer;

[0031] If 64≤s<128, select the P5 feature layer;

[0032] If 128≤s<256, select the P6 feature layer;

[0033] If s ≥ 256, select the P7 feature layer.

[0034] Based on the above technical solution, preferably, the total loss function of the instance segmentation model is:

[0035]

[0036] Among them, L total is the total loss function, L cls represents the classification loss; L reg represents the detection box regression loss; L ctr-ness represents the centrality loss; L mask represents the mask prediction loss;

[0037] The expressions of various types of losses are:

[0038] L cls =-(1-p t ) γ log(p t )

[0039]

[0040] L mask =-[ylog(p)+(1-y)log(1-p)]

[0041] Among them, α, β, γ, is the weight coefficient, p is the value of each pixel of the predicted mask, and y is the value of each pixel of the real mask; p t represents the probability of being predicted as the true category; A represents the area of ​​the predicted detection box, B represents the area of ​​the true detection box; C represents the area of ​​the minimum bounding rectangle that contains both the predicted detection box and the true detection box; s i represents the predicted centrality; represents the true centrality label.

[0042] On the basis of the above technical solution, preferably, the step of calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains specifically includes:

[0043] According to the sand grain segmentation result, sand grains with complete mask information in the image are selected, pixel points in the mask of each sand grain are traversed, connected domains of each sand grain are obtained respectively, and shape features of the sand grains are calculated according to the connected domains; the shape features include projected area, longest and shortest Fret diameters and / or major and minor axes of Legendre's ellipse of inertia;

[0044] The correspondence between the pixel image size and the actual size in SI units is calibrated, and the shape characteristics of sand grains are converted from pixel size to actual size;

[0045] The equivalent diameter of each sand grain is calculated based on its actual projected area, and the shape factor of the sand grain is calculated based on the longest and shortest Fret diameters. The formula is:

[0046]

[0047] Among them, A i is the actual projected area of ​​the ith sand grain, X A,i is the equivalent diameter of the i-th sand particle, is the shape factor of the ith sand particle, are the longest and shortest Frett diameters respectively.

[0048] On the basis of the above technical solution, preferably, the mesh classification is performed according to the equivalent diameter and shape factor of the sand particles, and the number of sand particles of each mesh is counted to obtain the identification result of the particle size distribution of the raw casting sand, which specifically includes:

[0049] Based on the comparison table of test sieve and sieve hole size, determine the mesh number of the sand particles according to their equivalent diameter;

[0050] The volume of each sand grain is calculated using the equivalent diameter of each sand grain;

[0051] The total volume of sand particles corresponding to each mesh size is counted, and the particle size distribution of the original casting sand is characterized according to the percentage of the total volume of sand particles of each mesh size to the total volume of all sand particles.

[0052] In a second aspect, the present invention discloses an image recognition system for the particle size distribution of raw casting sand, the system comprising:

[0053] Data acquisition module: used to obtain the original sand particle image and perform preprocessing;

[0054] Instance segmentation module: used to introduce instance segmentation branch network into the convolutional neural network model and build an instance segmentation model; perform sand segmentation on the preprocessed original sand particle image through the pre-trained instance segmentation model;

[0055] Shape calculation module: used for calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains;

[0056] Grain size identification module: used to classify the sand particles according to their equivalent diameter and shape factor, count the number of sand particles of each mesh size, and obtain the identification result of the particle size distribution of the original foundry sand.

[0057] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0058] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0059] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0060] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1) The present invention introduces an instance segmentation branch network into the convolutional neural network model to construct an instance segmentation model, converts the target detection task into a pixel-level classification and regression problem, realizes the instance segmentation of adhered sand grains, reduces the statistical error caused by the adhesion of sand grains, and performs mesh classification and casting original sand particle size distribution statistics based on the shape characteristics of the sand grains on the basis of the instance segmentation of the sand grains, weakens the influence of the irregular shape of the sand grains on the particle size distribution statistics, and improves the accuracy of the calculation.

[0063] 2) To address the classification accuracy issue, the instance segmentation model of the present invention introduces the CBAM attention mechanism in the head network. Combining the channel attention and spatial attention mechanisms, it can enhance the feature representation capability of the detection head and improve the classification and regression accuracy.

[0064] 3) To address the problem of sand grain segmentation, the present invention introduces an instance segmentation branch network into the instance segmentation model, calculates the scale of each detection box according to the regression results of the head network, selects the feature layer of the corresponding scale of the FPN module according to the scale of each detection box, and extracts fixed-size feature blocks from the feature layer generated by FPN through the RoIAlign technology for predicting the mask. A more refined mask can be generated for each detected target, so that the model can accurately distinguish and segment the sand grains that are stuck together, thereby improving the accuracy of sand grain segmentation.

[0065] 4) Regarding the statistical problem of sand particle distribution, the present invention takes into account the irregular shape of sand particles and uses the equivalent diameter of the projected area to represent the size of the sand particles. Since classification only by equivalent particle diameter will make the overall measurement result of the image method too large and the AFS too small, the present invention adopts a method of combining equivalent diameter and shape factor classification to determine the mesh size of the sand particles, which can improve the accuracy of the sand particle mesh size classification. In addition, the present invention uses volume distribution to characterize the particle size distribution of the sand particles, which is convenient for verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 It is a schematic diagram of the image of raw sand particles with adhesion;

[0068] Figure 2 A flow chart of an image recognition method for the particle size distribution of raw casting sand according to the present invention;

[0069] Figure 3 A schematic diagram of a backlight source image acquisition device used in the present invention;

[0070] Figure 4 is a schematic diagram of the structure of the instance segmentation model of the present invention;

[0071] Figure 5 This is an example of the segmentation result of the original sand particle image of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] See also Figure 2 The present invention discloses an image recognition method for the particle size distribution of raw casting sand, the method comprising:

[0074] S1. Take the original sand particle image and perform preprocessing.

[0075] When shooting images of raw sand particles, the color, material and light intensity of the shooting background will have a great impact on the particle information. Choosing the right background and light intensity can greatly reduce the difficulty of image processing. The common method is to choose different backgrounds for different sand particles to enhance the contrast between the target particles and the background, but the disadvantage of this method is that the background needs to be changed repeatedly. The commonly used light source placement method is the top light source, where the light source and the camera are located on the same side of the sand particles. The information of the sand particles is obtained by obtaining the reflection on the surface of the sand particles. Due to the irregular shape of the sand particles, the light intensity reflected from each part is different, which may cause shadow problems.

[0076] In order to solve the background problem, the present invention adopts a bottom-mounted light source, i.e., a backlight source. Figure 3 The figure shows a schematic diagram of a backlight image acquisition device. The sand grains are blocked by the light to obtain the sand grain outline. In this way, glass with good light transmittance can be used as the background, which solves the problem of changing the background for different sand grains. At the same time, glass is a wear-resistant material, which can solve the problem of easy wear of the screen. The backlight source can make the sand grain outline clear. For the problem of light source intensity, a point light source with adjustable light intensity can be used. The light source intensity can be appropriately adjusted according to the appearance of the sand grains to obtain a clear outline of the original sand particles.

[0077] The present invention adopts Figure 3 The device shown takes images of raw sand particles and performs preprocessing.

[0078] S2. Introduce the instance segmentation branch network into the convolutional neural network model to build an instance segmentation model.

[0079] Conventional convolutional neural networks can achieve accurate positioning and classification of targets in images. However, generally only the bounding box of the target can be generated. When processing images of adhered sand grains, the adhered grains cannot be directly segmented. Since a backlight light source is used in this paper, the sand grains are used to block the light to obtain the outline of the sand grains. For transparent or nearly transparent sand grains, the outline of the sand grains is very unclear, and it is very difficult to extract the sand grain outline during image processing. In order to achieve instance segmentation of adhered sand grains and transparent sand grains, the present invention introduces an instance segmentation branch network into the convolutional neural network model to construct an instance segmentation model. Using the instance segmentation model, the target detection task can be converted into a pixel-level classification and regression problem through pixel-by-pixel prediction, thereby achieving instance segmentation of adhered sand grains and transparent sand grains.

[0080] like Figure 4The structure diagram of the instance segmentation model of the present invention is shown. The instance segmentation model includes a ResNet module, an FPN module, a head module and an instance segmentation branch network. The ResNet module, the FPN module and the head module are connected in sequence, and the input end of the instance segmentation branch network is connected to the output end of the FPN module and the head module respectively.

[0081] The ResNet module is used to extract three feature maps C3, C4, and C5 of different scales from the input image, and output the three feature maps C3, C4, and C5 of different sizes to the FPN module.

[0082] The FPN module is used to perform top-down upsampling and feature fusion on the three feature maps of different scales output by the ResNet module, and output feature layers P3 to P7 of multiple scales.

[0083] like Figure 2 As shown in the figure, the feature map C5 output by the ResNet module is subjected to a 1*1 convolution to obtain the feature layer P5. The feature map C4 output by the ResNet module is subjected to a 1*1 convolution and then fused with the P5 feature after 2 times upsampling to obtain the feature layer P4. In the process of feature fusion, since the sizes of P4 and P5 are different, the difference is exactly 2 times, so P5 must be 2 times upsampled before it can be added to P4. The upsampling uses the nearest neighbor sampling. The P4 obtained after the addition must also pass through a 3*3 convolution layer before it can be predicted. Similarly, P3 is obtained by top-down upsampling and feature fusion of C3 and P4. In addition, P5 is subjected to a 1*1 convolution to obtain the feature layer P6, and P6 is subjected to a 1*1 convolution to obtain the feature layer P7, and finally five feature layers of different scales P3 to P7 are obtained.

[0084] The head module includes multiple head networks, which are respectively used to perform target recognition and regression on feature layers of multiple scales output by the FPN module.

[0085] In an embodiment of the present invention, the head network is a fully convolutional network that introduces multiple CBAM attention mechanisms.

[0086] like Figure 4As shown, in the head module, the dotted box after the head network is the specific structure of the head network. In order to enhance the feature extraction network's ability to capture important features, the present invention reintroduces the CBAM (Convolutional Block Attention Module) attention mechanism in the detection head module, combining the two mechanisms of channel attention and spatial attention to enhance the expressiveness of feature maps. The CBAM module first selects feature channels that are important for classification and segmentation tasks through the channel attention mechanism, and then focuses on important spatial positions through the spatial attention mechanism. Through this dual attention mechanism, target detection can be performed more accurately. Figure 4 As shown, the head network includes three branches: classification, detection box regression, and center detection. The present invention adds a CBAM module to each of these three branches to obtain a head module containing three CBAM modules, which can enhance the feature representation ability of the detection head and improve the classification and regression accuracy.

[0087] The instance segmentation branch network is used to extract fixed-size feature blocks from the feature layer of the FPN module through the RoIAlign technology according to the detection results of the head module to perform mask prediction and realize instance segmentation.

[0088] It can be understood that the multi-scale feature layers P3 to P7 have different resolutions and different characterization capabilities for targets of different sizes. The lower feature layers (such as P3) have higher resolutions and smaller receptive fields, which are suitable for capturing the detailed features of small targets, while the higher-level feature layers (such as P7) have lower resolutions and larger receptive fields, which are suitable for capturing the overall features of large targets. Therefore, in order to ensure that the instance segmentation branch achieves more accurate mask prediction. The present invention selects a suitable feature layer according to the size of the detection frame, so that the feature blocks extracted by RoIAlign can accurately represent the target in the detection frame.

[0089] In an embodiment of the present invention, the instance segmentation branch network includes a feature selection unit and a mask prediction unit connected in sequence.

[0090] The feature selection unit is used to calculate the size of each detection frame according to the regression result of the head module, and select a suitable feature layer from the FPN module according to the size of each detection frame.

[0091] Specifically, first calculate the size s of each detection box:

[0092]

[0093] Among them, s represents the scale of the detection box, w and h represent the width and height of the detection box respectively;

[0094] Assume that the corresponding feature layers from bottom to top in the FPN module are P3 to P7, and select the feature layer corresponding to the FPN module according to the size s of each detection frame:

[0095] If s<32, select P3 feature layer;

[0096] If 32≤s<64, select P4 feature layer;

[0097] If 64≤s<128, select the P5 feature layer;

[0098] If 128≤s<256, select the P6 feature layer;

[0099] If s ≥ 256, select the P7 feature layer.

[0100] The mask prediction unit is used to extract a fixed-size feature block from the corresponding feature layer using the RoIAlign technology according to the selection result of the feature selection unit, perform mask prediction on the fixed-size feature block through a Mask branch including multiple convolutional layers, and generate a single-channel binary mask.

[0101] like Figure 4 As shown, the instance segmentation branch network of the present invention includes a feature selection unit and a mask prediction unit. The dotted box after the mask prediction unit is the specific structure of the mask prediction unit, which includes multiple convolutional layers. The number of channels of each convolutional layer is 256, the convolution kernel size is 3×3, the stride is 1, and the padding is 1. After each convolutional layer, a ReLU activation function is added to ensure the nonlinear expression ability of the feature. In the last layer of convolution, the number of channels is reduced from 256 to 1, and the convolution kernel size is 1×1, which is used to generate a single-channel binary mask. The reason for generating a single-channel mask is that in the head network, a judgment has been made on the category to which the detection box belongs, so the instance segmentation branch does not need to generate a multi-channel mask to judge the category when performing mask prediction, which can greatly reduce the number of model parameters and improve the calculation speed.

[0102] The main function of the instance segmentation branch introduced in the present invention is to generate a more refined mask for each detected target, so that the network can accurately distinguish and segment the sand grains that stick together. The instance segmentation branch extracts fixed-size feature blocks from the feature layer generated by FPN for predicting masks by combining RoIAlign (Region of Interest Align) technology.

[0103] S3. Perform sand grain segmentation on the preprocessed original sand particle image through the pre-trained instance segmentation model.

[0104] Before sand grain segmentation, the VGG Image Annotator software is used to annotate the captured raw sand particle images, and the raw sand particle image dataset is obtained to train the instance segmentation model.

[0105] During the training process, the total loss function of the instance segmentation model is designed by a multi-task classification method that combines classification, regression, centrality, and instance segmentation branches.

[0106] In an embodiment of the present invention, the total loss function of the instance segmentation model is:

[0107]

[0108] Among them, L total is the total loss function, L cls represents the classification loss; L reg represents the detection box regression loss; L ctr-ness represents the centrality loss; L mask represents the mask prediction loss;

[0109] The expressions of various types of losses are:

[0110] L cls =-(1-p t ) γ log(p t )

[0111]

[0112] L mask =-[ylog(p)+(1-y)log(1-p)]

[0113] Among them, α, β, γ, is the weight coefficient, p is the value of each pixel of the predicted mask, and y is the value of each pixel of the real mask; p t represents the probability of being predicted as the true category; A represents the area of ​​the predicted detection box, B represents the area of ​​the true detection box; C represents the area of ​​the minimum bounding rectangle that contains both the predicted detection box and the true detection box; s i represents the predicted centrality; represents the true centrality label.

[0114] The instance segmentation model constructed by the present invention has the advantages of fast detection speed and high accuracy, and can achieve accurate positioning and classification of targets in images.

[0115] The trained instance segmentation model network is used to predict the images of adhered sand grains and transparent sand grains. The prediction results are as follows: Figure 5As shown in the figure, (a) is the image recognition result of the adhesion sand particles, and (b) is the image recognition result of the transparent sand particles. Figure 5 It can be seen that the example segmentation method of the present invention can achieve accurate segmentation of adhered sand particles, and also has a good segmentation effect on transparent sand particles. The segmentation accuracy can meet the requirements of original sand particle size distribution detection.

[0116] S4. Calculate the shape characteristics of the sand grains according to the sand grain segmentation results, convert the shape characteristics of the sand grains from pixel size to actual size, and calculate the equivalent diameter and shape factor of the sand grains.

[0117] S41, selecting sand grains with complete mask information in the image.

[0118] According to the sand grain segmentation result obtained in step S3, the mask information and detection box information of each sand grain can be obtained. The box parameter of each sand grain is used to determine whether the sand grain is at the boundary position of the image. If the sand grain is at the boundary of the image, that is, the sand grain is cut by the image frame, the mask information of the sand grain is incomplete, so the sand grain is not in the final consideration range. In this way, the sand grains with complete mask information in the image are selected.

[0119] S42, traversing the pixel points in the mask of each sand grain, respectively obtaining the connected domain of each sand grain, and calculating the shape feature of the sand grain according to the connected domain.

[0120] The shape features include projected area, longest and shortest Frett diameters and / or major and minor axes of the Legendre ellipse of inertia.

[0121] S42. Use a film calibration plate (standard circle size is 2 mm×2 mm) to calibrate the correspondence between the pixel image size and the actual size in SI units, and convert the shape characteristics of the sand particles from the pixel size to the actual size.

[0122] The formula for converting the shape characteristics of sand grains from pixel size to actual size is:

[0123] X m =α 2 X p

[0124] Among them, α represents the actual physical size represented by the unit pixel at this magnification; X m Indicates the actual size of the sand object; X p Indicates the size of the sand object expressed in pixels.

[0125] S44. Calculate the equivalent diameter of each sand grain based on its actual projected area, and calculate the shape factor of the sand grain based on the longest and shortest Fret diameters.

[0126] Since the shape of sand particles is irregular, their size cannot be directly expressed by diameter. For non-spherical particles, the projected area equivalent diameter X is usually used. A,i To indicate the size of sand particles. For the classification of sand particles, the screening method determines the specific mesh size of the sand particles based on whether the sand particles can pass through the sieve holes. However, due to the irregular shape of the sand particles, the image method cannot directly classify them according to the equivalent particle size. Therefore, the shape factor of the sand particles must also be considered.

[0127] The present invention calculates the equivalent diameter of each sand grain according to the actual projected area of ​​each sand grain, and calculates the shape factor of the sand grain according to the longest and shortest Fret diameters, and the formula is:

[0128]

[0129] Among them, A i is the actual projected area of ​​the ith sand grain, X A,i is the equivalent diameter of the i-th sand particle, is the shape factor of the ith sand particle, are the longest and shortest Frett diameters respectively.

[0130] S5. Classify the sand particles by mesh size according to their equivalent diameter and shape factor, count the number of sand particles of each mesh size, and obtain the identification result of the particle size distribution of the original casting sand.

[0131] S51. Based on the comparison table of test sieves and sieve hole sizes, determine the mesh size of the sand particles according to their equivalent diameter.

[0132] According to the equivalent diameter X of the sand particles A,i and form factor Combined with the comparison table of medium test sieves and sieve hole sizes in the national standard "Silica Sand for Foundry" (as shown in Table 1), the sand particles are jointly classified.

[0133] Table 1 Comparison table of basic dimensions of test sieve numbers and sieve holes for casting

[0134]

[0135] If the minimum cross-section of a sand grain with a very small shape factor is smaller than the mesh aperture, it will fall to the next mesh during the actual vibration screening process. If it is classified only by the equivalent particle size, the overall measurement result of the image method will be too large, and the AFS will be too small. Therefore, the present invention adopts a method of joint classification of equivalent particle size and shape factor. First, the mesh number of the sand grain is determined according to the equivalent diameter of the sand grain, and then it is determined according to the shape factor. If the shape factor of the sand grain is less than a preset threshold (for example, 0.3), it is considered that the minimum cross-sectional area of ​​the sand grain is less than the mesh size of the current mesh, so the sand grain is classified as the mesh number corresponding to the next mesh.

[0136] S52. Count the total volume of sand particles corresponding to each mesh size, and characterize the particle size distribution of the original casting sand according to the percentage of the total volume of sand particles of each mesh size to the total volume of all sand particles.

[0137] In order to compare and verify the results obtained by the imaging method with those obtained by the mass-based sieving method, the present invention counts the volume distribution of each mesh size instead of the number distribution, and uses the volume distribution to characterize the particle size of the sand particles.

[0138] Using the equivalent diameter X of each sand particle A,i Calculate the volume V of the sand grain i , the calculation formula is:

[0139]

[0140] Then the total volume of sand particles in each mesh size is counted, and the particle size distribution is characterized according to the percentage of the volume of each mesh size in the total volume.

[0141] Finally, the AFS average fineness of the sand is calculated according to the sieve number and the corresponding fineness factor. The calculation formula is as follows:

[0142]

[0143] Among them, P n It indicates the percentage of the volume of sand particles retained on any sieve to the total volume; X n represents the fineness factor; n represents the sieve number. The sieve number and the corresponding fineness factor are shown in Table 2.

[0144] Table 2 Sieve number and corresponding fineness factor

[0145]

[0146] In view of the problems that the current screening method is greatly affected by human factors and the screen is easily worn, the present invention proposes an image recognition method for the particle size distribution of raw sand for casting, which can well identify different types of raw sand particles.

[0147] In order to compare with the results of the screening method, the particle size of dried sand, pearl sand and baked sand was tested using the method proposed in the present invention, and the volume distribution of the original sand particles in 50 pictures was counted respectively. The comparison results are shown in the following figure:

[0148] Table 3 Comparison of test results of 100 / 140 mesh baked sand screening method and image method

[0149]

[0150] Table 4 Comparison of test results of 140 mesh pearl sand screening method and image method

[0151]

[0152] Table 5 Comparison of test results of 70 / 140 mesh drying sand screening method and image method

[0153]

[0154] It can be seen from the data in Tables 3 to 5 that the particle size of several common raw sands is tested using the method of the present invention. The test results are compared with the screening method, and the absolute value error is within 3%, which can meet the test requirements.

[0155] The present invention introduces an instance segmentation branch network into a convolutional neural network model to construct an instance segmentation model, converts the target detection task into a pixel-level classification and regression problem, realizes the instance segmentation of adhered sand grains, reduces the statistical error caused by the adhesion of sand grains, and performs mesh classification and casting original sand particle size distribution statistics according to the shape characteristics of the sand grains on the basis of the instance segmentation of the sand grains, weakens the influence of the irregular shape of the sand grains on the particle size distribution statistics, and improves the accuracy of the calculation.

[0156] Corresponding to the above method embodiment, the present invention also discloses an image recognition system for the particle size distribution of raw casting sand, the system comprising:

[0157] Data acquisition module: used to obtain the original sand particle image and perform preprocessing;

[0158] Instance segmentation module: used to introduce instance segmentation branch network into the convolutional neural network model and build an instance segmentation model; perform sand segmentation on the preprocessed original sand particle image through the pre-trained instance segmentation model;

[0159] Shape calculation module: used for calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains;

[0160] Grain size identification module: used to classify the sand particles according to their equivalent diameter and shape factor, count the number of sand particles of each mesh size, and obtain the identification result of the particle size distribution of the original foundry sand.

[0161] The above system embodiments and method embodiments correspond one to one, and for a brief description of the system embodiments, please refer to the method embodiments.

[0162] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0163] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0164] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image recognition method for the particle size distribution of raw casting sand, characterized in that: The method comprises: Obtain the original sand particle image and perform preprocessing; Introduce instance segmentation branch network into the convolutional neural network model to build an instance segmentation model; The pre-processed original sand particle image is segmented by using the pre-trained instance segmentation model; Calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains; The sand particles are classified by mesh size according to their equivalent diameter and shape factor, and the number of sand particles of each mesh size is counted to obtain the particle size distribution identification result of the raw casting sand; The instance segmentation model includes a ResNet module, an FPN module, a head module and an instance segmentation branch network; The ResNet module, the FPN module, and the head module are connected in sequence, and the input end of the instance segmentation branch network is connected to the output ends of the FPN module and the head module respectively; The ResNet module is used to extract three feature maps of different scales from the input image, and output the three feature maps of different sizes to the FPN module; The FPN module is used to perform top-down upsampling and feature fusion on the three feature maps of different scales output by the ResNet module, and output feature layers of multiple scales; The head module includes multiple head networks, which are respectively used to perform target recognition and regression on the feature layers of multiple scales output by the FPN module; the head network is a fully convolutional network that introduces multiple CBAM attention mechanisms; The instance segmentation branch network is used to extract a fixed-size feature block from the feature layer of the FPN module through the RoIAlign technology according to the detection result of the head module to perform mask prediction and realize instance segmentation; The step of calculating the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains specifically includes: According to the sand grain segmentation result, sand grains with complete mask information in the image are selected, pixel points in the mask of each sand grain are traversed, connected domains of each sand grain are obtained respectively, and shape features of the sand grains are calculated according to the connected domains; the shape features include projected area, longest and shortest Fret diameters and / or major and minor axes of Legendre's ellipse of inertia; The correspondence between the pixel image size and the actual size in SI units is calibrated, and the shape characteristics of sand grains are converted from pixel size to actual size; The equivalent diameter of each sand grain is calculated based on its actual projected area, and the shape factor of the sand grain is calculated based on the longest and shortest Fret diameters. The formula is: Among them, A i is the actual projected area of ​​the ith sand grain, X A,i is the equivalent diameter of the i-th sand particle, is the shape factor of the ith sand particle, are the longest and shortest Frett diameters, respectively; The method of classifying the sand particles by mesh size according to the equivalent diameter and shape factor, counting the number of sand particles of each mesh size, and obtaining the particle size distribution identification result of the raw casting sand specifically includes: Based on the comparison table of test sieve and sieve hole size, determine the mesh number of the sand particles according to their equivalent diameter; Using the equivalent diameter X of each sand particle A,i Calculate the volume V of the sand grain i ; The total volume of sand particles corresponding to each mesh size is counted, and the particle size distribution of the original casting sand is characterized according to the percentage of the total volume of sand particles of each mesh size to the total volume of all sand particles.

2. The image recognition method for the particle size distribution of raw casting sand according to claim 1, characterized in that: The instance segmentation branch network includes a feature selection unit and a mask prediction unit connected in sequence; The feature selection unit is used to calculate the size of each detection frame according to the regression result of the head module, and select the feature layer corresponding to the FPN module according to the size of each detection frame; The mask prediction unit is used to extract a fixed-size feature block from the corresponding feature layer using the RoIAlign technology according to the selection result of the feature selection unit, perform mask prediction on the fixed-size feature block through a Mask branch including multiple convolutional layers, and generate a single-channel binary mask.

3. The image recognition method for the particle size distribution of raw casting sand according to claim 2, characterized in that: Calculating the size of each detection frame according to the regression result of the head module, and selecting the feature layer corresponding to the FPN module according to the size of each detection frame specifically includes: Calculate the size s of each detection box: Among them, s represents the scale of the detection box, w and h represent the width and height of the detection box respectively; Assume that the corresponding feature layers from bottom to top in the FPN module are P3 to P7, and select the feature layer corresponding to the FPN module according to the size s of each detection frame: If s<32, select P3 feature layer; If 32≤s<64, select P4 feature layer; If 64≤s<128, select the P5 feature layer; If 128≤s<256, select the P6 feature layer; If s ≥ 256, select the P7 feature layer.

4. The image recognition method for the particle size distribution of raw casting sand according to claim 1, characterized in that: The total loss function of the instance segmentation model is: Among them, L total is the total loss function, L cls represents the classification loss; L reg represents the detection box regression loss; L ctr-ness represents the centrality loss; L mask represents the mask prediction loss; The expressions of various types of losses are: L cls =-(1-p t ) γ log(p t ) L mask =-[ylog(p)+(1-y)log(1-p)] Among them, α, β, γ, is the weight coefficient, p is the value of each pixel of the predicted mask, and y is the value of each pixel of the real mask; p t represents the probability of being predicted as the true category; A represents the area of ​​the predicted detection box, B represents the area of ​​the true detection box; C represents the area of ​​the minimum bounding rectangle that contains both the predicted detection box and the true detection box; s i represents the predicted centrality; represents the true centrality label.

5. An image recognition system for the particle size distribution of raw casting sand, characterized in that: The system comprises: Data acquisition module: used to obtain the original sand particle image and perform preprocessing; Instance segmentation module: used to introduce an instance segmentation branch network into the convolutional neural network model to construct an instance segmentation model; perform sand segmentation on the preprocessed original sand particle image through the pre-trained instance segmentation model; the instance segmentation model includes a ResNet module, an FPN module, a head module and an instance segmentation branch network; The ResNet module, the FPN module, and the head module are connected in sequence, and the input end of the instance segmentation branch network is connected to the output ends of the FPN module and the head module respectively; The ResNet module is used to extract three feature maps of different scales from the input image, and output the three feature maps of different sizes to the FPN module; The FPN module is used to perform top-down upsampling and feature fusion on the three feature maps of different scales output by the ResNet module, and output feature layers of multiple scales; The head module includes multiple head networks, which are respectively used to perform target recognition and regression on the feature layers of multiple scales output by the FPN module; the head network is a fully convolutional network that introduces multiple CBAM attention mechanisms; The instance segmentation branch network is used to extract a fixed-size feature block from the feature layer of the FPN module through the RoIAlign technology according to the detection result of the head module to perform mask prediction and realize instance segmentation; Shape calculation module: used to calculate the shape characteristics of sand grains according to the sand grain segmentation results, convert the shape characteristics of the sand grains from pixel size to actual size, and calculate the equivalent diameter and shape factor of the sand grains; the calculation of the shape characteristics of the sand grains according to the sand grain segmentation results, converting the shape characteristics of the sand grains from pixel size to actual size, and calculating the equivalent diameter and shape factor of the sand grains specifically include: According to the sand grain segmentation result, sand grains with complete mask information in the image are selected, pixel points in the mask of each sand grain are traversed, connected domains of each sand grain are obtained respectively, and shape features of the sand grains are calculated according to the connected domains; the shape features include projected area, longest and shortest Fret diameters and / or major and minor axes of Legendre's ellipse of inertia; The correspondence between the pixel image size and the actual size in SI units is calibrated, and the shape characteristics of sand grains are converted from pixel size to actual size; The equivalent diameter of each sand grain is calculated based on its actual projected area, and the shape factor of the sand grain is calculated based on the longest and shortest Fret diameters. The formula is: Among them, A i is the actual projected area of ​​the ith sand grain, X A,i is the equivalent diameter of the i-th sand particle, is the shape factor of the ith sand particle, are the longest and shortest Frett diameters, respectively; The particle size identification module is used to classify the sand particles by mesh size according to the equivalent diameter and shape factor, count the number of sand particles of each mesh size, and obtain the particle size distribution identification result of the raw casting sand; the method of classifying the sand particles by mesh size according to the equivalent diameter and shape factor, counting the number of sand particles of each mesh size, and obtaining the particle size distribution identification result of the raw casting sand specifically includes: Based on the comparison table of test sieve and sieve hole size, determine the mesh number of the sand particles according to their equivalent diameter; Using the equivalent diameter X of each sand particle A,i Calculate the volume V of the sand grain i ; The total volume of sand particles corresponding to each mesh size is counted, and the particle size distribution of the original casting sand is characterized according to the percentage of the total volume of sand particles of each mesh size to the total volume of all sand particles.

6. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to claim 1.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions cause a computer to implement the method of claim 1.

Images