Intelligent green pellet detection system

Abstract

The intelligent green pellet detection system comprises an image acquisition module, a green pellet particle size detection module, a green pellet strength detection module and a field data real-time display module, the data acquisition module is used for acquiring pellet image information; the green pellet particle size detection module is used for calculating green pellet particle size information based on the pellet image information; the green pellet strength detection module is used for calculating green pellet strength information based on the green pellet particle size information; and the field data real-time display module is used for outputting the green pellet strength detection information to a display interface. According to the invention, accurate and effective detection of the particle size and the strength of the pellet in the pellet image is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of pelletization detection, and in particular relates to an intelligent pelletization detection system. Background Art

[0002] At present, the pelletizing production line of the pelletizing plant mainly adopts the process flow of chain grate machine-rotary kiln. Among them, the pelletizing production process is an important process of the production line, and a stable pelletizing production process is of great significance to improving the quality and output of raw balls. Stable and precise control of the pelletizing process is an important consideration for the economic and technical indicators of the pelletizing plant. Therefore, the production of stable and high-quality raw balls has become a key technical link in the pelletizing production process. At present, the main parameters of the pelletizing production process still rely on the experience and feeling of on-site operators for subjective adjustment, which is unreliable, unstable, and inefficient. Although most of the existing pelletizing machines are equipped with a series of control devices, they are still at a low level of manual adjustment of single parameter values. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention proposes an intelligent detection system for pellets, which realizes accurate and effective detection of the particle size and strength of pellets in the pellet image.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A pellet raw ball intelligent detection system, comprising: an image acquisition module, a raw ball particle size detection module, a raw ball strength detection module and a field data real-time display module;

[0006] The data acquisition module is used to collect pellet image information;

[0007] The green ball particle size detection module is used to calculate the green ball particle size information based on the pellet image information;

[0008] The green ball strength detection module is used to calculate the green ball strength information based on the green ball particle size information;

[0009] The field data real-time display module is used to output the ball strength detection information to the display interface.

[0010] Preferably, the image acquisition module comprises: a light source, a camera and an image acquisition card;

[0011] The light source is used to select the appropriate light source type and arrangement according to the color, material and reflective properties of the pellets;

[0012] The camera includes a camera and a lens for capturing clear and stable pellet image information;

[0013] The image acquisition card is used to transmit the pellet image information.

[0014] Preferably, the green ball particle size detection module includes: an image preprocessing unit, a green ball segmentation unit and a particle size measurement unit;

[0015] The image preprocessing unit is used to filter and remove noise from the pellet image information;

[0016] The raw ball segmentation unit is used to perform image segmentation, raw ball area filling and incomplete raw balls on the pellet image information after filtering and denoising, so as to obtain a single sphere segmentation map;

[0017] The particle size determination unit is used to calculate the cross-sectional area of ​​the raw ball and the statistical particle size distribution based on the single sphere segmentation diagram.

[0018] Preferably, the raw ball segmentation unit comprises: a preliminary ball segmentation subunit, a morphological processing subunit and a monomer ball segmentation subunit;

[0019] The pellet preliminary segmentation subunit is used to perform preliminary segmentation of pellets using an improved U-net semantic segmentation network to obtain a binary image;

[0020] The morphological processing subunit is used to perform morphological processing on the segmented binary image to remove impurities and fill holes;

[0021] The single sphere segmentation subunit is used to perform distance transformation, find the labeling map of each area in the image according to the image after the distance transformation, and then perform label-based watershed segmentation to obtain a single sphere segmentation map.

[0022] Preferably, the particle size determination unit comprises: an image marking subunit, a region boundary perimeter calculation subunit, a material ball rejection subunit and a particle size detection subunit;

[0023] The image marking subunit is used to mark the boundaries of all the ball regions in the single sphere segmentation image with different serial numbers, and then mark the pixels inside each ball region with the same number as the region boundary;

[0024] The area boundary perimeter calculation subunit is used to count the sum of the boundary pixels of each ball area as the perimeter of the area boundary;

[0025] The ball rejection subunit is used to calculate the circularity of the ball contour based on the perimeter of the region boundary, and to reject the pellets with a circularity less than a set value as incomplete balls;

[0026] The particle size detection subunit is used to calculate the cross-sectional area of ​​the raw balls whose roundness meets preset requirements and to statistically analyze the particle size distribution.

[0027] Preferably, the green ball strength detection module comprises: a green ball rupture state assessment unit, a green ball crack detection unit and a green ball strength calculation unit;

[0028] The green ball rupture state evaluation unit is used to perform circle fitting on a single ball that meets the particle size requirements, evaluate the fit between the segmented monomer and the fitting circle, and obtain the green ball rupture state result;

[0029] The green ball crack detection unit is used to obtain a green ball crack detection result based on a green ball in a non-cracked state by using a YOLOv5s-CBE green ball crack detection model;

[0030] The green ball strength calculation unit is used to count the number of times a corresponding number of green balls fall before cracks appear on the green balls, so as to obtain the batch green ball strength.

[0031] Preferably, the YOLOv5s-CBE green ball crack detection model includes:

[0032] The CA attention module is added to the backbone C3 module and between C3 and SPPF to capture channel relationship and position information from both channel and spatial dimensions;

[0033] In the Neck part of YOLOv5s, a weighted bidirectional feature pyramid structure is used to fuse information of different scales to obtain an output feature map with rich semantic information. At the same time, the bidirectional feature fusion pyramid structure adjusts the contribution of input feature maps of different scales to the output by introducing weights to optimize the feature fusion effect.

[0034] Change the loss function CloU in the original YOLOv5s to EloU.

[0035] Preferably, the on-site data real-time display module further includes: a monitoring unit and an alarm unit;

[0036] The monitoring unit is used to monitor various data in the green ball production process in real time;

[0037] The alarm unit is used to send out an alarm signal when the data is abnormal.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention collects pellet raw ball images and pre-processes the collected pellet raw ball images; performs image segmentation, raw ball area filling and incomplete raw balls elimination on the pre-processed pellet raw ball images to obtain a single sphere segmentation map; based on the single sphere segmentation map, the cross-sectional area of ​​the raw balls and the statistical particle size distribution are calculated. A circle fitting is performed on the single ball that meets the particle size requirements, and the fit between the segmented monomer and the fitting circle is evaluated to obtain the raw ball rupture state result; based on the raw balls in the non-rupture state, the YOLOv5s-CBE raw ball crack detection model is used to obtain the raw ball crack detection result; if cracks appear in the raw balls, the number of drops of the corresponding number of raw balls before cracks appear is counted to obtain the batch raw ball strength. Thus, accurate and effective detection of the particle size and strength of the pellets in the pellet image is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of the process of a pelletizing intelligent detection system according to an embodiment of the present invention;

[0042] Figure 2 A diagram of a system network topology according to an embodiment of the present invention;

[0043] Figure 3 A flowchart of evaluating the rupture state of a green ball according to an embodiment of the present invention;

[0044] Figure 4 It is a structural diagram of the DSC-Unet model of an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of pellet segmentation using the DSC-Unet algorithm according to an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of the pellet distance transformation according to an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of mask marking of a sticky ball mass according to an embodiment of the present invention;

[0048] Figure 8 A schematic diagram showing a comparison of the profiles of a single pellet according to an embodiment of the present invention;

[0049] Fig. 9 A contour circumscribed circle search diagram according to an embodiment of the present invention;

[0050] Fig.10A schematic diagram of a minimum circumscribed circle search process according to an embodiment of the present invention;

[0051] Fig.11 The following is a schematic diagram of the operation of the Focus module according to an embodiment of the present invention;

[0052] Fig.12 This is a schematic diagram of a C3 module according to an embodiment of the present invention;

[0053] Fig.13 This is a schematic diagram of the Resunit structure of an embodiment of the present invention;

[0054] Fig.14 Schematic diagram of the structure of C3_X and C3X_F according to an embodiment of the present invention;

[0055] Fig.15 It is a schematic diagram of the SPPF structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] Embodiment 1

[0059] like Figure 1 As shown, the present invention provides a pellet raw ball intelligent detection system, including: an image acquisition module, a raw ball particle size detection module, a raw ball strength detection module and a field data real-time display module;

[0060] The data acquisition module is used to collect pellet image information;

[0061] The green ball particle size detection module is used to calculate the green ball particle size information based on the pellet image information;

[0062] The ball strength detection module is used to calculate the ball strength information based on the ball particle size information;

[0063] The real-time display module of field data is used to output the ball strength detection information to the display interface.

[0064] like Figure 2As shown, this embodiment discloses a system network topology structure, which is used to implement the ball-forming intelligent detection system of the present invention. The specific implementation process is as follows:

[0065] In this embodiment, the image acquisition module includes: a light source, a camera and an image acquisition card;

[0066] The light source is used to select the appropriate light source type and arrangement according to the color, material and reflective characteristics of the pellets;

[0067] The camera includes a camera and a lens for capturing clear and stable pellet image information;

[0068] The image acquisition card is used to transmit the pellet image information.

[0069] Specifically, the image acquisition module transmits the collected green ball image information to the image processing host (including: green ball particle size detection module and green ball strength detection module) for analysis and processing, so the collected image information is directly related to the subsequent image processing. Furthermore, if the quality of the collected image is too poor, it may lead to a series of defects such as poor accuracy and slow speed of the entire image detection system. Therefore, for the green ball strength detection system, a good image acquisition module is essential.

[0070] The working process of the image acquisition module is: the raw ball is placed in the ball drop plate by using an electromagnet. After the electromagnet moves into place, the control system sends a collection signal to trigger the industrial camera to collect the raw ball image information, and at the same time send the collected image information to the image processing host. It mainly consists of the following parts: lighting source, industrial camera, lens, etc. Each part complements each other, is independent of each other, but also interconnected.

[0071] The surface light source is one of the important components of the image acquisition module. A good lighting source can well highlight the features of the object, present the edge details of the object, and clearly distinguish the target features of the object from the background, which is convenient for subsequent image processing. In addition, a good light source should also avoid interference from external factors, such as the basic properties of the object, the working properties of the workshop, and the influence of external light. In view of the characteristics of LED light sources such as long service life and stable light intensity, the present invention selects LED light sources.

[0072] According to the installation orientation of the light source and its light distribution, the lighting methods of the light source can be divided into backlighting, general lighting, continuous diffuse reflection lighting, area lighting, coaxial lighting, etc.

[0073] The strength test of green balls mainly detects whether there are cracks on the surface of green balls, and secondly, it is necessary to measure the size of green balls through industrial cameras. According to the shape characteristics of green balls, dome-shaped shadowless light sources are more suitable for this scene. After diffuse reflection from the spherical surface, the light is smoothly and evenly irradiated on the object to be tested. It is suitable for objects with uneven and reflective surfaces, and is suitable for detecting curved surfaces, metal or non-metal objects with uneven surfaces.

[0074] The camera is an indispensable component in the image acquisition module. If the selected camera is not appropriate, the collected raw ball image information may have large noise and low precision. Therefore, choosing a suitable camera is a link that cannot be ignored. The raw ball image collected by the image acquisition module in the present invention not only needs to calculate the particle size information of the raw ball based on the image, but also needs to judge whether there are cracks in the free-falling raw ball based on the image. Compared with the surface of the raw ball, the depth of field difference is small, so the camera sensitivity is the primary factor to be considered in this system. Secondly, in order to facilitate the subsequent image analysis and processing, this system should choose a camera with smaller noise points. Finally, good real-time performance and high precision are the primary requirements of this system. Therefore, from the perspective of speed and resolution, the CMOS camera is the best choice for this system.

[0075] The two basic principles of the green ball detection system are accuracy and stability, and the principles of the image acquisition module are speed and precision. Therefore, this system needs to select a suitable camera based on parameters such as resolution, noise and speed. According to the actual needs of the workshop, the specific indicators that this system should meet are:

[0076] Detection ball diameter range: 8-16mm

[0077] Maximum speed: 0.1 seconds / time

[0078] Detection accuracy: 0.5mm

[0079] Recognition rate: ≥98%

[0080] According to the above indicators, the field of view resolution of the camera should be greater than 1920*1080, and the frame rate of the camera should be greater than 25 frames per second. This system intends to use a high-resolution industrial camera.

[0081] In the image acquisition module, the lighting source and the camera are both extremely important and irreplaceable parts. However, if you want to obtain clearer images, better contrast and less noise, a suitable lens is also necessary. Therefore, choosing a suitable lens is also one of the important tasks of this system.

[0082] The focal length of the lens is an important parameter of the lens, which can reflect the ability of the lens to resolve details after imaging. It is mainly affected by diffraction and aberrations. When the aperture is reduced, the diffraction is heavier and the resolution is reduced, but the aberration will also be reduced. At the same time, reducing the aperture can increase the depth of field.

[0083] The formula for calculating the focal length of a lens is:

[0084] ,

[0085] Among them, f represents the focal length of the lens, v represents the lateral size of the pixel, h represents the longitudinal size of the pixel, V represents the lateral size of the object to be measured, H represents the longitudinal size of the object to be measured, and D represents the working distance of the lens. According to the above formula, when the working distance D of the lens is constant, the lateral field of view V and the longitudinal field of view H are inversely proportional to the focal length f of the lens. The larger the focal length of the lens, the smaller the field of view and the higher the image resolution; the smaller the focal length of the lens, the larger the field of view and the lower the image resolution. In addition, the raw ball to be measured is a spherical object. When detecting cracks, it is necessary to observe whether there are cracks on the surface around the raw ball. Therefore, when the number of cameras is fixed and there is only one, it is a more appropriate choice to choose a 360-degree catadioptric lens to shoot the surrounding images and the top image.

[0086] In this embodiment, the iPlat platform uses Chengchao to intelligently control the existing platform system and server system of the iPlat center. Baowu Industrial Internet Platform iPlat is an important part of Baowu's new generation ecological technology platform system (ePlat+iPlat) independently developed by Baosight Software, and is professionally oriented to support smart manufacturing application scenarios.

[0087] The main functions include:

[0088] Data collection, governance and services: Support multi-source heterogeneous data collection, data processing and alignment, data governance and data services.

[0089] Production process monitoring and video AI: Unified monitoring management functions provide 2D, 3D, and video seamlessly integrated monitoring images to achieve a full range of immersive control experience; video AI applications help factory production and safety.

[0090] Production process control: Enhanced process control function, dynamic tracking of production process, organic combination of mechanism model and mathematical model, in-depth optimization of production process control.

[0091] Manufacturing Execution Management: Use digital intelligent applications to empower manufacturing execution processes, achieve online optimization of production and processes, and establish a dynamic factory management and control system.

[0092] Multi-source data fusion and innovative applications: Provide multi-source data fusion and aggregation for process, equipment, quality, energy and cost, and provide data-based innovative application platform tools to empower production management and production control.

[0093] In this embodiment, the green ball particle size detection module includes: an image preprocessing unit, a green ball segmentation unit and a particle size measurement unit;

[0094] The image preprocessing unit is used to filter and denoise the pellet image information; specifically, the preprocessing grayscales the collected image, filters and denoises the grayscale image, enhances the image, increases the contrast between the target pellet and the background, and makes the pellet edge appear as clearly as possible.

[0095] The raw ball segmentation unit is used to perform image segmentation, raw ball area filling and incomplete raw balls on the pellet image information after filtering and denoising, so as to obtain a single sphere segmentation map;

[0096] The particle size measurement unit is used to calculate the cross-sectional area of ​​the raw ball and the statistical particle size distribution based on the single sphere segmentation diagram.

[0097] like Figure 3 As shown, the raw ball segmentation unit includes: a ball preliminary segmentation subunit, a morphological processing subunit and a monomer ball segmentation subunit;

[0098] The ball cluster preliminary segmentation subunit is used to perform preliminary segmentation of the ball cluster using an improved U-net semantic segmentation network to obtain a binary image;

[0099] The morphological processing subunit is used to perform morphological processing on the segmented binary image to remove impurities and fill holes;

[0100] The single sphere segmentation subunit is used to perform distance transformation, find the label map of each area in the image according to the image after distance transformation, and then perform label-based watershed segmentation to obtain a single sphere segmentation map.

[0101] Specifically, the broken balls after being beaten will stick together with the complete balls, and there is a certain degree of occlusion between the balls. It is difficult to directly segment a single raw ball from the image. Therefore, it is necessary to first obtain the complete balls and eliminate background interference. This system combines the high-resolution characteristics of U-Net and the lightweight characteristics of the DSC module to construct the DSC-Unet network. The network integrates the separable convolution module, improves the feature extraction part, reduces the network complexity, and improves the segmentation efficiency.

[0102] like Figure 4As shown, the improved U-Net network still uses the symmetrical structure of encoding and decoding. The encoding uses the lightweight DSC network structure, and the decoding still uses the U-net decoding structure. Compared with the traditional U-Net network, this network mainly uses the DSC module to replace the traditional convolution module in the feature extraction part, so that the network parameter volume is reduced from the original 13.33MB to the current 0.95MB; at the same time, the BN layer and ReLU activation function are used after the deep convolution and point-by-point convolution, in order to avoid the gradient disappearance of the network during training and speed up the network training; finally, in the fifth layer of the feature extraction network, the DSC module with a step size of 1 is used 5 times in succession to make up for the loss caused by the semantic segmentation network, while enhancing the network's ability to extract features and increasing the network segmentation accuracy, so that the network model achieves a better balance between accuracy and real-time performance. As shown Figure 5 As shown in the figure, the pellet segmentation result using the DSC-Unet algorithm. It can be seen that the formed pellets can be completely segmented and the edges are relatively smooth. However, there are still some adhesion phenomena. Therefore, this system uses a watershed adhesion segmentation algorithm to further segment the pellets.

[0103] Since the balls are sticky to each other and there are over-segmented tiny particles, the segmentation map needs to be processed in the next step. First, a morphological opening operation is performed to remove isolated tiny particles and eliminate the "burrs" around the target pixels. At the same time, the Euclidean distance transformation algorithm is used to effectively separate the sticky balls that are in contact with each other through an internal labeling method to solve the problem of over-segmentation of balls.

[0104] Perform Euclidean distance transformation on the adhesion balls to obtain the distance image. The distance calculation formula is as follows:

[0105] ,

[0106] ,

[0107] ,

[0108] Among them, i, j represent the foreground image coordinates; x, y represent the background image coordinates; M, N represent the image has M rows and N columns. According to the characteristic that the farther the pixel in the distance image is from the background pixel, the brighter the image is, the approximate position of the highlighted pixel exceeding a certain threshold in the adhesion image can be easily determined. The ball cluster image through distance transformation is as follows Figure 6 shown.

[0109] Then, the connected region is introduced to mark the region of the binary image of the ball-cluster distance. A connected region generally refers to an image region composed of foreground pixels with the same pixel value and adjacent positions in the image, and connected region analysis refers to finding and marking each connected region in the image. For each connected region found, a unique identifier is given to it to distinguish it from other connected regions, and all connected regions of the image can be obtained.

[0110] The mask mark is made by marking the image through morphological operations, distance images and connected region analysis. First, determine the real background, real foreground, and both background and foreground areas in the image. Then mark the connected regions to obtain the watershed mask mark, such as Figure 7 shown.

[0111] The particle size determination unit includes: an image marking subunit, an area boundary perimeter calculation subunit, a material ball rejection subunit and a particle size detection subunit;

[0112] The image marking subunit is used to mark the boundaries of all the ball regions in the single sphere segmentation image with different serial numbers, and then mark the pixels inside each ball region with the same number as the region boundary;

[0113] The area boundary perimeter calculation subunit is used to count the sum of the boundary pixels of each ball area as the perimeter of the area boundary;

[0114] The ball rejection subunit is used to calculate the roundness of the ball contour based on the perimeter of the area boundary, and to reject the pellets with a roundness less than the set value as incomplete balls;

[0115] The particle size detection subunit is used to calculate the cross-sectional area of ​​raw balls whose roundness meets the preset requirements and to statistically analyze the particle size distribution.

[0116] Specifically, the detection process is as follows: Use area marking to track the boundaries of the ball area, mark the boundaries of all ball areas in the image with different serial numbers, and then mark the pixels inside each ball area with the same number as the area boundary. After marking, count the sum of the boundary pixels of each ball area as the perimeter of the area boundary. Calculate the circularity of the ball outline, and remove the ball with a circularity less than the set value as an incomplete ball. Assuming that each block in the image is circular, after obtaining the surface area of ​​each block, use the formula Get its approximate radius; use the calculated approximate radius to classify the block and classify it as a raw material of a certain particle size level; calculate the average particle size after calculating the particle size of all blocks, that is, the mass of a single block multiplied by the sum of the particle sizes and then divided by the total mass; count the average particle size of all material blocks within a certain period of time, and add the area value of each material block to the corresponding distribution interval, and then calculate the percentage of each distribution interval.

[0117] In this embodiment, the green ball strength detection module includes: a green ball rupture state evaluation unit, a green ball crack detection unit and a green ball strength calculation unit;

[0118] The green ball rupture state evaluation unit is used to perform circle fitting on a single ball that meets the particle size requirements, evaluate the fit between the segmented monomer and the fitting circle, and obtain the green ball rupture state result;

[0119] The green ball crack detection unit is used to obtain the green ball crack detection results based on the green ball in a non-cracked state using the YOLOv5s-CBE green ball crack detection model;

[0120] The green ball strength calculation unit is used to count the number of times a corresponding number of green balls fall before cracks appear on the green balls, so as to obtain the batch green ball strength.

[0121] Specifically, damage assessment based on circularity calculation: After a single pellet is segmented, the shape of the single pellet can be evaluated. A complete pellet is spherical and close to a circle in a two-dimensional image, while a damaged pellet has a defective edge contour due to the lack of material, and the circularity will be much worse (e.g. Figure 8 ). Therefore, we can use the ball roundness to evaluate the pellet defect.

[0122] 1) Minimum circumscribed circle fitting of pellets

[0123] After binarizing the pellets, find the edge and consider using the minimum circumscribed circle method to extract the pellet contour features, such as Fig. 9 shown.

[0124] The working principle of the minimum circumscribed circle algorithm:

[0125] (1) Randomly select three points A, B, and C on the edge of the contour to fit a minimum circumscribed circle MinR{A, B, C}, and find the point farthest from the center of the circle;

[0126] (2) Determine whether the point is on the circle or inside the circle. If the point is on the circle or inside the circle, the circumscribed circle MinR{A,B,C} is the required minimum circle. If the point is outside the circle, then randomly select three points from the four points A, B, and C to fit a circle R{A,B,C}. At the same time, find the point j farthest from the circle R{A,B,C}. If the point j is on the circle or inside the circle, the circle R{A,B,C} is the required minimum circle.

[0127] (3) If point j is not on the circle R{A,B,C}, repeat the second step until the minimum circumscribed circle of all points on the fitting contour is found. ;

[0128] 2) Circularity calculation

[0129] After the minimum circumscribed circle of the pellet is found, the ratio of the pellet area to the minimum circumscribed circle area is used as the circularity evaluation standard.

[0130] , where A is the area of ​​the pellet (in pixels), which can be obtained using the traversal method; r is the radius of the circumscribed circle. This method can be used to evaluate the damage state of the pellet, such as Fig.10 shown.

[0131] Specifically, industrial inspection is a crucial task in the production process. By conducting real-time inspection of products and timely discovering and removing defective products, production efficiency can be improved, defective product rates can be reduced, costs can be reduced, and product quality can be ensured. Traditional industrial inspection usually relies on manual inspection or specific sensor equipment. These methods often have problems such as slow detection speed, low accuracy, and susceptibility to environmental influences. Therefore, the use of computer vision technology for industrial inspection has great potential and application value.

[0132] YOLOv5 (You Only Look Once version5) is a very popular real-time object detection algorithm. Compared with other object detection algorithms, YOLOv5 has higher detection speed and lower latency while maintaining higher accuracy. This makes YOLOv5 an ideal choice for industrial detection scenarios and can meet the needs of real-time detection on production lines.

[0133] In this system, we will use YOLOv5 to detect crack images in a green ball drop test. First, we need to prepare an image dataset containing these defects. Then, we will train a YOLOv5 model to detect these defects, and evaluate and optimize the model. Finally, we will discuss how to deploy the trained model into an actual production environment.

[0134] The methods for green ball crack detection have the problems of poor versatility, low precision and low recall rate. Since green ball crack detection needs to be real-time in practical applications, YOLOv5s is the version with the least parameters and calculation in the YOLOv5 basic model, so this system is based on the YOLOv5s algorithm with the smallest volume in the YOLOv5-6.1 version for research and improvement, and the YOLOv5s-CBE green ball crack detection network is proposed. The specific contents are as follows:

[0135] (1) The CA attention (Coordinate Attention) module is added to the backbone C3 module and between C3 and SPPF (Spatial Pyramid Pooling-Fast) to capture channel relationship and position information from both channel and spatial dimensions, thereby improving the feature extraction capability of the YOLOv5s backbone network.

[0136] (2) In the Neck part of YOLOv5s, a weighted bidirectional feature pyramid network (BidirectionalFeaturePyramid Network, BiFPN) is used to fuse information of different scales to obtain output feature maps with rich semantic information. At the same time, the bidirectional feature fusion pyramid structure adjusts the contribution of input feature maps of different scales to the output by introducing weights to optimize the feature fusion effect.

[0137] (3) The loss function CloU in the original YOLOv5s is changed to EloU. EloU not only considers the center point distance and aspect ratio, but also considers the actual difference in width and height between the predicted box and the true box, thus improving the prediction accuracy of the anchor box.

[0138] First, the YOLOv5 input can perform Mosaic data enhancement on the dataset. This data enhancement method uses 4 pictures to splice in a random scaling, random cropping, and random arrangement manner to enrich the dataset. In particular, the random scaling function adds a lot of small targets, improves the network's ability to detect small objects, and makes the network more robust. Secondly, YOLOv5 can also adaptively generate various prediction boxes based on the original anchor box during model training, and use NMS (NonMaximumSuppression) to select the prediction box closest to the real box. Finally, in order to adapt to input pictures of different sizes, YOLOv5's adaptive image scaling function allows the picture to be scaled to the appropriate size before entering the network, avoiding problems such as mismatch between feature tensors and fully connected layers.

[0139] Backbone

[0140] The YOLOv5 backbone uses the Focus module, improved Bottleneck CSP and spatial pyramid pooling (SPP) structures to extract the feature information of the image. The YOLOv5-6.1 version used in this system is different from YOLOv5 in structure. Fig.11 As shown in the figure, the Backbone of YOLOv5-6.1 version is mainly divided into Conv, C3 and SPPF.

[0141] The Focus module divides the input RGB image into 4 parts and performs two downsamplings. The Concat module concatenates them in the channel dimension to obtain a 12-dimensional feature map, which is then sent to the Convolution module for feature extraction using a 3x3 convolution kernel to obtain a 32-dimensional feature layer. Focus downsampling reduces the size of the feature map by increasing its dimension without sacrificing any information and improves the performance of the network. In the YOLOv5-6.1 version, a convolution kernel with a size of 6x6, a stride of 2, and a padding of 2 is used to replace the Focus module, which is more efficient and convenient for model export and deployment.

[0142] Early BottleneckCSP borrowed the idea of ​​CSPNet and applied it to the DarkNet53 backbone network.

[0143] The main purpose of CSPNet is to reduce the amount of calculation and improve the reasoning speed without reducing the accuracy of model detection and recognition. Its main idea is to split the gradient flow so that it propagates through different network paths. Through operations such as splicing and transition, richer gradient combination information can be achieved.

[0144] In this system, the C3 module is used to replace the BottleneckCSP module in the YOLOv5-6.1 version. Fig.12 As shown in Figure 1, the C3 module contains three standard convolutional layers and multiple Resunit modules. The difference between the C3 module and the BottleneckCSP module is that the Conv module after the output of the Resunit module is removed.

[0145] The Resunit structure in the C3 module draws on the residual idea of ​​ResNet, such as Fig.13 As shown in the figure, one path first performs a 1x1 convolution to reduce the number of channels of the feature map by 1 / 2, thereby reducing the amount of calculation, and then extracts features through a 3x3 convolution and doubles the number of channels, so that the number of channels of the input and output does not change. The other path performs a residual connection through a shortcut and adds it to the output feature map of the first path to achieve feature fusion.

[0146] YOLOv5-6.1 contains two different types of C3 modules, C3_X and C3._X_F. Fig.14 As shown in the figure, the two structures have basically the same functions, but they differ in the choice of correction units. C3_X is used in the backbone network to increase the gradient value of back propagation between layers, thereby obtaining a finer feature granularity. In Neck, the C3_X_F structure is used to improve the feature fusion ability of the network by inserting two residual blocks, allowing the network to reduce the amount of calculation while maintaining richer feature information.

[0147] YOLOv5-6.1 version uses SPPF module to replace SPP (Spatial Pyramid Pooling) module. SPP can convert feature maps of any size into feature vectors of fixed size without reducing the recognition accuracy of the image. SPPF module uses multiple small-sized pooling kernels in cascade to replace the single large-sized pooling kernel in SPP module, thereby further improving the running speed while retaining the original function, that is, fusing feature maps of different receptive fields and enriching the expression ability of feature maps. The structure of SPPF is as follows Fig.15 shown.

[0148] Neck

[0149] The Neck layer mainly includes Feature Pyramid Network (FPN) and Path Aggregation Network (PAN). FPN transmits deep semantic features from top to bottom, while PAN transmits the location information of the target from bottom to top. The model transmits the feature information of objects of different sizes by fusing top-down and bottom-up feature information, solving the problem of multi-scale object detection.

[0150] Output

[0151] The head part includes bounding box regression loss, confidence loss and classification loss. Among them, the bounding box regression loss function uses CIOU_ Loss, which fully considers the three important geometric factors of overlapping area, center point distance and aspect ratio, improves some occluded and overlapping targets, and greatly improves the speed and accuracy of prediction box regression. YOLOv5 uses weighted NMS to filter multiple target anchor boxes and eliminates redundant candidate boxes in the post-processing of target detection.

[0152] The YOLOv5-6.1 target detection algorithm brings together the advantages of multiple algorithm networks to achieve a high level of detection speed and accuracy, and has the advantages of convenient format conversion and easier deployment. However, since the YOLOv5 experiment is mainly designed for natural scene data sets, such as the MS COCO data set, its detection targets are too broad and there is still room for improvement in the field of raw ball detection. For this reason, this system uses the smallest YOLOv5s in the YOLOv5-6.1 series as the basic algorithm, and optimizes it through a series of experiments to obtain a target detection algorithm that is more suitable for raw ball detection.

[0153] In this embodiment, this system is used in conjunction with a raw ball grabbing mechanical arm to assemble into an automatic measurement device for the falling strength of raw balls, which mainly includes a six-axis mechanical arm, a ball drop plate, and a raw ball crack visual detection system developed by this system. The function of the mechanical arm is equivalent to that of a human hand. A raw ball suction cup is installed at the end of the mechanical arm, which can lift the raw ball to 0.5 meters and then drop the raw ball. Then the raw ball is taken out from the ball drop plate and sent to the bottom of the camera. The camera captures the front and back images of the raw ball by rotating the raw ball. Compared with traditional transportation methods such as rails, the raw ball is sucked by the raw ball suction cup and transported by the mechanical arm, and will not collide with other objects, so no cracks will be generated during the transportation process. The main function of the ball drop plate is to catch the raw ball falling from the mechanical arm. The bottom of the ball drop plate is a steel plate with a thickness of 10mm, which meets the requirements of the raw ball falling strength measurement standard. The function of the raw ball crack automatic detection system developed by this system is equivalent to that of the human eye, which can determine whether cracks appear after the raw ball falls.

[0154] (2) The drop strength measurement process based on the ball crack automatic detection system of this system

[0155] In the actual production process, the drop strength of the same batch of green balls is determined by the following method: 10 green balls are tested each time, and the number of drops when cracks first appear on each green ball is counted. The average number of drops of the 10 green balls is the drop strength of this batch of green balls. Therefore, the drop strength of green balls measured by this system is:

[0156] (a) The robot arm picks up a green ball from the green ball pile, lifts it to 50 cm and drops it into the ball drop tray;

[0157] (b) The robotic arm takes the raw ball out of the ball drop tray and moves it under the camera;

[0158] (c) The camera takes a picture of the front of the raw ball, and then the software developed by this system is used to detect cracks. If the front detection result is "cracked", the drop intensity of the ball is recorded as 1, and the above detection process is repeated for the next raw ball. If the front detection result of the raw ball is "no crack", the robot arm twists the end and takes a picture of the back of the raw ball. If the back detection result is "cracked", the drop intensity of the raw ball is recorded as 1, and the next raw ball is tested. If the back detection result is "no crack", the robot arm moves to the top of the ball drop plate and drops for the second time; repeat the above process until cracks appear, and record the total number of drops of the raw ball, which is the drop intensity of the raw ball.

[0159] (d) Repeat the process (a)-(c) and finally take the average of the drop strengths of the 10 green balls and output it as the drop strength of the batch of green balls.

[0160] (3) The measurement results of the drop strength of green balls were obtained by using the above-mentioned automatic drop strength detection device and detection process. A total of 3 groups of experiments were carried out. In each group of experiments, 10 green balls were randomly selected from the same batch of green ball samples for drop strength measurement, and the automatic measurement results of the drop strength of green balls were obtained.

[0161] In this embodiment, the on-site data real-time display module further includes: a monitoring unit and an alarm unit;

[0162] The monitoring unit is used to monitor various data in the green ball production process in real time;

[0163] The alarm unit is used to send out an alarm signal when the data is abnormal.

[0164] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A pelleting intelligent detection system, characterized in that: include: Image acquisition module, ball size detection module, ball strength detection module and field data real-time display module; The data acquisition module is used to collect pellet image information; The green ball particle size detection module is used to calculate the green ball particle size information based on the pellet image information; The green ball strength detection module is used to calculate the green ball strength information based on the green ball particle size information; The field data real-time display module is used to output the ball strength detection information to the display interface.

2. The pelleting intelligent detection system according to claim 1 is characterized in that: The image acquisition module includes: a light source, a camera and an image acquisition card; The light source is used to select the appropriate light source type and arrangement according to the color, material and reflective properties of the pellets; The camera includes a camera and a lens for capturing clear and stable pellet image information; The image acquisition card is used to transmit the pellet image information.

3. The pelleting intelligent detection system according to claim 1 is characterized in that: The green ball particle size detection module includes: an image preprocessing unit, a green ball segmentation unit and a particle size measurement unit; The image preprocessing unit is used to filter and remove noise from the pellet image information; The raw ball segmentation unit is used to perform image segmentation, raw ball area filling and incomplete raw balls on the pellet image information after filtering and denoising, so as to obtain a single sphere segmentation map; The particle size determination unit is used to calculate the cross-sectional area of ​​the raw ball and the statistical particle size distribution based on the single sphere segmentation diagram.

4. The pelletizing intelligent detection system according to claim 3 is characterized in that: The raw ball segmentation unit includes: a ball preliminary segmentation subunit, a morphological processing subunit and a monomer ball segmentation subunit; The pellet preliminary segmentation subunit is used to perform preliminary segmentation of pellets using an improved U-net semantic segmentation network to obtain a binary image; The morphological processing subunit is used to perform morphological processing on the segmented binary image to remove impurities and fill holes; The single sphere segmentation subunit is used to perform distance transformation, find the labeling map of each area in the image according to the image after the distance transformation, and then perform label-based watershed segmentation to obtain a single sphere segmentation map.

5. The pelletizing intelligent detection system according to claim 3 is characterized in that: The particle size determination unit includes: an image marking subunit, an area boundary perimeter calculation subunit, a material ball rejection subunit and a particle size detection subunit; The image marking subunit is used to mark the boundaries of all the ball regions in the single sphere segmentation image with different serial numbers, and then mark the pixels inside each ball region with the same number as the region boundary; The area boundary perimeter calculation subunit is used to count the sum of the boundary pixels of each ball area as the perimeter of the area boundary; The ball rejection subunit is used to calculate the circularity of the ball contour based on the perimeter of the region boundary, and to reject the pellets with a circularity less than a set value as incomplete balls; The particle size detection subunit is used to calculate the cross-sectional area of ​​the raw balls whose roundness meets preset requirements and to statistically analyze the particle size distribution.

6. The pelleting intelligent detection system according to claim 1 is characterized in that: The green ball strength detection module includes: a green ball rupture state evaluation unit, a green ball crack detection unit and a green ball strength calculation unit; The green ball rupture state evaluation unit is used to perform circle fitting on a single ball that meets the particle size requirements, evaluate the fit between the segmented monomer and the fitting circle, and obtain the green ball rupture state result; The green ball crack detection unit is used to obtain a green ball crack detection result based on a green ball in a non-cracked state by using a YOLOv5s-CBE green ball crack detection model; The green ball strength calculation unit is used to count the number of times a corresponding number of green balls fall before cracks appear on the green balls, so as to obtain the batch green ball strength.

7. The pelleting intelligent detection system according to claim 6 is characterized in that: The YOLOv5s-CBE green ball crack detection model includes: The CA attention module is added to the backbone C3 module and between C3 and SPPF to capture channel relationship and position information from both channel and spatial dimensions; In the Neck part of YOLOv5s, a weighted bidirectional feature pyramid structure is used to fuse information of different scales to obtain an output feature map with rich semantic information. At the same time, the bidirectional feature fusion pyramid structure adjusts the contribution of input feature maps of different scales to the output by introducing weights to optimize the feature fusion effect. Change the loss function CloU in the original YOLOv5s to EloU.

8. The pelleting intelligent detection system according to claim 1 is characterized in that: The on-site data real-time display module also includes: a monitoring unit and an alarm unit; The monitoring unit is used to monitor various data in the green ball production process in real time; The alarm unit is used to send out an alarm signal when the data is abnormal.

Images