An online measurement method for particle fluid indicators in the material drum disassembly and crushing process

Abstract

The present invention discloses an online measurement method for particle flow indicators in a rotary drum disassembling and crushing process of materials. By installing a camera, images of the disassembling and crushing process are obtained online during the operation of the rotary drum disassembling and crushing equipment without stopping the equipment. In addition, step S2 extracts a particle area image, which is divided into multiple particle area units and particle area unit contour data is obtained in step S3. Step 4 performs adhesion particle separation processing. In addition, step S2 extracts particle area images of two consecutive frames of disassembling and crushing process images, and step S5 uses a pyramid structure Siamese network model to obtain a dense displacement vector field and then calculates it. In this way, the following are obtained by online calculation: particle size distribution data of the rotor of the rotary drum disassembling and crushing equipment at the moment of shooting the disassembling and crushing process images, and a particle flow velocity field of the rotor of the rotary drum disassembling and crushing equipment at the moment of shooting the two consecutive frames of the disassembling and crushing process images. The method has the advantages of high precision and high speed.

Description

Technical Field

[0001] The invention relates to particle fluid index measurement of rotary drum dismantling and crushing equipment, in particular to an online measurement method for particle fluid index in a rotary drum dismantling and crushing process of materials. Background Art

[0002] Disassembly and crushing are key processes in the recycling, chemical, and metallurgical industries, and can be achieved using a rotary drum disassembly and crushing process. This process has a direct impact on subsequent resource utilization and also affects energy consumption and equipment wear during production. Therefore, by online acquiring key fluid indicators, such as particle size distribution and particle velocity field, that reflect the flow characteristics of granular fluids during the disassembly and crushing process, it is possible to adjust equipment operating parameters in a timely manner. This is of great significance and value in improving product disassembly and crushing efficiency, reducing production costs, and enhancing the economic benefits of the enterprise.

[0003] Currently, online measurement of disassembly and crushing process parameters can be achieved through related technologies such as machine vision, which has the characteristics of high speed and online operation. However, there is currently a lack of end-to-end online measurement solutions for key indicators of particle flow (including particle size distribution and velocity field). Among them, the existing particle flow visual measurement methods are mainly implemented through morphological calculations, which have problems such as light sensitivity, misidentification of adhesion and overlapping particles, and large influence of particle shape, resulting in low measurement accuracy; when using traditional PIV methods for velocity field measurement, there is poor generalization ability and parameters need to be adjusted according to the input image. At the same time, because its perception accuracy is mostly at the pixel level and follows the principle of light conservation, it is more sensitive to the color and brightness of the object being measured, resulting in low measurement accuracy. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an online measurement method for particle fluid indicators in a material rotary drum disassembly and crushing process.

[0005] To solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] An online measurement method for particle fluid indicators in a drum-type material disassembly and crushing process, applicable to drum-type disassembly and crushing equipment that achieves material disassembly and crushing through drum rotation;

[0007] The invention is characterized by comprising:

[0008] Step S1, see Figure 3 During the operation of the rotary drum disassembling and crushing equipment, a camera is used to continuously shoot the interior of the drum to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles formed by disassembling and crushing in the drum;

[0009] The drum port is preferably covered with a transparent baffle, allowing the camera to capture the interior of the drum at an angle parallel to the drum's axis of rotation, thereby reducing the algorithmic complexity of subsequent image processing of the disassembly and crushing process. Preferably, the camera is mounted 1.5 meters in front of the transparent baffle, and the camera's pose correction and intrinsic parameter calibration are completed using the camera's SDK software.

[0010] Among them, the frame rate of the camera should be high enough to ensure the accuracy of the disassembly and crushing process image. The frame rate is generally required to be no less than 150fps, the resolution is generally 640×480, and under indoor daily light conditions, the exposure time is preferably 5000μs, and the gain range is preferably 12.0 to 18.0.

[0011] Step S2, see Figure 4 , using a data processing device to acquire the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image, that is, the area composed of all particles in the disassembly and crushing process image, to obtain a particle area image;

[0012] Step S3, see Figure 6 and Figure 7 , dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, wherein the contour of each particle region unit is continuous and closed, and any two particle region units are not connected to each other;

[0013] Among them, step S3 can be implemented using the existing U-shaped lightweight deep network for semantic segmentation.

[0014] Among them, the particle area unit in step S3 includes two cases: one is a particle area unit formed by only one independent particle; the other is a particle area unit formed by the adhesion of at least two particles; and for the particle area unit located at the edge of the particle area image and tightly fitted with the inner wall of the rotating drum, its contour portion is composed of the arc line of the inner wall of the rotating drum.

[0015] Step S4: Perform adhesion particle separation processing on each particle area unit obtained in step S3 to obtain the independent particle area contour and particle center point coordinate information corresponding to each particle in the disassembly and crushing process image, for example: Figure 8 for Figure 4 Two independent particle regions are obtained after the two particles adhering to each other in the particle region image are processed by steps S3 and S4; and the least squares ellipse fitting algorithm is used in combination with the particle center point coordinate information to complete the fitting of the corresponding independent particle region contour, so as to obtain the particle size of each particle in the disassembly and crushing process image, that is, the particle size distribution data of the drum of the rotary drum disassembly and crushing equipment at the moment of shooting the disassembly and crushing process image.

[0016] Therefore, in steps S1 to S4 of the present invention, by installing a camera with a high frame rate, during the operation of the rotary drum disassembling and crushing equipment, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment, and by extracting the particle area image in step S2, dividing it into multiple particle area units and obtaining particle area unit contour data in step S3, and performing adhesion particle separation processing in step 4, the particle size distribution data of the rotor of the rotary drum disassembling and crushing equipment at the time of shooting the disassembling and crushing process image is obtained by online calculation. This has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment.

[0017] Among them, the particle size distribution data can be used to conduct online analysis of the mixing status of particles in the drum, so as to make timely and reasonable adjustments to the working parameters such as the rotation speed of the drum, so that the particles are fully mixed in the drum, and provide strong support for optimizing the disassembly and crushing process of the drum-type disassembly and crushing equipment and improving work efficiency and production efficiency.

[0018] Preferred: See Figure 5 In step S2, the method of extracting the particle area from the disassembly and crushing process image includes:

[0019] Step S2-1, processing the disassembly and crushing process image by using a K-means clustering algorithm to distinguish the drum area from the area outside the drum in the disassembly and crushing process image, so as to extract the drum area in the disassembly and crushing process image;

[0020] Step S2-2, extracting the fuzzy area where the particles are roughly located from the drum area using an Alpha contour extraction algorithm;

[0021] Step S2-3, extracting the region where the particles are more accurately located from the fuzzy region where the particles are located by using the Felzenszwalb image segmentation algorithm;

[0022] Step S2-4: using closed region detection and connected region detection algorithms, eliminating external interference other than the particles in the region where the particles are located, to obtain the particle region;

[0023] Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

[0024] Preferably, step S3 is implemented using a lightweight deep semantic segmentation network (LUnet). This solves the problem of illumination interference when segmenting a particle region image into multiple particle region units, thereby improving segmentation accuracy. Furthermore, since the lightweight deep semantic segmentation network (LUnet) has fewer network parameters, the processing time of step S3 can be reduced, solving the problem of time-consuming particle fluid index measurement in the prior art.

[0025] Preferred: See Figure 9 In step S4, the method of performing the adhesion particle separation process on any particle region unit includes:

[0026] Step S4-1, using an exclusive-OR operation to process the particle region unit and its particle region unit contour data, so as to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to a particle;

[0027] Among them, for a particle region unit formed by only one independent particle, the particle region unit itself is an independent particle region; for a particle region unit formed by the adhesion of N ≥ 2 particles, the particle region unit is separated into N independent particle regions with discontinuous and closed contours, for example: Figure 8 for Figure 7 The particle area unit shown is separated into two independent particle areas.

[0028] Step S4-2: using a connected domain detection algorithm to process each of the independent particle regions to remove the interference of tiny particles;

[0029] Step S4-3: Obtain the independent particle region contour of each independent particle region processed in step S4-2, and use a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

[0030] Among them, for the independent particle area separated by the particle area unit formed by only one independent particle, its independent particle area contour is the continuous closed contour of the particle; for the independent particle area separated by the particle area unit formed by the adhesion of N≥2 particles, its independent particle area contour is the discontinuous closed arc contour, for example: Figure 8 The discontinuous closed arc outlines of two independent particle regions are shown.

[0031] Therefore, steps S4-1 to S4-3 of the present invention can perform adhesion particle separation processing on any particle area unit, and can solve the problem of easy misidentification caused by densely stacked particles in the rotating drum, so as to improve the accuracy of particle size distribution data.

[0032] As a preferred embodiment of the present invention: the online measurement method for particle fluid indicators in the material rotary drum disassembly and crushing process also includes:

[0033] Step S5: First, obtain particle region images of two consecutive frames of disassembly and crushing process images through step S2; then, process the particle region images of the two consecutive frames of disassembly and crushing process images using a pyramid-structured Siamese network model to obtain a dense displacement vector field of all pixel points in the particle region image; finally, perform a dot product calculation using the dense displacement vector field and the particle center point coordinate information obtained in step S4, and, in combination with the camera frame rate, obtain: the particle flow velocity field of the rotary drum of the rotary drum disassembly and crushing equipment at the time when the two consecutive frames of the disassembly and crushing process images were captured;

[0034] And, as Figure 10 As shown in FIG, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

[0035] Among them, see Figure 10 The feature extraction module adopts a dual pyramid structure network consisting of two pyramid structures, wherein each pyramid structure is composed of four convolutional layers cascaded in sequence (i.e., the first convolutional layer Conv+BN1, the second convolutional layer Conv+BN2, the third convolutional layer Conv+BN3, and the fourth convolutional layer Conv+BN4), and each convolutional layer is composed of a convolution module Conv with a kernel of 3×3 and a batch normalization module BN in series, that is, the image data input to the convolutional layer is first convolved by the convolution module Conv and then BN calculated by the batch normalization module BN, and the obtained regularized image features are then sent to the next convolutional layer for processing; and the particle area images (I1, I2) of the two consecutive frames of the disassembly and fragmentation process images are respectively input into the first convolutional layer Conv+BN1 of the two pyramid structures, and the fourth convolutional layer Conv+BN4 of the two pyramid structures respectively output a 4-level feature matrix sequence corresponding to the two particle area images (I1, I2).

[0036] In addition, the regularized image features output by each convolutional layer are upsampled to the size of the original input image (i.e., the particle area image), added and merged, and then sent to the backward transfer module for processing.

[0037] Among them, see Figure 11The correlation calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), three composite layers consisting of a 3×3 kernel convolution module (Conv 3×3) and a ReLU function module in series, a convolution layer Conv 7×7 with a 7×7 kernel, a convolution layer Conv 5×5 with a 5×5 kernel, and two convolution layers Conv 3×3 with a 3×3 kernel, which are connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), and the output of the convolution layer Conv 3×3 with a 3×3 kernel, which is the last in the series relationship, is used as the output result of the correlation calculation module.

[0038] Among them, see Figure 12 The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module (Conv3×3) and a ReLU function module in series, a convolution layer Conv 3×3 with a kernel of 3×3, a convolution layer Conv 2×2 with a kernel of 2×2, and two convolution layers Conv 1×1 with a kernel of 1×1 in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels (Conv3×3Ch=514) in parallel; and the output result of the relevant calculation module is input into the parallel Conv convolution layer, and the output of the convolution layer Conv1×1 with a kernel of 1×1, which is the last in the series relationship, is used as the output result of the sub-pixel module.

[0039] Among them, see Figure 13 , the regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit calculation result by the series unit, then inputs the series unit calculation result into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the series unit calculation result, the activation function calculation result and the front-end input feature matrix by the multiplication result as the output result of the regularization module; wherein, the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules in series, 1 convolution layer Conv 7×7 with a kernel of 7×7, 1 convolution layer Conv 5×5 with a kernel of 5×5, and 2 convolution layers Conv3×3 with a kernel of 3×3 in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels (Conv 3×3Ch=131) and 2 convolution layers with 3×3 kernels and 259 channels (Conv 3×3Ch=259) are connected in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

[0040] The weighted fusion module combines the four-level feature map data output by the regularization module into one-level map data as the output result through a weighted fusion calculation method according to the following formula, which is the dense displacement vector field of all pixels in the particle area image;

[0041]

[0042] Where, X Fus is the output result of the weighted fusion module, namely the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Indicates Reg l The preset weighted values ​​are generally preferred to be the first to fourth levels W l Set to 12.7, 5.5, 4.39, 3.9 in sequence; Reg l Represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

[0043] Therefore, in steps S1, S2, and S5 of the present invention, by installing a camera with a high frame rate, images of the disassembling and crushing process are obtained online during the operation of the rotary drum disassembling and crushing equipment without stopping the rotary drum disassembling and crushing equipment. In addition, by extracting particle area images of two consecutive frames of the disassembling and crushing process images in step S2 and obtaining a dense displacement vector field using a pyramid structure Siamese network model in step S5 and then performing calculation, the particle flow velocity field of the rotor of the rotary drum disassembling and crushing equipment at the moment of shooting the two consecutive frames of the disassembling and crushing process images is obtained by online calculation. This method has the advantages of high precision and high speed, and can fully ensure the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment.

[0044] Among them, the particle flow velocity field can be used to conduct online analysis of the mixing status of the particles in the rotating drum, so as to make reasonable adjustments to the working parameters such as the rotating speed of the rotating drum in a timely manner, so that the particles are fully mixed in the rotating drum, and provide strong support for optimizing the disassembly and crushing process of the rotary drum disassembly and crushing equipment and improving the efficiency of the cyclic disassembly work.

[0045] Furthermore, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a correlation calculation module, which can solve the interference of illumination on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0046] In addition, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the pyramid structure Siamese network model has fewer network parameters, the processing time of step S5 can be reduced, solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0047] Moreover, step S5 of the present invention obtains a dense displacement vector field through a pyramid-structured Siamese network model. Since the pyramid-structured Siamese network model introduces a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles of different shapes, colors, and scales, and solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] First, in steps S1 to S4 of the present invention, by installing a camera 2 with a high frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, images of the disassembling and crushing process are obtained online without stopping the rotary drum disassembling and crushing equipment 1. In addition, by extracting the particle area image in step S2, dividing it into multiple particle area units and obtaining particle area unit contour data in step S3, and performing adhesion particle separation processing in step 4, the particle size distribution data of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of capturing the disassembling and crushing process image are obtained by online calculation. This method has the advantages of high precision and high speed, and can fully ensure the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0050] Second, in steps S1, S2, and S5 of the present invention, by installing a camera 2 with a relatively high frame rate, images of the disassembling and crushing process are obtained online during the operation of the rotary drum disassembling and crushing equipment 1 without the need to shut down the rotary drum disassembling and crushing equipment 1. Furthermore, by extracting particle area images of two consecutive frames of the disassembling and crushing process images in step S2 and obtaining a dense displacement vector field using a pyramid-structured Siamese network model in step S5 and then performing calculations, the particle flow velocity field of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the moment of shooting two consecutive frames of the disassembling and crushing process images is obtained by online calculation. This method has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0051] Third, step S3 of the present invention is implemented by a lightweight deep semantic segmentation network LUnet, which can solve the illumination interference problem that exists when segmenting a particle region image into multiple particle region units, thereby improving the segmentation accuracy.

[0052] Fourth, steps S4-1 to S4-3 of the present invention can separate the adhering particles in any particle area unit, and can solve the problem of easy misidentification caused by densely stacked particles 3 in the drum 1-1, thereby improving the accuracy of the particle size distribution data.

[0053] Fifth, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a correlation calculation module, which can solve the interference of illumination on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0054] Sixth, step S3 of the present invention is implemented through the lightweight deep semantic segmentation network LUnet, and step S5 obtains a dense displacement vector field through the pyramid structure Siamese network model. Since their network parameters are relatively small, the processing time of steps S3 and S5 can be reduced, solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0055] Seventh, step S3 of the present invention is implemented by the lightweight deep semantic segmentation network LUnet, and step S5 obtains the dense displacement vector field through the pyramid structure Siamese network model. Since they both introduce a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles 3 of different shapes, colors, and scales, which solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0056] In summary, the present invention realizes the online measurement of two particle flow indicators, namely, particle size distribution data and particle flow velocity field. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0058] Figure 1 It is a schematic diagram of the process of the present invention;

[0059] Figure 2 Schematic diagram of the arrangement of cameras in step S1 of the present invention;

[0060] Figure 3 The image of the disassembly and crushing process obtained in step S1 of the present invention;

[0061] Figure 4 The particle area image extracted in step S2 of the present invention;

[0062] Figure 5 This is a flowchart of extracting particle area images in step S2 of the present invention;

[0063] Figure 6 for Figure 4Two particles are stuck to each other in the particle area image shown;

[0064] Figure 7 Step S3 of the present invention is Figure 4 After processing the particle area image shown in the figure, the corresponding Figure 6 Particle area units of the two particles shown;

[0065] Figure 8 For step S4-1 of the present invention, Figure 7 The two independent particle regions are obtained by separating the particle region unit shown;

[0066] Figure 9 This is a flow chart of the step S4 of the present invention for separating the adhered particles in the particle area unit;

[0067] Figure 10 This is a schematic diagram of the network structure of the pyramid structure Siamese network model in step S5 of the present invention;

[0068] Figure 11 Schematic diagram of the network structure of the relevant computing modules of the pyramid structure Siamese network model in the present invention;

[0069] Figure 12 Schematic diagram of the network structure of the sub-pixel module of the pyramid structure Siamese network model in the present invention;

[0070] Figure 13 Schematic diagram of the network structure of the regularization module of the pyramid-structured Siamese network model in the present invention;

[0071] In the figure, represents upsampling, Indicates addition, Indicates multiplication. DETAILED DESCRIPTION

[0072] The present invention is described in detail below in conjunction with the embodiments and the accompanying drawings to help those skilled in the art better understand the inventive concept of the present invention. However, the scope of protection of the claims of the present invention is not limited to the following embodiments. For those skilled in the art, all other embodiments obtained without creative work without departing from the inventive concept of the present invention shall fall within the scope of protection of the present invention.

[0073] Example 1

[0074] like Figures 1 to 9 As shown, the present invention discloses an online measurement method for particle fluid indicators in a material drum disassembly and crushing process, which is applicable to a drum disassembly and crushing device 1 that realizes material disassembly and crushing by rotating a drum 1-1;

[0075] include:

[0076] Step S1, see Figure 3 During the operation of the drum-type disassembling and crushing equipment 1, the interior of the drum 1-1 is continuously photographed by the camera 2 to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles 3 formed by disassembling and crushing in the drum 1-1;

[0077] The drum port of the drum 1-1 is preferably covered with a transparent baffle 1-2, allowing the camera 2 to capture the interior of the drum 1-1 at an angle parallel to the rotation axis of the drum 1-1, thereby reducing the algorithm complexity of the subsequent image processing of the disassembly and crushing process. Preferably, the camera 2 is installed 1.5 meters in front of the transparent baffle 1-2, and the camera 2's pose correction and internal parameter calibration are completed using the SDK software provided with the camera 2.

[0078] Among them, the frame rate of the camera 2 should be high enough to ensure the accuracy of the disassembly and crushing process image. The frame rate is generally required to be no less than 150fps, the resolution is generally 640×480, and under indoor daily light conditions, the exposure time is preferably 5000μs, and the gain range is preferably 12.0 to 18.0.

[0079] Step S2, see Figure 4 , using a data processing device 4 to obtain the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image, that is, the area composed of all particles 3 in the disassembly and crushing process image, to obtain a particle area image;

[0080] Step S3, see Figure 6 and Figure 7 , dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, wherein the contour of each particle region unit is continuous and closed, and any two particle region units are not connected to each other;

[0081] Among them, step S3 can be implemented using the existing U-shaped lightweight deep network for semantic segmentation.

[0082] Among them, the particle area unit in step S3 includes two cases: one is a particle area unit formed independently by only one particle 3; the other is a particle area unit formed by the adhesion of at least two particles 3; and for the particle area unit located at the edge of the particle area image and tightly fitted with the inner wall of the rotating drum 1-1, its contour part is composed of the arc line of the inner wall of the rotating drum 1-1.

[0083] Step S4: Perform adhesion particle separation processing on each particle area unit obtained in step S3 to obtain the independent particle area contour and particle center point coordinate information corresponding to each particle 3 in the disassembly and crushing process image, for example: Figure 8 for Figure 4 Two particles 3 adhering to each other in the particle area image shown are two independent particle areas obtained after processing in steps S3 and S4; and the least squares ellipse fitting algorithm is used in combination with the particle center point coordinate information to complete the fitting of the corresponding independent particle area contour, so as to obtain the particle size of each particle 3 in the disassembly and crushing process image, that is, the particle size distribution data of the drum 1-1 of the rotary drum disassembly and crushing equipment 1 at the time of shooting the disassembly and crushing process image.

[0084] Therefore, in steps S1 to S4 of the present invention, by installing a camera 2 with a high frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment 1, and by extracting the particle area image in step S2, dividing it into multiple particle area units and obtaining particle area unit contour data in step S3, and performing adhesion particle separation processing in step 4, the particle size distribution data of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of shooting the disassembling and crushing process image is obtained by online calculation. This has the advantages of high precision and high speed, and can fully ensure the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0085] Among them, the particle size distribution data can be used to conduct online analysis of the mixing status of the particles 3 in the drum 1-1, so as to make reasonable adjustments to the working parameters such as the rotation speed of the drum 1-1 in a timely manner, so that the particles 3 are fully mixed in the drum 1-1, and provide strong support for optimizing the disassembly and crushing process of the drum-type disassembly and crushing equipment 1 and improving the efficiency of the circulating disassembly work.

[0086] The above is a basic implementation of the first embodiment of the present invention. Further optimization, improvement and limitation can be made based on this basic implementation:

[0087] Preferred: See Figure 5 In step S2, the method of extracting the particle area from the disassembly and crushing process image includes:

[0088] Step S2-1, processing the disassembly and crushing process image by using a K-means clustering algorithm to distinguish the drum area from the area outside the drum in the disassembly and crushing process image, so as to extract the drum area in the disassembly and crushing process image;

[0089] Step S2-2: using an Alpha contour extraction algorithm, extracting a fuzzy region where the particle 3 is approximately located from the drum region;

[0090] Step S2-3, using the Felzenszwalb image segmentation algorithm, extracting the region where the particle 3 is more accurately located from the fuzzy region where the particle is located;

[0091] Step S2-4: Eliminate external interference except for particle 3 in the region where the particle is located by using closed region detection and connected region detection algorithms to obtain the particle region;

[0092] Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

[0093] Preferably, step S3 is implemented using a lightweight deep semantic segmentation network (LUnet). This solves the problem of illumination interference when segmenting a particle region image into multiple particle region units, thereby improving segmentation accuracy. Furthermore, since the lightweight deep semantic segmentation network (LUnet) has fewer network parameters, the processing time of step S3 can be reduced, solving the problem of time-consuming particle fluid index measurement in the prior art.

[0094] Preferred: See Figure 9 In step S4, the method of performing the adhesion particle separation process on any particle region unit includes:

[0095] Step S4-1, using an exclusive OR operation to process the particle region unit and its particle region unit contour data, so as to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to a particle 3;

[0096] Among them, for a particle region unit formed by only one particle 3, the particle region unit itself is an independent particle region; for a particle region unit formed by the adhesion of N ≥ 2 particles 3, the particle region unit is separated into N independent particle regions with discontinuous and closed contours, for example: Figure 8 for Figure 7 The particle area unit shown is separated into two independent particle areas.

[0097] Step S4-2: using a connected domain detection algorithm to process each of the independent particle regions to remove the interference of tiny particles;

[0098] Step S4-3: Obtain the independent particle region contour of each independent particle region processed in step S4-2, and use a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

[0099] Among them, for the independent particle area separated by the particle area unit formed by only one particle 3, its independent particle area contour is the continuous closed contour of the particle 3; for the independent particle area separated by the particle area unit formed by the adhesion of N≥2 particles 3, its independent particle area contour is the discontinuous closed arc contour, for example: Figure 8 The discontinuous closed arc outlines of two independent particle regions are shown.

[0100] Therefore, steps S4-1 to S4-3 of the present invention can perform adhesion particle separation processing on any particle area unit, and can solve the problem of easy misidentification caused by dense stacking of particles 3 in the drum 1-1, so as to improve the accuracy of particle size distribution data.

[0101] Example 2

[0102] On the basis of the above-mentioned embodiment 1, this embodiment 2 further adopts the following preferred implementation manner:

[0103] The method for online measurement of particle fluid indicators in the material rotary drum disassembly and crushing process also includes:

[0104] Step S5: First, obtain particle region images of two consecutive frames of disassembly and crushing process images through step S2; then, process the particle region images of the two consecutive frames of disassembly and crushing process images using a pyramid-structured Siamese network model to obtain a dense displacement vector field of all pixel points in the particle region image; finally, perform a dot product calculation using the dense displacement vector field and the particle center point coordinate information obtained in step S4, and, in conjunction with the frame rate of the camera 2, obtain: the particle flow velocity field of the drum 1-1 of the drum-type disassembly and crushing equipment 1 at the time when the two consecutive frames of the disassembly and crushing process images were captured;

[0105] And, as Figure 10 As shown in FIG, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

[0106] Among them, see Figure 10The feature extraction module adopts a dual pyramid structure network consisting of two pyramid structures, wherein each pyramid structure is composed of four convolutional layers cascaded in sequence (i.e., the first convolutional layer Conv+BN1, the second convolutional layer Conv+BN2, the third convolutional layer Conv+BN3, and the fourth convolutional layer Conv+BN4), and each convolutional layer is composed of a convolution module Conv with a kernel of 3×3 and a batch normalization module BN in series, that is, the image data input to the convolutional layer is first convolved by the convolution module Conv and then BN calculated by the batch normalization module BN, and the obtained regularized image features are then sent to the next convolutional layer for processing; and the particle area images I1 and I2 of the two consecutive frames of the disassembly and fragmentation process images are respectively input into the first convolutional layer Conv+BN1 of the two pyramid structures, and the fourth convolutional layer Conv+BN4 of the two pyramid structures respectively output a 4-level feature matrix sequence corresponding to the two particle area images I1 and I2.

[0107] In addition, the regularized image features output by each convolutional layer are upsampled to the size of the original input image (i.e., the particle area image), added and merged, and then sent to the backward transfer module for processing.

[0108] Among them, see Figure 11 The correlation calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), three composite layers consisting of a 3×3 kernel convolution module (Conv 3×3) and a ReLU function module in series, a convolution layer Conv 7×7 with a 7×7 kernel, a convolution layer Conv 5×5 with a 5×5 kernel, and two convolution layers Conv 3×3 with a 3×3 kernel, which are connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), and the output of the convolution layer Conv 3×3 with a 3×3 kernel, which is the last in the series relationship, is used as the output result of the correlation calculation module.

[0109] Among them, see Figure 12The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module (Conv3×3) and a ReLU function module in series, a convolution layer Conv 3×3 with a kernel of 3×3, a convolution layer Conv 2×2 with a kernel of 2×2, and two convolution layers Conv 1×1 with a kernel of 1×1 in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels (Conv3×3Ch=514) in parallel; and the output result of the relevant calculation module is input into the parallel Conv convolution layer, and the output of the convolution layer Conv1×1 with a kernel of 1×1, which is the last in the series relationship, is used as the output result of the sub-pixel module.

[0110] Among them, see Figure 13 , the regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit calculation result by the series unit, then inputs the series unit calculation result into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the series unit calculation result, the activation function calculation result and the front-end input feature matrix by the multiplication result as the output result of the regularization module; wherein, the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules in series, 1 convolution layer Conv 7×7 with a kernel of 7×7, 1 convolution layer Conv 5×5 with a kernel of 5×5, and 2 convolution layers Conv3×3 with a kernel of 3×3 in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels (Conv 3×3Ch=131) and 2 convolution layers with 3×3 kernels and 259 channels (Conv 3×3Ch=259) are connected in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

[0111] The weighted fusion module combines the four-level feature map data output by the regularization module into one-level map data as the output result through a weighted fusion calculation method according to the following formula, which is the dense displacement vector field of all pixels in the particle area image;

[0112]

[0113] Where, X Fus is the output result of the weighted fusion module, namely the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Indicates Regl The preset weighted values ​​are generally preferred to be the first to fourth levels W l Set to 12.7, 5.5, 4.39, 3.9 in sequence; Reg l Represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

[0114] Therefore, in steps S1, S2, and S5 of the present invention, by installing a camera 2 with a high frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, images of the disassembling and crushing process are obtained online without stopping the rotary drum disassembling and crushing equipment 1. In addition, by extracting the particle area images of two consecutive frames of the disassembling and crushing process images in step S2 and obtaining the dense displacement vector field using the pyramid structure Siamese network model in step S5 and then performing calculation, the particle flow velocity field of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the moment of shooting two consecutive frames of the disassembling and crushing process images is obtained by online calculation. This method has the advantages of high precision and high speed, and can fully ensure the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0115] Among them, the particle flow velocity field can be used to perform online analysis of the mixing status of the particles 3 in the drum 1-1, so as to make reasonable adjustments to the working parameters such as the rotation speed of the drum 1-1 in a timely manner, so that the particles 3 are fully mixed in the drum 1-1, and provide strong support for optimizing the disassembly and crushing process of the drum-type disassembly and crushing equipment 1 and improving the efficiency of the cyclic disassembly work.

[0116] Furthermore, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a correlation calculation module, which can solve the interference of illumination on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0117] In addition, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the pyramid structure Siamese network model has fewer network parameters, the processing time of step S5 can be reduced, solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0118] Moreover, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the pyramid structure Siamese network model introduces a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles 3 of different shapes, colors, and scales, which solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0119] in addition:

[0120] The pyramid structure Siamese network model can be obtained by unsupervised training on a particle flow public database, such as: JHTDH-channel, Backstep, Cylinder, DNS turbulence, SQG, JHTDB-isotropic1024.

[0121] Preferably, the pyramid structure Siamese network model further includes a reverse transfer module; the reverse transfer module is used to calculate the reverse correlation coefficient matrix of the particle area images I1 and I2 of two consecutive frames of disassembly and crushing process images, as shown in Formula 1 and Formula 2 respectively.

[0122]

[0123] Where Corrl represents the reverse correlation coefficient matrix of the lth layer; l is the number of levels, l∈[1,4]; warp is the warping operator; and Represent the output results of the lth convolutional layer in the first and second pyramid structures respectively.

[0124] The feature extraction result of the pyramid structure of the first input image and the feature extraction result of the second input image are used to calculate the calculation result Corrl of the reverse transfer module, which can be used to calculate the Loss value of the pyramid structure Siamese network model during the training process.

[0125] The loss function of the pyramid-structured Siamese network model during training consists of three sub-functions: photometric loss function, smoothness loss function, and consistency loss function.

[0126] The photometric loss function can be expressed as:

[0127]

[0128] Where I1 is the characteristic matrix of particle region image I1; The characteristic matrix of the particle region image I2 is calculated by the reverse transfer module; i, j are the coordinates of the input image respectively; ρ D is the penalty function, which can be expressed as:

[0129] ρ D (x)=(x 2 +ε 2 ) α [Formula 4]

[0130] The coefficients α and ε are set to 0.45 and 10-3 respectively.

[0131] The smoothness loss function can be expressed as:

[0132]

[0133] Among them, u and v are the two component projections of the two input images of particle area image I1 and particle area image I2 on the X and Y axes in the set coordinate system.

[0134] The consistency loss function can be expressed as:

[0135] L con =∑ i,j [ρ D (I1(i,j)-u i,j )+ρ D (v i,j -I2(i,j))] [Formula 6]

[0136] Finally, the above three sub-functions are weighted and summed according to a certain ratio to obtain the overall loss function, which is used in the pyramid network training process described in this application. It can be expressed as:

[0137] L=λ1L pjo +λ2L smo +λ3L con [Formula 7]

[0138] The values ​​of coefficients λ1, λ2, and λ3 are set to 1, 2, and 0.2, respectively.

[0139] in addition:

[0140] In step S1, the camera 2 can complete the posture correction and internal parameter calibration in the following ways, including:

[0141] Step S1-1, pose correction: Use the SDK software of camera 2 to observe the camera's field of view, ensuring that the camera lens plane is parallel to the device's measured surface. At the same time, the collected images of the disassembly and crushing process can include all particles 3 in the device. Record the distance from the camera lens to the device's measured surface.

[0142] Step S1-2, internal parameter calibration: The Zhang Zhengyou camera calibration method is used to obtain the camera internal parameters, which are used to calculate the expression of the particle flow velocity field measurement results in the world coordinate system.

[0143] Step S1-3, camera exposure time and gain adjustment: by adjusting and setting a suitable camera exposure time and gain value range, the image data acquired by the camera can be acquired clearly.

[0144] The present invention is not limited to the above-mentioned specific implementation methods. According to the above content, in accordance with the common technical knowledge and customary means in this field, without departing from the above-mentioned basic technical ideas of the present invention, the present invention can also make other various forms of equivalent modifications, replacements or changes, all of which fall within the scope of protection of the present invention.

Claims

1. A method for online measurement of particle fluid indicators in a drum-type material disassembly and crushing process, applicable to a drum-type disassembly and crushing device (1) that achieves material disassembly and crushing by rotating a drum (1-1); It is characterized in that include: Step S1: During the operation of the rotary drum disassembling and crushing equipment (1), the interior of the rotary drum (1-1) is continuously photographed using a camera (2) to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles (3) formed by disassembling and crushing in the rotary drum (1-1); Step S2: acquiring the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image to obtain a particle area image; Step S3, dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, wherein the contour of each particle region unit is continuous and closed, and any two particle region units are not connected to each other; Step S4, performing adhesion particle separation processing on each particle area unit obtained in step S3, obtaining the independent particle area contour and particle center point coordinate information corresponding to each particle (3) in the disassembly and crushing process image; and using the least squares ellipse fitting algorithm in combination with the particle center point coordinate information to complete the fitting of the corresponding independent particle area contour, thereby obtaining the particle size of each particle (3) in the disassembly and crushing process image, that is, the particle size distribution data of the drum (1-1) of the drum-type disassembly and crushing equipment (1) at the time of shooting the disassembly and crushing process image.

2. The method for online measurement of particle and fluid indicators in a material rotary drum disassembly and crushing process according to claim 1 is characterized by: In step S2, the method of extracting the particle area from the disassembly and crushing process image includes: Step S2-1, processing the disassembly and crushing process image by using a K-means clustering algorithm to distinguish the drum area from the area outside the drum in the disassembly and crushing process image, so as to extract the drum area in the disassembly and crushing process image; Step S2-2: extracting the fuzzy area where the particles are located from the drum area using an Alpha contour extraction algorithm; Step S2-3, extracting the particle area from the fuzzy area where the particle is located by using the Felzenszwalb image segmentation algorithm; Step S2-4, using closed region detection and connected region detection algorithms, eliminating external interference except for particle (3) in the region where the particle is located, to obtain the particle region; Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

3. The method for online measurement of particle and fluid indicators in a material rotary drum disassembly and crushing process according to claim 1 is characterized by: The step S3 is implemented by a lightweight deep semantic segmentation network LUnet.

4. The method for online measurement of particle and fluid indicators in a material rotary drum disassembly and crushing process according to claim 1 is characterized by: In step S4, the method of performing the adhesion particle separation process on any particle region unit includes: Step S4-1, using an exclusive OR operation to process the particle region unit and its particle region unit contour data to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to a particle (3); Step S4-2: using a connected domain detection algorithm to process each of the independent particle regions; Step S4-3: Obtain the independent particle region contour of each independent particle region processed in step S4-2, and use a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

5. The method for online measurement of particle fluid indicators in a material rotary drum disassembly and crushing process according to any one of claims 1 to 4, characterized in that: The method for online measurement of particle fluid indicators in the material rotary drum disassembly and crushing process also includes: Step S5: first obtain particle region images of two consecutive frames of disassembly and crushing process images through step S2; then process the particle region images of the two consecutive frames of disassembly and crushing process images using a pyramid structure Siamese network model to obtain a dense displacement vector field of all pixel points in the particle region image; finally, perform a dot multiplication calculation using the dense displacement vector field and the particle center point coordinate information obtained in step S4 to obtain: the particle flow velocity field of the rotary drum (1-1) of the rotary drum disassembly and crushing equipment (1) at the time when the two consecutive frames of disassembly and crushing process images are taken; Furthermore, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

6. The method for online measurement of particle and fluid indicators in a material rotary drum disassembly and crushing process according to claim 5 is characterized by: The feature extraction module adopts a dual-pyramid structure network consisting of two pyramid structures, wherein each pyramid structure is composed of four convolutional layers cascaded in sequence, and each convolutional layer is composed of a convolutional module with a kernel of 3×3 and a batch normalization module connected in series; and the particle area images of the two consecutive frames of disassembly and fragmentation process images are respectively input into the first convolutional layer of the two pyramid structures, and the fourth convolutional layer of the two pyramid structures respectively outputs a four-level feature matrix sequence corresponding to the two particle area images.

7. The method for online measurement of particle and fluid indicators in a rotary drum material disassembly and crushing process according to claim 5 is characterized by: The related calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels, three composite layers consisting of a 3×3 kernel convolution module and a ReLU function module connected in series, a convolution layer with a 7×7 kernel, a convolution layer with a 5×5 kernel, and two convolution layers with a 3×3 kernel connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels, and the output of the convolution layer with a 3×3 kernel last in the series relationship is used as the output result of the related calculation module.

8. The method for online measurement of particle and fluid indicators in a rotary drum material disassembly and crushing process according to claim 5 is characterized by: The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module and a ReLU function module in series, a convolution layer with a kernel of 3×3, a convolution layer with a kernel of 2×2, and two convolution layers with a kernel of 1×1 in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels in parallel; and the output result of the relevant calculation module is input into the parallel Conv convolution layer, and the output of the convolution layer with a kernel of 1×1 at the end of the series relationship is used as the output result of the sub-pixel module.

9. The method for online measurement of particle and fluid indicators in a rotary drum material disassembly and crushing process according to claim 5, characterized in that: The regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit calculation result by the series unit, then inputs the series unit calculation result into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the series unit calculation result, the activation function calculation result and the front-end input feature matrix by multiplying the result as the output result of the regularization module; wherein, the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules and ReLU function modules in series, 1 convolution layer with a kernel of 7×7, 1 convolution layer with a kernel of 5×5, and 2 convolution layers with a kernel of 3×3 in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels and 2 convolution layers with 3×3 kernels and 259 channels in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

10. The method for online measurement of particle and fluid indicators in a material rotary drum disassembly and crushing process according to claim 5, characterized in that: The weighted fusion module combines the four-level feature map data output by the regularization module into one-level map data as the output result through a weighted fusion calculation method according to the following formula, which is a dense displacement vector field of all pixels in the particle area image; Where, X Fus is the output result of the weighted fusion module, i.e., the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Indicates Reg l Preset weighted value; Reg represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

Images