Particle fluid index on-line measurement method in material rotary drum type disassembling and crushing process

Abstract

The invention discloses a particle fluid index on-line measurement method in a material drum type disassembling and crushing process, which comprises the following steps of: acquiring an image of the disassembling and crushing process on line without shutdown in an operation process of drum type disassembling and crushing equipment by installing a camera; a particle area image is extracted in the step S2, the particle area image is divided into a plurality of particle area units in the step S3, and the contour data of the particle area units are obtained; the method comprises the following steps: extracting particle region images of two continuous frames of images in the disassembling and crushing process in the step S2, obtaining a dense displacement vector field by using a pyramid structure Siamese network model in the step S5, and then performing calculation. The particle size distribution data of the rotating drum of the rotating drum type disassembling and crushing equipment at the shooting moment of the disassembling and crushing process images and the particle flow velocity field of the rotating drum of the rotating drum type disassembling and crushing equipment at the shooting moment of two continuous frames of disassembling and crushing process images are obtained through online calculation, and the method has the advantages of being high in precision and speed.

Description

Technical Field

[0001] The invention relates to the measurement of particle fluid indexes of rotary drum dismantling and crushing equipment, in particular to an online measurement method of particle fluid indexes in a rotary drum dismantling and crushing process of materials. Background Art

[0002] Disassembly and crushing are key links in the production process of recycling, chemical industry, metallurgy and other industries, and rotary drum disassembly and crushing technology can be used. The disassembly and crushing process has a direct impact on subsequent resource utilization, and is also related to energy consumption and equipment wear in the production process. Therefore, by online obtaining the key fluid indicators of particle size distribution and particle flow velocity field that reflect the flow characteristics of particle fluids during the disassembly and crushing process, the working parameters of the equipment can be adjusted in time, which is of great significance and value to improving product disassembly and crushing efficiency, reducing production costs, and improving the economic benefits of enterprises.

[0003] At present, the online measurement of disassembly and crushing process parameters can be achieved through machine vision and other related technologies, which has the characteristics of fast speed and online operation. However, there is currently a lack of end-to-end online measurement solutions for key indicators of particle flow (including particle size distribution and velocity field). Among them, the existing particle flow visual measurement methods are mainly implemented through morphological calculations, which have problems such as light sensitivity, misidentification of adhesion and overlapping particles, and large influence of particle shape, resulting in low measurement accuracy; when using the traditional PIV method for velocity field measurement, there is poor generalization ability and parameters need to be adjusted according to the input image. At the same time, because its perception accuracy is mostly at the pixel level and follows the principle of light conservation, it is more sensitive to the color and brightness of the object being measured, resulting in low measurement accuracy. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an online measurement method for particle fluid indexes in a material rotary drum disassembly and crushing process.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] An online measurement method for particle fluid indexes in a drum-type disassembly and crushing process of materials, which is applicable to drum-type disassembly and crushing equipment that realizes material disassembly and crushing through drum rotation;

[0007] The invention is characterized by comprising:

[0008] Step S1, see Figure 3 During the operation of the rotary drum disassembling and crushing equipment, the interior of the rotary drum is continuously photographed with a camera to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles formed by disassembling and crushing in the rotary drum;

[0009] The drum port of the drum is preferably covered with a transparent baffle, so that the camera can capture the interior of the drum at an angle parallel to the rotation axis of the drum, so as to reduce the algorithm complexity of the image processing of the disassembly and crushing process in the subsequent steps. Preferably, the camera is installed 1.5m in front of the transparent baffle, and the camera's posture correction and internal reference calibration are completed through the SDK software that comes with the camera.

[0010] Among them, the frame rate of the camera should be high enough to ensure the accuracy of the disassembly and crushing process image. The frame rate is generally required to be no less than 150fps, the resolution is generally 640×480, and under indoor daily light conditions, the exposure time is preferably 5000μs, and the gain range is preferably 12.0 to 18.0.

[0011] Step S2, see Figure 4 , using a data processing device to obtain the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image, that is, the area composed of all particles in the disassembly and crushing process image, to obtain a particle area image;

[0012] Step S3, see Figure 6 and Figure 7 , dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, and the contour of each particle region unit is continuously closed, and any two particle region units are not connected to each other;

[0013] Among them, step S3 can be implemented using the existing U-shaped structure lightweight deep network for semantic segmentation.

[0014] Among them, the particle area unit in step S3 includes two cases: first, a particle area unit formed by only one particle independently; second, a particle area unit formed by the adhesion of at least two particles; and for the particle area unit located at the edge of the particle area image and tightly fitted with the inner wall of the rotating drum, its contour portion is composed of the arc line of the inner wall of the rotating drum.

[0015] Step S4: performing a process of separating the adhered particles on each particle region unit obtained in step S3 to obtain the independent particle region contour and particle center point coordinate information corresponding to each particle in the disassembly and crushing process image, for example: Figure 8 for Figure 4 Two particles adhering to each other in the particle area image shown are processed by steps S3 and S4 to obtain two independent particle areas; and the least squares ellipse fitting algorithm is used in conjunction with the particle center point coordinate information to complete the fitting of the corresponding independent particle area contour, so as to obtain the particle size of each particle in the disassembly and crushing process image, that is, the particle size distribution data of the drum of the rotary drum disassembly and crushing equipment at the time of shooting the disassembly and crushing process image.

[0016] Therefore, in steps S1 to S4 of the present invention, by installing a camera with a high frame rate, during the operation of the rotary drum disassembling and crushing equipment, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment, and by extracting the particle area image in step S2, dividing it into multiple particle area units and obtaining particle area unit contour data in step S3, and performing adhesion particle separation processing in step 4, the particle size distribution data of the drum of the rotary drum disassembling and crushing equipment at the time of shooting the disassembling and crushing process image is obtained by online calculation. The method has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment.

[0017] Among them, the particle size distribution data can be used for online analysis of the mixing condition of the particles in the drum, so as to make timely and reasonable adjustments to the working parameters such as the rotation speed of the drum, so that the particles are fully mixed in the drum, and provide strong support for optimizing the disassembly and crushing process of the drum disassembly and crushing equipment and improving work efficiency and production efficiency.

[0018] Preferred: See Figure 5 In step S2, the method of extracting the particle area from the disassembly and crushing process image includes:

[0019] Step S2-1, processing the disassembling and crushing process image by using a K-means clustering algorithm, distinguishing the drum area from the drum external area in the disassembling and crushing process image, so as to extract the drum area in the disassembling and crushing process image;

[0020] Step S2-2, extracting the fuzzy area where the particles are roughly located from the drum area by using an Alpha contour extraction algorithm;

[0021] Step S2-3, extracting the region where the particles are more accurately located from the fuzzy region where the particles are located by using the Felzenszwalb image segmentation algorithm;

[0022] Step S2-4, eliminating external interference other than the particles in the region where the particles are located by closed region detection and connected region detection algorithms to obtain the particle region;

[0023] Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary, and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

[0024] Preferably, step S3 is implemented by a lightweight deep semantic segmentation network LUnet. Thus, the illumination interference problem existing when the particle region image is segmented into a plurality of particle region units can be solved to improve the segmentation accuracy; and, since the network parameters of the lightweight deep semantic segmentation network LUnet are relatively small, the processing time of step S3 can be reduced, solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0025] Preferred: See Fig. 9 In step S4, the method of performing a process of separating the adhering particles on any particle region unit includes:

[0026] Step S4-1, using an exclusive-OR operation to process the particle region unit and its particle region unit contour data, so as to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to one particle;

[0027] Among them, for a particle region unit formed by only one independent particle, the particle region unit itself is an independent particle region; for a particle region unit formed by the adhesion of N≥2 particles, the particle region unit is separated into N independent particle regions with discontinuous and closed contours, for example: Figure 8 for Figure 7 The particle region unit shown is separated into two independent particle regions.

[0028] Step S4-2, using a connected domain detection algorithm to process each of the independent particle regions to remove the interference of tiny particles;

[0029] Step S4-3, obtaining the independent particle region contour of each independent particle region processed by step S4-2, and using a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

[0030] Among them, for the independent particle area separated by the particle area unit formed by only one independent particle, its independent particle area contour is the continuous closed contour of the particle; for the independent particle area separated by the particle area unit formed by the adhesion of N≥2 particles, its independent particle area contour is the discontinuous closed arc contour, for example: Figure 8 The discontinuous closed arc contours of two independent particle regions are shown.

[0031] Therefore, steps S4-1 to S4-3 of the present invention can perform a process of separating adhering particles in any particle area unit, and can solve the problem of easy misidentification caused by densely stacked particles in the drum, so as to improve the accuracy of particle size distribution data.

[0032] As a preferred embodiment of the present invention: the online measurement method for particle fluid index in the material rotary drum disassembly and crushing process also includes:

[0033] Step S5, first obtain the particle area images of two consecutive frames of disassembly and crushing process images through step S2; then use the pyramid structure Siamese network model to process the particle area images of the two consecutive frames of disassembly and crushing process images, and obtain the dense displacement vector field of all pixel points in the particle area image; finally, use the dense displacement vector field and the particle center point coordinate information obtained in step S4 to perform point multiplication calculation, and cooperate with the frame rate of the camera to obtain: the particle flow velocity field of the rotary drum of the rotary drum disassembly and crushing equipment at the time of shooting the two consecutive frames of disassembly and crushing process images;

[0034] And, if Fig.10 As shown, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

[0035] Among them, see Fig.10 The feature extraction module adopts a double pyramid structure network composed of two pyramid structures, wherein each pyramid structure is composed of four convolutional levels cascaded in sequence (i.e., the first convolutional level Conv+BN1, the second convolutional level Conv+BN2, the third convolutional level Conv+BN3, and the fourth convolutional level Conv+BN4), and each convolutional level is composed of a convolutional module Conv with a kernel of 3×3 and a batch normalization module BN in series, that is: the image data input to the convolutional level is first convolutionally operated by the convolutional module Conv and then BN calculated by the batch normalization module BN, and the obtained regularized image features are then sent to the next convolutional level for processing; and the particle area images (I1, I2) of the two consecutive frames of disassembly and crushing process images are respectively input into the first convolutional level Conv+BN1 of the two pyramid structures, and the fourth convolutional level Conv+BN4 of the two pyramid structures respectively outputs a 4-level feature matrix sequence corresponding to the two particle area images (I1, I2).

[0036] In addition, the regularized image features output by each convolutional layer are upsampled to the size of the original input image (i.e., the particle area image), added and merged, and then sent to the backward transfer module for processing.

[0037] Among them, see Fig.11The related calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), three composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules connected in series, a convolution layer Conv 7×7 with a kernel of 7×7, a convolution layer Conv 5×5 with a kernel of 5×5, and two convolution layers Conv 3×3 with a kernel of 3×3 connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), and the output of the convolution layer Conv 3×3 with a kernel of 3×3 located at the last in the series relationship is used as the output result of the related calculation module.

[0038] Among them, see Fig.12 The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module (Conv3×3) and a ReLU function module in series, a convolution layer Conv 3×3 with a kernel of 3×3, a convolution layer Conv 2×2 with a kernel of 2×2, and two convolution layers Conv 1×1 with a kernel of 1×1 in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels (Conv3×3Ch=514) in parallel; and the output result of the relevant calculation module is input into the parallel Conv convolution layer, and the output of the convolution layer Conv1×1 with a kernel of 1×1 at the last in the series relationship is used as the output result of the sub-pixel module.

[0039] Among them, see Fig.13 , the regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit calculation result by the series unit, then inputs the series unit calculation result into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the series unit calculation result, the activation function calculation result and the front-end input feature matrix to obtain the result as the output result of the regularization module; wherein the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules connected in series, 1 convolution layer Conv 7×7 with a kernel of 7×7, 1 convolution layer Conv 5×5 with a kernel of 5×5, and 2 convolution layers Conv3×3 with a kernel of 3×3 connected in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels (Conv 3×3Ch=131) and 2 convolution layers with 3×3 kernels and 259 channels (Conv 3×3Ch=259) are connected in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

[0040] The weighted fusion module combines the four-level feature map data output by the regularization module into one layer of map data as the output result through a weighted fusion calculation method according to the following formula, which is a dense displacement vector field of all pixels in the particle area image;

[0041]

[0042] In the formula, X Fus is the output result of the weighted fusion module, i.e., the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Indicates Reg l The preset weighted value is generally preferred to set the W of the first to fourth levels l Set to 12.7, 5.5, 4.39, 3.9; Reg l Represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

[0043] Therefore, step S1, step S2 and step S5 of the present invention, by installing a camera with a higher frame rate, can obtain the disassembling and crushing process image online during the operation of the rotary drum disassembling and crushing equipment without stopping the rotary drum disassembling and crushing equipment, and extract the particle area image of two consecutive frames of the disassembling and crushing process image by step S2, and obtain the dense displacement vector field by using the pyramid structure Siamese network model in step S5 and then calculate, so as to obtain the particle flow velocity field of the rotary drum of the rotary drum disassembling and crushing equipment at the moment of shooting two consecutive frames of the disassembling and crushing process image online. The invention has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment.

[0044] Among them, the particle flow velocity field can be used to conduct online analysis of the mixing condition of the particles in the drum, so as to make reasonable adjustments to the working parameters such as the rotation speed of the drum in time, so that the particles are fully mixed in the drum, and provide strong support for optimizing the disassembly and crushing process of the drum disassembly and crushing equipment and improving the efficiency of the cyclic disassembly work.

[0045] Furthermore, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a related calculation module, which can solve the interference of light interference on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0046] In addition, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the network parameters of the pyramid structure Siamese network model are relatively small, the processing time of step S5 can be reduced, thereby solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0047] Moreover, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the pyramid structure Siamese network model introduces a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles of different shapes, colors, and scales, which solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] First, in steps S1 to S4 of the present invention, by installing a camera 2 with a relatively high frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment 1, and by extracting the particle area image in step S2, dividing it into a plurality of particle area units and obtaining the particle area unit contour data in step S3, and performing the adhesion particle separation process in step 4, the particle size distribution data of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of shooting the disassembling and crushing process image is obtained by online calculation, which has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0050] Second, in step S1, step S2 and step S5 of the present invention, by installing a camera 2 with a relatively high frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment 1, and by extracting the particle area image of two consecutive frames of the disassembling and crushing process image by step S2, and obtaining the dense displacement vector field by using the pyramid structure Siamese network model in step S5 and then calculating, the particle flow velocity field of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of shooting two consecutive frames of the disassembling and crushing process image is obtained by online calculation, which has the advantages of high precision and high speed, and can fully guarantee the continuity and high efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0051] Third, step S3 of the present invention is implemented by a lightweight deep semantic segmentation network LUnet, which can solve the problem of illumination interference when segmenting a particle region image into multiple particle region units, so as to improve the segmentation accuracy.

[0052] Fourthly, steps S4-1 to S4-3 of the present invention can separate the adhering particles in any particle area unit, and can solve the problem of easy misidentification caused by densely stacked particles 3 in the drum 1-1, so as to improve the accuracy of the particle size distribution data.

[0053] Fifth, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a related calculation module, which can solve the interference of light interference on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0054] Sixth, step S3 of the present invention is implemented through a lightweight deep semantic segmentation network LUnet, and step S5 obtains a dense displacement vector field through a pyramid structure Siamese network model. Since their network parameters are relatively small, the processing time of steps S3 and S5 can be reduced, solving the problem of time-consuming measurement of particle fluid indicators in the prior art.

[0055] Seventh, step S3 of the present invention is implemented by a lightweight deep semantic segmentation network LUnet, and step S5 obtains a dense displacement vector field through a pyramid structure Siamese network model. Since they both introduce a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles 3 of different shapes, colors, and scales, which solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0056] In summary, the present invention realizes the online measurement of two particle fluid indicators, namely, particle size distribution data and particle flow velocity field. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0058] Figure 1 It is a schematic diagram of the process of the present invention;

[0059] Figure 2 Schematic diagram of the arrangement of cameras in step S1 of the present invention;

[0060] Figure 3 The disassembly and crushing process image obtained in step S1 of the present invention;

[0061] Figure 4 The particle area image extracted in step S2 of the present invention;

[0062] Figure 5 This is a flow chart of extracting particle area images in step S2 of the present invention;

[0063] Figure 6 for Figure 4Two particles are stuck to each other in the particle area image shown;

[0064] Figure 7 Step S3 of the present invention is Figure 4 After processing the particle area image shown in the figure, the corresponding Figure 6 The particle area unit of the two particles shown;

[0065] Figure 8 For step S4-1 of the present invention, Figure 7 The two independent particle regions are obtained by separating the particle region unit shown;

[0066] Fig. 9 This is a flow chart of the step S4 of the present invention for separating the adhered particles in the particle area unit;

[0067] Fig.10 A schematic diagram of the network structure of the pyramid structure Siamese network model in step S5 of the present invention;

[0068] Fig.11 A schematic diagram of the network structure of the relevant computing modules of the pyramid structure Siamese network model in the present invention;

[0069] Fig.12 A schematic diagram of the network structure of a sub-pixel module of a pyramid-structured Siamese network model in the present invention;

[0070] Fig.13 Schematic diagram of the network structure of the regularization module of the pyramid structure Siamese network model in the present invention;

[0071] In the figure, represents upsampling, Indicates addition, Indicates multiplication. DETAILED DESCRIPTION

[0072] The present invention is described in detail below in conjunction with the embodiments and the accompanying drawings to help those skilled in the art better understand the inventive concept of the present invention. However, the protection scope of the claims of the present invention is not limited to the following embodiments. For those skilled in the art, all other embodiments obtained without creative work without departing from the inventive concept of the present invention belong to the protection scope of the present invention.

[0073] Embodiment 1

[0074] like Figures 1 to 9 As shown, the present invention discloses an online measurement method for particle fluid indexes in a material drum disassembling and crushing process, which is applicable to a drum disassembling and crushing device 1 that realizes material disassembling and crushing by rotating a drum 1-1;

[0075] include:

[0076] Step S1, see Figure 3 During the operation of the drum type disassembling and crushing equipment 1, the camera 2 is used to continuously shoot the interior of the drum 1-1 to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles 3 formed by disassembling and crushing in the drum 1-1;

[0077] The drum port of the drum 1-1 is preferably covered with a transparent baffle 1-2, so that the camera 2 can shoot the inside of the drum 1-1 at an angle parallel to the rotation axis of the drum 1-1, so as to reduce the algorithm complexity of the image processing of the disassembly and crushing process in the subsequent steps. Preferably, the camera 2 is installed 1.5m in front of the transparent baffle 1-2, and the position correction and internal reference calibration of the camera 2 are completed by the SDK software provided by the camera 2.

[0078] Among them, the frame rate of the camera 2 should be high enough to ensure the accuracy of the disassembly and crushing process image. The frame rate is generally required to be no less than 150fps, the resolution is generally 640×480, and under indoor daily light conditions, the exposure time is preferably 5000μs, and the gain range is preferably 12.0 to 18.0.

[0079] Step S2, see Figure 4 , using a data processing device 4 to obtain the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image, that is, the area composed of all particles 3 in the disassembly and crushing process image, to obtain a particle area image;

[0080] Step S3, see Figure 6 and Figure 7 , dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, and the contour of each particle region unit is continuously closed, and any two particle region units are not connected to each other;

[0081] Among them, step S3 can be implemented using the existing U-shaped structure lightweight deep network for semantic segmentation.

[0082] Among them, the particle area unit in step S3 includes two cases: first, a particle area unit independently formed by only one particle 3; second, a particle area unit formed by the adhesion of at least two particles 3; and, for the particle area unit located at the edge of the particle area image and tightly fitted with the inner wall of the rotating drum 1-1, its contour portion is composed of the arc of the inner wall of the rotating drum 1-1.

[0083] Step S4: perform adhesion particle separation processing on each particle area unit obtained in step S3 to obtain the independent particle area contour and particle center point coordinate information corresponding to each particle 3 in the disassembly and crushing process image, for example: Figure 8 for Figure 4 Two particles 3 adhering to each other in the particle area image shown are two independent particle areas obtained after being processed by steps S3 and S4; and, the least squares ellipse fitting algorithm is used in conjunction with the particle center point coordinate information to complete the fitting of the corresponding independent particle area contour, so as to obtain the particle size of each particle 3 in the disassembly and crushing process image, that is, the particle size distribution data of the drum 1-1 of the drum-type disassembly and crushing equipment 1 at the time of shooting the disassembly and crushing process image.

[0084] Therefore, in steps S1 to S4 of the present invention, by installing a camera 2 with a higher frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment 1, and by extracting the particle area image in step S2, dividing it into multiple particle area units and obtaining the particle area unit contour data in step S3, and performing adhesion particle separation processing in step 4, the particle size distribution data of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of shooting the disassembling and crushing process image is obtained by online calculation, which has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0085] Among them, the particle size distribution data can be used to conduct online analysis of the mixing condition of the particles 3 in the drum 1-1, so as to make reasonable adjustments to the working parameters such as the rotation speed of the drum 1-1 in time, so that the particles 3 are fully mixed in the drum 1-1, and provide strong support for optimizing the disassembly and crushing process of the drum disassembly and crushing equipment 1 and improving the efficiency of the circulating disassembly work.

[0086] The above is a basic implementation of the first embodiment of the present invention, and further optimization, improvement and limitation can be made on the basis of the basic implementation:

[0087] Preferred: See Figure 5 In step S2, the method of extracting the particle area from the disassembly and crushing process image includes:

[0088] Step S2-1, processing the disassembling and crushing process image by using a K-means clustering algorithm, distinguishing the drum area from the drum external area in the disassembling and crushing process image, so as to extract the drum area in the disassembling and crushing process image;

[0089] Step S2-2, extracting the fuzzy area where the particle 3 is approximately located from the drum area by using an Alpha contour extraction algorithm;

[0090] Step S2-3, using the Felzenszwalb image segmentation algorithm, extracting the region where the particle 3 is more accurately located from the fuzzy region where the particle is located;

[0091] Step S2-4, using closed region detection and connected domain detection algorithms, eliminating external interference except for particle 3 in the region where the particle is located, so as to obtain the particle region;

[0092] Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary, and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

[0093] Preferably, step S3 is implemented by a lightweight deep semantic segmentation network LUnet. Thus, the illumination interference problem existing when the particle region image is segmented into a plurality of particle region units can be solved to improve the segmentation accuracy; and, since the network parameters of the lightweight deep semantic segmentation network LUnet are relatively small, the processing time of step S3 can be reduced, solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0094] Preferred: See Fig. 9 In step S4, the method of performing a process of separating the adhering particles on any particle region unit includes:

[0095] Step S4-1, using an exclusive-OR operation to process the particle region unit and its particle region unit contour data, so as to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to a particle 3;

[0096] Among them, for a particle region unit independently formed by only one particle 3, the particle region unit itself is an independent particle region; for a particle region unit formed by the adhesion of N≥2 particles 3, the particle region unit is separated into N independent particle regions with discontinuous and closed contours, for example: Figure 8 for Figure 7 The particle region unit shown is separated into two independent particle regions.

[0097] Step S4-2, using a connected domain detection algorithm to process each of the independent particle regions to remove the interference of tiny particles;

[0098] Step S4-3, obtaining the independent particle region contour of each independent particle region processed by step S4-2, and using a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

[0099] Among them, for the independent particle area separated by the particle area unit formed by only one particle 3, its independent particle area contour is the continuous closed contour of the particle 3; for the independent particle area separated by the particle area unit formed by the adhesion of N≥2 particles 3, its independent particle area contour is the discontinuous closed arc contour, for example: Figure 8 The discontinuous closed arc contours of two independent particle regions are shown.

[0100] Therefore, steps S4-1 to S4-3 of the present invention can perform a process of separating adhering particles in any particle area unit, and can solve the problem of easy misidentification caused by densely stacked particles 3 in the drum 1-1, so as to improve the accuracy of particle size distribution data.

[0101] Embodiment 2

[0102] On the basis of the above-mentioned embodiment 1, this embodiment 2 also adopts the following preferred implementation manner:

[0103] The method for online measurement of particle fluid indicators in the material drum disassembly and crushing process also includes:

[0104] Step S5, first obtain the particle area images of two consecutive frames of disassembly and crushing process images through step S2; then use the pyramid structure Siamese network model to process the particle area images of the two consecutive frames of disassembly and crushing process images, and obtain the dense displacement vector field of all pixel points in the particle area image; finally, use the dense displacement vector field and the particle center point coordinate information obtained in step S4 to perform point multiplication calculation, and cooperate with the frame rate of the camera 2 to obtain: the particle flow velocity field of the drum 1-1 of the drum type disassembly and crushing equipment 1 at the time of shooting the two consecutive frames of disassembly and crushing process images;

[0105] And, if Fig.10 As shown, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

[0106] Among them, see Fig.10The feature extraction module adopts a double pyramid structure network composed of two pyramid structures, wherein each pyramid structure is composed of four convolutional levels cascaded in sequence (i.e., the first convolutional level Conv+BN1, the second convolutional level Conv+BN2, the third convolutional level Conv+BN3, and the fourth convolutional level Conv+BN4), and each convolutional level is composed of a convolutional module Conv with a kernel of 3×3 and a batch normalization module BN in series, that is: the image data input to the convolutional level is first convolutionally operated by the convolutional module Conv and then BN calculated by the batch normalization module BN, and the obtained regularized image features are then sent to the next convolutional level for processing; and the particle area images I1 and I2 of the two consecutive frames of disassembly and crushing process images are respectively input into the first convolutional level Conv+BN1 of the two pyramid structures, and the fourth convolutional level Conv+BN4 of the two pyramid structures respectively outputs a 4-level feature matrix sequence corresponding to the two particle area images I1 and I2.

[0107] In addition, the regularized image features output by each convolutional layer are upsampled to the size of the original input image (i.e., the particle area image), added and merged, and then sent to the backward transfer module for processing.

[0108] Among them, see Fig.11 The related calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), three composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules connected in series, a convolution layer Conv 7×7 with a kernel of 7×7, a convolution layer Conv 5×5 with a kernel of 5×5, and two convolution layers Conv 3×3 with a kernel of 3×3 connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels (ConvT 4×4Ch=49), and the output of the convolution layer Conv 3×3 with a kernel of 3×3 located at the last in the series relationship is used as the output result of the related calculation module.

[0109] Among them, see Fig.12The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module (Conv3×3) and a ReLU function module in series, a convolution layer Conv 3×3 with a kernel of 3×3, a convolution layer Conv 2×2 with a kernel of 2×2, and two convolution layers Conv 1×1 with a kernel of 1×1 in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels (Conv3×3Ch=514) in parallel; and the output result of the relevant calculation module is input into the parallel Conv convolution layer, and the output of the convolution layer Conv1×1 with a kernel of 1×1 at the last in the series relationship is used as the output result of the sub-pixel module.

[0110] Among them, see Fig.13 , the regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit calculation result by the series unit, then inputs the series unit calculation result into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the series unit calculation result, the activation function calculation result and the front-end input feature matrix to obtain the result as the output result of the regularization module; wherein the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules (Conv 3×3) and ReLU function modules connected in series, 1 convolution layer Conv 7×7 with a kernel of 7×7, 1 convolution layer Conv 5×5 with a kernel of 5×5, and 2 convolution layers Conv3×3 with a kernel of 3×3 connected in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels (Conv 3×3Ch=131) and 2 convolution layers with 3×3 kernels and 259 channels (Conv 3×3Ch=259) are connected in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

[0111] The weighted fusion module combines the four-level feature map data output by the regularization module into one layer of map data as the output result through a weighted fusion calculation method according to the following formula, which is a dense displacement vector field of all pixels in the particle area image;

[0112]

[0113] In the formula, X Fus is the output result of the weighted fusion module, i.e., the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Indicates Regl The preset weighted value is generally preferred to set the W of the first to fourth levels l Set to 12.7, 5.5, 4.39, 3.9; Reg l Represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

[0114] Therefore, in step S1, step S2 and step S5 of the present invention, by installing a camera 2 with a higher frame rate, during the operation of the rotary drum disassembling and crushing equipment 1, the disassembling and crushing process image is obtained online without stopping the rotary drum disassembling and crushing equipment 1, and by extracting the particle area image of two consecutive frames of the disassembling and crushing process image by step S2, and obtaining the dense displacement vector field by using the pyramid structure Siamese network model in step S5 and then calculating, the particle flow velocity field of the drum 1-1 of the rotary drum disassembling and crushing equipment 1 at the time of shooting two consecutive frames of the disassembling and crushing process image is obtained by online calculation, which has the advantages of high precision and high speed, and can fully guarantee the continuity and efficiency of the production of the rotary drum disassembling and crushing equipment 1.

[0115] Among them, the particle flow velocity field can be used to perform online analysis on the mixing condition of the particles 3 in the drum 1-1, so as to make reasonable adjustments to the working parameters such as the rotation speed of the drum 1-1 in time, so that the particles 3 are fully mixed in the drum 1-1, and provide strong support for optimizing the disassembly and crushing process of the drum disassembly and crushing equipment 1 and improving the efficiency of the circulating disassembly work.

[0116] Furthermore, the pyramid structure Siamese network model in step S5 of the present invention adopts a feature extraction module and a related calculation module, which can solve the interference of light interference on the dense displacement vector field and improve the calculation accuracy of the dense displacement vector field.

[0117] In addition, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the network parameters of the pyramid structure Siamese network model are relatively small, the processing time of step S5 can be reduced, thereby solving the problem of long time consumption in measuring particle fluid indicators in the prior art.

[0118] Moreover, step S5 of the present invention obtains a dense displacement vector field through a pyramid structure Siamese network model. Since the pyramid structure Siamese network model introduces a large number of BN layers for calculation, the generalization ability is improved, so that the present invention has a good measurement effect for particles 3 of different shapes, colors, and scales, which solves the problem of poor generalization ability of measuring particle fluid indicators in the prior art.

[0119] in addition:

[0120] The pyramid structure Siamese network model can be obtained by unsupervised training in a particle flow public database, such as: JHTDH-channel, Backstep, Cylinder, DNS turbulence, SQG, JHTDB-isotropic1024.

[0121] Preferably: the pyramid structure Siamese network model also includes a reverse transfer module; the reverse transfer module is used to calculate the reverse correlation coefficient matrix of the particle area images I1 and I2 of two consecutive frames of disassembly and crushing process images, as shown in Formula 1 and Formula 2 respectively.

[0122]

[0123] Where Corrl represents the reverse correlation coefficient matrix of the lth layer; l is the number of levels, l∈[1,4]; warp is the warping operator; and Represent the output results of the lth convolutional layer in the first and second pyramid structures respectively.

[0124] The feature extraction result of the pyramid structure of the first input image and the feature extraction result of the second image are used to calculate the Corrl result of the reverse transfer module, which can be used to calculate the Loss value of the pyramid structure Siamese network model during the training process.

[0125] The loss function of the pyramid-structured Siamese network model during training consists of three sub-functions: photometric loss function, smoothness loss function, and consistency loss function.

[0126] The photometric loss function can be expressed as:

[0127]

[0128] Where I1 is the characteristic matrix of particle region image I1; represents the result of the feature matrix of the particle region image I2 calculated by the reverse transfer module; i, j are the coordinates of the input image respectively; ρ D is the penalty function, which can be expressed as:

[0129] ρ D (x)=(x 2 +ε 2 ) α [Formula 4]

[0130] The coefficients α and ε are set to 0.45 and 10-3, respectively.

[0131] The smoothness loss function can be expressed as:

[0132]

[0133] Among them, u and v are the two component projections of the two input images of particle area image I1 and particle area image I2 on the X and Y axes in the set coordinate system.

[0134] The consistency loss function can be expressed as:

[0135] L con =∑ i,j [ρ D (I1(i,j)-u i,j )+ρ D (v i,j -I2(i,j))] [Formula 6]

[0136] Finally, the above three sub-functions are weighted and summed according to a certain ratio to obtain the overall loss function, which is used in the pyramid network training process described in this application. It can be expressed as:

[0137] L=λ1L pjo +λ2L smo +λ3L con [Formula 7]

[0138] The values ​​of coefficients λ1, λ2 and λ3 are set to 1, 2 and 0.2 respectively.

[0139] in addition:

[0140] In step S1, the camera 2 can complete posture correction and internal parameter calibration in the following manner, including:

[0141] Step S1-1, posture correction: Use the SDK software of camera 2 to observe the camera field of view, so that the camera lens plane is parallel to the measured surface of the device, and at the same time, the collected disassembly and crushing process image can contain all particles 3 in the device. Record the distance from the camera lens to the measured surface of the device.

[0142] Step S1-2, internal parameter calibration: Zhang Zhengyou camera calibration method is used to obtain the camera internal parameters, which are used to calculate the expression of the particle flow velocity field measurement results in the world coordinate system.

[0143] Step S1-3, camera exposure time and gain adjustment: by adjusting and setting a suitable camera exposure time and gain value range, the image data acquired by the camera can be acquired clearly.

[0144] The present invention is not limited to the above-mentioned specific implementation modes. According to the above-mentioned contents, in accordance with the common technical knowledge and customary means in the field, without departing from the above-mentioned basic technical ideas of the present invention, the present invention can also make other various forms of equivalent modifications, replacements or changes, all of which fall within the protection scope of the present invention.

Claims

1. An online measurement method for particle fluid indexes in a material drum disassembly and crushing process, applicable to a drum disassembly and crushing device (1) that realizes material disassembly and crushing by rotating a drum (1-1); It is characterized in that include: Step S1: During the operation of the rotary drum disassembling and crushing equipment (1), the interior of the rotary drum (1-1) is continuously photographed with a camera (2) to continuously obtain a plurality of frames of disassembling and crushing process images, wherein the disassembling and crushing process images include all particles (3) formed by disassembling and crushing in the rotary drum (1-1); Step S2, acquiring the disassembly and crushing process image, and extracting the particle area of ​​the disassembly and crushing process image to obtain a particle area image; Step S3, dividing the particle region image into a plurality of particle region units, and obtaining particle region unit contour data of each particle region unit, and the contour of each particle region unit is continuously closed, and any two particle region units are not connected to each other; Step S4, performing a process of separating the adhered particles on each particle area unit obtained in step S3, and obtaining the independent particle area contour and particle center point coordinate information corresponding to each particle (3) in the disassembly and crushing process image; and using a least squares ellipse fitting algorithm in conjunction with the particle center point coordinate information to complete the fitting of the corresponding independent particle area contour, so as to obtain the particle size of each particle (3) in the disassembly and crushing process image, that is, the particle size distribution data of the drum (1-1) of the drum-type disassembly and crushing equipment (1) at the time of shooting the disassembly and crushing process image.

2. According to claim 1, the method for online measurement of particle fluid indexes in the material drum disassembly and crushing process is characterized by: In step S2, the method of extracting the particle area from the disassembly and crushing process image includes: Step S2-1, processing the disassembling and crushing process image by using a K-means clustering algorithm, distinguishing the drum area from the drum external area in the disassembling and crushing process image, so as to extract the drum area in the disassembling and crushing process image; Step S2-2, extracting the fuzzy area where the particles are located from the drum area by using an Alpha contour extraction algorithm; Step S2-3, extracting the region where the particles are located from the fuzzy region where the particles are located by using the Felzenszwalb image segmentation algorithm; Step S2-4, using closed region detection and connected domain detection algorithms, eliminating external interference other than the particle (3) in the region where the particle is located, so as to obtain the particle region; Step S2-5: extracting the boundary of the particle region by using a threshold method, retaining the pixel values ​​within the boundary, and setting the pixel values ​​outside the boundary to 0, so as to obtain the particle region image.

3. According to claim 1, the method for online measurement of particle fluid indexes in the material drum disassembly and crushing process is characterized by: The step S3 is implemented by a lightweight deep semantic segmentation network LUnet.

4. According to claim 1, the method for online measurement of particle fluid indexes in the material rotary drum disassembly and crushing process is characterized by: In the step S4, the method of performing a process of separating the adhering particles on any particle region unit includes: Step S4-1, using an exclusive-OR operation to process the particle region unit and its particle region unit contour data, so as to separate the particle region unit into a plurality of independent particle regions, and each independent particle region corresponds to a particle (3); Step S4-2, using a connected domain detection algorithm to process each of the independent particle regions; Step S4-3, obtaining the independent particle region contour of each independent particle region processed by step S4-2, and using a region center point detection algorithm to obtain the particle center point coordinate information of the independent particle region.

5. The method for online measurement of particle fluid indexes in the material drum disassembly and crushing process according to any one of claims 1 to 4, characterized in that: The method for online measurement of particle fluid indicators in the material drum disassembly and crushing process also includes: Step S5, firstly obtain the particle region images of two consecutive frames of disassembly and crushing process images through step S2; then use the pyramid structure Siamese network model to process the particle region images of the two consecutive frames of disassembly and crushing process images to obtain a dense displacement vector field of all pixel points in the particle region image; finally, use the dense displacement vector field and the particle center point coordinate information obtained in step S4 to perform a dot multiplication calculation to obtain: the particle flow velocity field of the rotary drum (1-1) of the rotary drum disassembly and crushing equipment (1) at the time of shooting the two consecutive frames of the disassembly and crushing process images; Furthermore, the pyramid structure Siamese network model consists of a feature extraction module, a correlation calculation module, a sub-pixel module, a regularization module and a weighted fusion module.

6. The method for online measurement of particle fluid indexes in the material drum disassembly and crushing process according to claim 5 is characterized by: The feature extraction module adopts a dual pyramid structure network consisting of two pyramid structures, wherein each pyramid structure is composed of four convolutional levels cascaded in sequence, and each convolutional level is composed of a convolutional module with a kernel of 3×3 and a batch regularization module connected in series; and the particle area images of the two consecutive frames of disassembly and crushing process images are respectively input into the first convolutional levels of the two pyramid structures, and the fourth convolutional levels of the two pyramid structures respectively output a 4-level feature matrix sequence corresponding to the two particle area images.

7. The method for online measurement of particle fluid indexes in the material rotary drum disassembly and crushing process according to claim 5 is characterized by: The related calculation module is composed of a deconvolution layer with a 4×4 kernel and 49 channels, 3 composite layers composed of 3×3 kernel convolution modules and ReLU function modules connected in series, a convolution layer with a kernel of 7×7, a convolution layer with a kernel of 5×5, and 2 convolution layers with kernels of 3×3 connected in series in sequence; and the output result of the feature extraction module is input into the deconvolution layer with a 4×4 kernel and 49 channels, and the output of the convolution layer with a kernel of 3×3, which is the last in the series relationship, is used as the output result of the related calculation module.

8. The method for online measurement of particle fluid indexes in the material rotary drum disassembly and crushing process according to claim 5 is characterized by: The sub-pixel module is composed of a parallel Conv convolution layer, a composite layer composed of a 3×3 kernel convolution module and a ReLU function module connected in series, a convolution layer with a kernel of 3×3, a convolution layer with a kernel of 2×2, and two convolution layers with a kernel of 1×1 connected in series, wherein the parallel Conv convolution layer is composed of four convolution layers with 3×3 kernels and 514 channels connected in parallel; and the output results of the relevant calculation modules are input into the parallel Conv convolution layer, and the output of the convolution layer with a kernel of 1×1 at the end of the series relationship is used as the output result of the sub-pixel module.

9. The method for online measurement of particle fluid indexes in the material rotary drum disassembly and crushing process according to claim 5 is characterized by: The regularization module uses the output result of the sub-pixel module as the front-end input feature matrix, and first calculates the series unit to obtain the calculation result of the series unit, then inputs the calculation result of the series unit into the Softmax activation function to obtain the activation function calculation result, and finally multiplies the calculation result of the series unit, the activation function calculation result and the front-end input feature matrix as the output result of the regularization module; wherein the series unit is composed of a second parallel Conv convolution layer, 5 composite layers composed of 3×3 kernel convolution modules and ReLU function modules connected in series, 1 convolution layer with a kernel of 7×7, 1 convolution layer with a kernel of 5×5, and 2 convolution layers with a kernel of 3×3 connected in series, the second parallel Conv convolution layer is composed of 2 convolution layers with 3×3 kernels and 131 channels and 2 convolution layers with 3×3 kernels and 259 channels connected in parallel, and the front-end input feature matrix is ​​input into the second parallel Conv convolution layer of the series unit.

10. The method for online measurement of particle and fluid indicators in the material drum disassembly and crushing process according to claim 5 is characterized by: The weighted fusion module combines the four-level feature map data output by the regularization module into one layer of map data as an output result through a weighted fusion calculation method according to the following formula, which is a dense displacement vector field of all pixels in the particle area image; Where, X Fus is the output result of the weighted fusion module, i.e., the dense displacement vector field; prod(W l ,Reg l ) indicates the l and Reg l Perform inner product operation; W l Expressing Reg l Preset weighted value; Reg represents the lth level feature map data among the 4 levels of feature map data output by the regularization module.

Images