A vision-based real-time detection method for mineral mixture components

Abstract

This invention discloses a vision-based real-time detection method for mineral mixture components, belonging to the field of mineral mixture production technology. The method includes camera selection and installation, acquisition of imaging videos of different material compositions, generation of a target detection dataset, analysis and calculation of the target detection dataset, setting an allowable error range, and finally timely detection of abnormal mineral mixture proportions through image detection and notification to production line workers. The proposed method is adaptable to all-weather lighting conditions, has good generalization ability, can adapt to different conveyor belt angles and camera imaging angles, can be used across different plant areas for the same mixture, improves the real-time performance of mixture proportion detection, and can record the actual proportions for each feeding cycle. It addresses the shortcomings of existing methods that rely on production line workers to remotely observe raw material proportions in real-time during step-by-step proportioning, leading to worker fatigue and difficulty in timely detection of abnormal material proportions.

Description

Technical Field

[0001] This invention relates to the field of mineral mixture production technology, and in particular to a vision-based real-time detection method for mineral mixture components. Background Technology

[0002] In the production process of mineral aggregates, whether the raw material ratio is correct will affect the quality of the target product of the aggregate. For example, the ratio of cement raw meal affects the performance of the cement clinker, and the ratio of road construction aggregate affects the performance of the road surface. The processes before and after these aggregates are often characterized by high energy consumption and irreversibility after the product is formed.

[0003] In actual production lines, raw materials are added step-by-step according to the specified proportions. However, due to various reasons, such as equipment malfunctions causing shutdowns, wooden or steel rods, and large-sized mineral materials often blocking the chute, the raw materials may not be added as intended during certain periods, resulting in a significant discrepancy between the gradation and the standard proportions. Currently, the common practice is for production line workers to remotely monitor the raw material proportions in real time. This can lead to rapid worker fatigue and difficulty in promptly detecting abnormal material proportions. To address these shortcomings, this invention proposes a vision-based real-time detection method for mineral mixture components. This method can detect the proportions of the mixture in real time in a semi-open-air production environment, providing timely alerts to production line workers when proportions are abnormal. This approach can effectively save energy, reduce emissions, and improve product quality. Summary of the Invention

[0004] The main objective of this invention is to provide a method for detecting the proportion of mineral raw materials based on visual analysis, which can effectively solve the problems in the background technology by using artificial intelligence vision technology.

[0005] The objective of this invention can be achieved by adopting the following technical solution:

[0006] A vision-based real-time detection method for mineral mixture components includes the following steps:

[0007] Step 1, Select and install an IP camera: Select a full-color night vision camera with automatic light source compensation function. The camera image quality should meet the analysis requirements. Install the camera above the feeding equipment for each mineral mixture component. The IP camera will convert the video into MP4 through the manufacturer's video backend software.

[0008] Step 2: Acquire imaging videos of different material compositions: During different time periods of light, control the switching on and off of the feeding equipment for each mineral mixture component, record the time periods, and collect 5 minutes of video data at different time periods within two consecutive weeks from the IP camera video backend software to obtain the "conveyor belt empty video", "limestone loading operation video", "limestone and loess mixture loading operation video" and "complete mixed raw material loading operation video" of the production line.

[0009] Step 3: Generate the object detection dataset: Using the OpenCV image processing library, extract image sequences from the video based on the production line's operating speed;

[0010] Step 4, Target Detection Set Analysis and Calculation: Use a deep learning network to perform multi-point analysis on a standard continuous material transport video. Within one feeding cycle, based on a general machine learning framework, train the deep learning network on the image data to calculate the proportion of each type of mixed material segment.

[0011] Step 5: Compile the distribution results;

[0012] Step 6: Set the allowable error range: Based on the actual experience of the process control engineer, set the actual allowable error range of the material composition, statistically analyze the detection results of the monitoring points, and report the results to the production operation and maintenance system. When the material ratio exceeds the limit for a period of time, the system will promptly and proactively alert the production line workers to the material batching abnormality, so that on-site personnel can be informed of the material batching abnormality in a timely manner and track the cause of the abnormality.

[0013] As a further method of the present invention, the time interval for extracting images in step two is controlled so that the content of each pair of images just does not overlap, and the formula for the image sequence extraction interval is:

[0014]

[0015] In the formula: J tp FR is the extraction interval of the image sequence, L is the physical length of the conveyor belt within the field of view, V is the running speed of the conveyor belt, and FR is the frame rate of the video.

[0016] As a further method of the present invention, the specific process of target detection set analysis and calculation in step four is as follows:

[0017] For the "limestone" monitoring point, empty-band running data was used as negative samples. A binary classification model was trained using the MobileNetV2 deep learning network in the general machine learning framework PyTorch. The images were cropped and reorganized to a size of 256×256 pixels. A test set image prediction model was formed using the stochastic gradient descent method and a learning rate of 0.0001. The model was used to predict the test set image sequence to determine which image was not properly delivered at the corresponding time point.

[0018] For the "limestone + loess" monitoring point, the limestone conveyor belt data is used as negative samples. A binary classification model is trained using the deep learning network MobileNetV2. The images are cropped and organized to a size of 256×256 pixels. A test set image prediction model is formed using the stochastic gradient descent method and a learning rate of 0.0001. The model is used to predict the test set image sequence and determine which image was not properly delivered at the corresponding time point.

[0019] For the monitoring point of "limestone + loess + correction material", the correction material was intermittently applied. The location boxes of the correction material on the map were marked, and the area within the location box was used as the detection dataset. A target detection model was trained using the YOLOv5 deep learning network. The images were cropped and resized to 640×640 pixels. A test set image prediction model was formed using the stochastic gradient descent method and a learning rate of 0.0001. The model was used to predict the sequence of test set images to determine which image had the correction material applied at the corresponding time point. The proportion of the correction material detection box in the conveyor belt carrying area was used as the estimated value of the amount of correction material applied.

[0020] As a further method of the present invention, the different light conditions in step one are 8:00 AM, 12:00 PM, 6:00 PM, and midnight.

[0021] As a further method of the present invention, the camera in step one has a resolution of 4 million pixels or more.

[0022] As a further method of the present invention, the feeding cycle in step four is the time span between two feedings of the feeding component with the longest interval.

[0023] As a further method of the present invention, the specific method for statistically analyzing the delivery results in step six is ​​as follows:

[0024] During the feeding cycle, the number of images showing missing limestone and loess is counted relative to the total number of images in the entire cycle. This proportion is then used as the degree of limestone and loess deficiency. The cumulative value of the correction material detection frame is also calculated as a percentage of the total number of images in the entire cycle. The formula for the degree of limestone and loess deficiency is as follows:

[0025]

[0026] In the formula: D SH N represents the degree of limestone and loess deficiency, N is the number of images with missing limestone and loess, and M is the total number of images in the entire period.

[0027] The fitting formula for the amount of corrective material added is:

[0028]

[0029] Where: IOU adj To correct the fitted value of the feed amount, S adj To correct the cumulative value of the material inspection frame area, S all This is the cumulative value of the material-carrying area of ​​the conveyor belt throughout the entire image cycle.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] First, it has good versatility, can adapt to all-weather lighting environments, can adapt to different conveyor belts and camera imaging angles, and can be used across different plant areas for the same mixture;

[0032] Second, this method has high real-time performance, capable of recording the actual proportion of each feeding cycle, and automatically starting and stopping the mixing production equipment by using the proportion of mixed material segments in the video, and automatically operating the equipment to replenish missing components.

[0033] Third, the transformation cost is low and the interference with the original production line is minimal. Apart from adding cameras, no other equipment is required, and the equipment can be put into operation without modifying the existing production process. Attached Figure Description

[0034] Figure 1 The flowchart shows the method for detecting the proportion of raw meal (limestone, loess, and iron-aluminum corrective material mixture) in a cement production line according to the vision-based real-time detection method for mineral mixture components of the present invention.

[0035] Figure 2 Extraction diagrams for unloaded materials, limestone feeding materials, limestone + loess mixture, and the final mixture of limestone + loess + corrective material.

[0036] Figure 3 Images were extracted from the mixture collected at different time periods;

[0037] Figure 4 To correct the proportion of the cumulative value of the material inspection frame throughout the entire cycle;

[0038] Figure 5 This is a schematic diagram for calibrating the material inspection frame. Detailed Implementation

[0039] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0040] The present invention provides a vision-based real-time detection method for mineral mixture components, comprising the following steps:

[0041] Step 1, Select and install an IP camera: Select a full-color night vision camera with automatic light source compensation function. The camera image quality should meet the analysis requirements. Install the camera above the feeding equipment for each mineral mixture component. The IP camera will convert the video into MP4 through the manufacturer's video backend software.

[0042] Step 2: Acquire imaging videos of different material compositions: During different time periods of light, control the switching on and off of the feeding equipment for each mineral mixture component, record the time periods, and collect 5 minutes of video data at different time periods within two consecutive weeks from the IP camera video backend software to obtain the "conveyor belt empty video", "limestone loading operation video", "limestone and loess mixture loading operation video" and "complete mixed raw material loading operation video" of the production line.

[0043] Step 3: Generate the object detection dataset: Using the OpenCV image processing library, extract image sequences from the video based on the production line's operating speed;

[0044] Step 4, Target Detection Set Analysis and Calculation: Use a deep learning network to perform multi-point analysis on a standard continuous material transport video. Within one feeding cycle, based on a general machine learning framework, train the deep learning network on the image data to calculate the proportion of each type of mixed material segment.

[0045] Step 5: Compile the distribution results;

[0046] Step 6: Set the allowable error range: Based on the actual experience of the process control engineer, set the actual allowable error range of the material composition, statistically analyze the detection results of the monitoring points, and report the results to the production operation and maintenance system. When the material ratio exceeds the limit for a period of time, the system will promptly and proactively alert the production line workers to the material batching abnormality, so that on-site personnel can be informed of the material batching abnormality in a timely manner and track the cause of the abnormality.

[0047] Compared with the prior art, the advantages of the present invention are:

[0048] First, it has good versatility, can adapt to all-weather lighting environments, can adapt to different conveyor belts and camera imaging angles, and can be used across different plant areas for the same mixture;

[0049] Second, this method has high real-time performance, capable of recording the actual proportion of each feeding cycle, and automatically starting and stopping the mixing production equipment by using the proportion of mixed material segments in the video, and automatically operating the equipment to replenish missing components.

[0050] Third, the transformation cost is low and the interference with the original production line is minimal. Apart from adding cameras, no other equipment is required, and the equipment can be put into operation without modifying the existing production process.

[0051] The time interval for extracting images in step two is controlled so that the content of each pair of images does not overlap. The formula for the image sequence extraction interval is:

[0052]

[0053] In the formula: J tp FR is the extraction interval of the image sequence, L is the physical length of the conveyor belt within the field of view, V is the running speed of the conveyor belt, and FR is the frame rate of the video.

[0054] By setting an extraction interval formula for image sequences, it is possible to quickly find methods and strategies for extracting image sequences under different conveyor belt speeds, making it easier to extract images applicable to different production line speeds and increasing the universality of this invention.

[0055] The specific process of target detection set analysis and calculation in step four is as follows:

[0056] For the "limestone" monitoring point, empty-band running data was used as negative samples. A binary classification model was trained using the MobileNetV2 deep learning network in the general machine learning framework PyTorch. The images were cropped and reorganized to a size of 256×256 pixels. A test set image prediction model was formed using the stochastic gradient descent method and a learning rate of 0.0001. The model was used to predict the test set image sequence to determine which image was not properly delivered at the corresponding time point.

[0057] For the "limestone + loess" monitoring point, the limestone conveyor belt data is used as negative samples. A binary classification model is trained using the deep learning network MobileNetV2. The images are cropped and organized to a size of 256×256 pixels. A test set image prediction model is formed using the stochastic gradient descent method and a learning rate of 0.0001. The model is used to predict the test set image sequence and determine which image was not properly delivered at the corresponding time point.

[0058] For the monitoring point of "limestone + loess + correction material", the correction material was intermittently applied. The location boxes of the correction material on the map were marked, and the area within the location box was used as the detection dataset. A binary classification model was trained using the MobileNetV2 deep learning network. The images were cropped and resized to 640×640 pixels. A test set image prediction model was formed using the stochastic gradient descent method and a learning rate of 0.0001. The model was used to predict the sequence of test set images to determine which image had the correction material applied at the corresponding time point. The proportion of the correction material detection box in the conveyor belt carrying area was used as the estimated value of the amount of correction material applied.

[0059] The use of deep learning network technology and stochastic gradient descent method can help learn the training dataset, so as to form their own prediction methods under different feeding methods, which can facilitate the identification of abnormal feeding situations under different mineral mixture compositions, and then transmit the information to the production operation and maintenance system in a timely manner.

[0060] The different lighting conditions in step one are 8:00 AM, 12:00 PM, 6:00 PM, and midnight.

[0061] By setting image detection under different lighting conditions, it is possible to detect and adapt to the mixing of mineral materials under different lighting conditions, so as to obtain the applicability and accuracy of the entire prediction and detection model, and at the same time increase the generalization degree of the model.

[0062] In step one, the camera has a resolution of 4 million pixels or more.

[0063] Limiting the camera's pixels ensures that the clarity and pixel level of the acquired video files meet the accuracy and clarity requirements of the detection, thus laying a data foundation for improving detection precision and accuracy.

[0064] The feeding cycle in step four is the time span between two feedings of the component with the longest interval.

[0065] The specific method for calculating the delivery results in step six is ​​as follows:

[0066] During the feeding cycle, the number of images showing missing limestone and loess is counted relative to the total number of images in the entire cycle. This proportion is then used as the degree of limestone and loess deficiency. The cumulative value of the correction material detection frame is also calculated as a percentage of the total number of images in the entire cycle. The formula for the degree of limestone and loess deficiency is as follows:

[0067]

[0068] In the formula: D SH N represents the degree of limestone and loess deficiency, N is the number of images with missing limestone and loess, and M is the total number of images in the entire period.

[0069] The fitting formula for the amount of corrective material added is:

[0070]

[0071] Where: IOU adj To correct the fitted value of the feed amount, S adj To correct the cumulative value of the material inspection frame area, S all This is the cumulative value of the material-carrying area of ​​the conveyor belt throughout the entire image cycle.

[0072] The mineral mixture involved in the method includes three or more different components. When there are three mineral mixture components, the three mixture components are a, b and c respectively. In step two, the compositions of the different mixtures are a, b, c, ab, ac, bc and abc respectively.

[0073] In short, the target scenario is a mineral mixture consisting of components a, b, and c, which are successively fed onto a conveyor belt. The goal is to promptly identify when one or more of the raw materials are not being fed correctly.

[0074] The following section uses the application of this method in the proportioning test of raw materials (limestone, loess, and iron-aluminum corrective material mixture) in a cement production line as a special case to illustrate the specific details of this technical solution.

[0075] Example 1

[0076] like Figure 1 As shown, taking cement clinker production as an example, at the beginning of the raw meal process, the raw meal mixing equipment runs empty, and crushed limestone is fed onto the empty belt. Video analysis point 1 is set on the conveyor belt after the crushed limestone is fed, and the limestone feeding is automatically monitored through video data captured by the camera. After the limestone is fed, crushed loess is fed onto the conveyor belt, and video analysis point 2 is set on the limestone and loess mixing section to monitor the ratio of limestone and loess feeding in real time. The corrective material is mixed into the original production line's batching section, and video analysis point 3 is set to monitor the feeding of the corrective material in real time. The data from each video analysis point is transmitted to the raw meal composition monitoring system. The raw meal composition monitoring system calculates and analyzes the video data, and transmits abnormal values ​​to the cement production operation and maintenance-inspection system to monitor and adjust the raw meal ratio. The adjusted mixture enters the calcination system to calcine cement clinker.

[0077] The method for testing the proportion of raw meal (mixture of limestone, loess, and iron-aluminum corrective material) in a cement production line includes the following steps:

[0078] Step 1: At a height of 3 meters above the top surface of the conveyor belt, facing the conveyor belt, install full-color night vision IP cameras with automatic light compensation function at the downstream node of the production process. These cameras generate video with a resolution of up to 4 megapixels. The IP cameras use the manufacturer's accompanying video processing software to convert the video stream into MP4 files. Over two consecutive weeks, collect five minutes of video data at four different times: 8:00 AM, 12:00 PM, 6:00 PM, and midnight. Obtain "Conveyor Belt Empty Operation Video," "Limestone Loading Operation Video," "Limestone and Loess Mixture Loading Operation Video," and "Complete Mixed Raw Material Loading Operation Video" from the production line. Figure 2 As shown;

[0079] Step two: Based on the production line operating speed, extract image sequences from the video. The time interval between image extractions is controlled so that the content of each pair of images does not overlap. For example, if the video frame rate FR = 25 FPS, the conveyor belt speed is V = 1 m / s, and the length of the conveyor belt within the field of view is L = 2 m, then the image sequence extraction interval is (L / V) * FR = 50, meaning one image is taken from every 50 images as the dataset. Different conveyor belt material loading part classification-detection datasets are generated, such as... Figure 3 As shown; and supplemented with image data under various viewing conditions, such as Figure 4 As shown;

[0080] Step 3: For the data from the "limestone" monitoring point, the empty band running data is used as negative samples. Based on the general machine learning framework PyTorch, a binary classification model is trained using the deep learning network MobileNetV2. The images are cropped and reorganized to a size of 256×256 pixels. Using the default stochastic gradient descent method and conventional hyperparameters such as a learning rate of 0.0001, the model is used to predict the image sequence of the test set, which can determine which image corresponds to the time point when limestone was not properly delivered.

[0081] Step 4: For the "limestone + loess" monitoring point, the limestone conveyor belt data is used as negative samples. Based on the general machine learning framework PyTorch, a binary classification model is trained using the deep learning network MobileNetV2 with the same parameters as in Step 3. Using this model, the test set image sequence is predicted, and it can be determined which image corresponds to the time point when loess was not properly delivered.

[0082] Step 5: For the monitoring point of "limestone + loess + correction material", since the correction material is intermittently added, it also appears on the coating where it is added. Therefore, the location box of the correction material on the coating is marked as the detection dataset. The detection model is trained using the YOLOv5 deep learning network. The images are cropped and reorganized to a size of 640×640 pixels. The weight file is set to YOLOv5s. The default stochastic gradient descent method is used to train for 200 epochs to obtain the correction material detection model. The model is used to predict the image sequence of the test set. It can determine which image corresponds to the time node where the correction material was added. The proportion of the correction material detection box in the conveyor belt carrying area is used as the estimated value of the amount of correction material added.

[0083] Step Six: Analyze the feeding results. The calibration material is fed intermittently, with the time span between two normal feeding cycles defined as the feeding period. Within this period, the proportion (n / m) of the number of images (n) showing missing limestone and loess is calculated to represent the degree of limestone and loess deficiency. The cumulative value of the calibration material detection frames is also calculated as a percentage of the total number of images (m) within the entire period. Figure 5As shown, box 1 is the conveyor belt carrying area after position correction, area A is other materials, area B is the last correction material added, box 2 is the detection box, and the proportion of the accumulated value of box 2 to the accumulated value of box 1 is calculated.

[0084] Step 7: Based on the practical experience of front-line quality control engineers, set the actual allowable range of missing materials for each component. For example, the missing proportion of limestone should not be less than 10%, the missing proportion of loess should not be less than 24%, and the area of ​​the correction material should not be less than 8%. By comparing the test results of the three test points, it can be determined whether the gradation of the mixture is normal. If it is abnormal, calculate the difference between the material input and the target, and report the results to the production operation and maintenance system. On-site personnel can promptly learn about the missing materials and track the cause based on the reported data.

[0085] This solution can be applied to the field of lean production intelligent monitoring of other mineral mixtures. The above figures are for illustrative purposes only, representing schematic diagrams rather than actual objects, and should not be construed as limiting this patent. To better illustrate the specific embodiments of the invention, some components in the figures may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable that those skilled in the art will find that some well-known structures and their descriptions may be omitted in the figures. Based on the specific embodiments of this invention, all other specific embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0086] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A real-time detection method of mineral mixture components based on vision, used for raw material proportioning detection method of cement production line, characterized in that, Comprising the following steps: Step one, select IP camera and install: select full-color night vision camera with automatic compensation light source function, camera image quality meets analysis requirements, and install the camera above each mineral mixture component discharging equipment, the IP camera converts video stream into MP4 through the manufacturer's video background software; Step two, obtain imaging video of different material composition: control the on-off of each mineral mixture component discharging equipment at different time periods, record the time period, and collect 5 minutes of video data at different time periods within two weeks from the IP camera video background software, to obtain "conveyer empty load video", "loading limestone running video", "loading limestone and loess mixture running video" and "loading complete mixed raw material running video" on the production line; Step three, generate target detection data set: use OpenCV image processing library to extract picture sequence in the video according to the production line running speed; Step four, target detection set analysis and calculation: use deep learning network to analyze multiple points in a standard continuous load transmission video, and based on the general machine learning framework, the picture data is trained and learned by deep learning network to calculate the proportion of each mixed material segment in a feeding period; Step five, count the feeding result; Step six, set the error allowable range: according to the actual experience of process control engineers, set the actual allowable error range of material composition, count the detection results of monitoring points, and report the results to the production operation and maintenance system, when the material ratio exceeds the threshold value for a long time, timely and actively prompt the production line workers of abnormal material mixing, so that the on-site personnel can know the abnormal material mixing and track the abnormal reasons in time; The specific process of target detection set analysis and calculation in step four is as follows: For the "limestone" monitoring point, use empty belt running data as negative sample, use deep learning network MobileNetV2 to train two-classification model under general machine learning framework Pytorch, cut and arrange the picture to 256x256 pixels size, use random gradient descent method and 0.0001 learning rate to form test set image prediction model, use the model to predict test set image sequence, and judge which picture is not normally fed with limestone at the corresponding time node; For the "limestone + loess" monitoring point, use limestone conveyer data as negative sample, use deep learning network MobileNetV2 to train two-classification model, cut and arrange the picture to 256x256 pixels size, use random gradient descent method and 0.0001 learning rate to form test set image prediction model, use the model to predict test set image sequence, and judge which picture is not normally fed with loess at the corresponding time node; For the "limestone + loess + correction material" monitoring point, the correction material is intermittently put into the position box of the correction material on the graph, and the area in the position box is used as the detection data set. The target detection model is trained using the deep learning network YoloV5. The picture is cut and arranged to 640x640 pixels in size. The random gradient descent method and the learning rate of 0.0001 are used to form the test set image prediction model. The model is used to predict the test set image sequence. It is judged which picture put the correction material at the corresponding time node. The proportion of the correction material detection box in the conveying belt load area is used as the estimated value of the correction material put quantity. The light in the step two is different, which is 8 o'clock in the morning, 12 o'clock at noon, 6 o'clock in the evening and 12 o'clock at midnight; the pixel of the camera in the step one is greater than or equal to 4 million; the feeding period in the step four is the time span of the longest interval between two feeding components; The specific way of the step five is: The number of pictures of limestone and loess missing in the feeding period is counted to obtain the proportion of the number of pictures of limestone and loess missing in the whole cycle image number, and the proportion is taken as the material shortage degree value of limestone and loess. The proportion of the cumulative value of the correction material detection box in the whole cycle is counted. The formula of the material shortage degree value of limestone and loess is: ； In the formula: is the material deficiency value of limestone and loess, is the number of images of limestone and loess deficiency, is the number of images in the whole cycle; The formula of the correction material put quantity is: ； In the formula: is the cumulative value of the correction material detection frame area, is the cumulative value of the correction material detection frame area, is the cumulative value of the correction material detection frame area.

2. A method for real-time detection of the ingredients of a mineral mixture based on vision as claimed in claim 1, characterized in that, The time interval of the step three is controlled to be just not coincident with the content of two pictures, and the formula of the image sequence extraction interval is: ； wherein: is the extraction interval of the image sequence, is the physical length of the conveyor belt within the field of view, is the running speed of the conveyor belt, is the frame rate of the video.

Images