A dynamic real-time monitoring, analysis and processing system for coal flow based on image recognition

Abstract

The present invention belongs to the technical field of coal flow monitoring, analysis and processing, and specifically discloses a coal flow dynamic real-time monitoring, analysis and processing system based on image recognition, including a coal flow image sampling module, an environmental information detection module, a coal image detection module, a coal flow template generation module and a coal flow template update module. The present invention effectively solves the problem that the influence of the external environment on image detection is not considered during the current coal flow monitoring, reduces the tediousness of subsequent coal flow determination, and provides a strong guarantee for the subsequent accurate determination of coal flow. At the same time, by generating the current coal flow laser line template and updating the coal flow laser line template, the dynamic and real-time update of the coal flow laser line template is realized, which can better deal with the coal flow accuracy error caused by belt offset and belt up and down floating, ensure the consistency and representativeness of the coal flow measurement results, and thus provide a reliable foundation for the determination of coal flow.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal flow monitoring, analysis and processing, and in particular, relates to a coal flow dynamic real-time monitoring, analysis and processing system based on image recognition. Background Art

[0002] With the rapid development of economy, the application of intelligent detection in coal mining is becoming more and more extensive. In order to ensure the safety of coal mining and operation management, it is necessary to use various advanced image detection and recognition technologies to improve the production environment of coal mines and enhance their risk resistance. Among them, coal flow detection is widely used because it can monitor the changes in the amount of coal on the belt 24 hours a day.

[0003] Prior art, such as a coal flow monitoring method, device, electronic device and storage medium based on image segmentation disclosed in a Chinese invention patent application with application number 202011101751.4, includes extracting a test image from a video of the process of conveyor belt transporting coal, inputting the test image into a trained super-resolution segmentation network model, and obtaining a segmentation feature image corresponding to the test image output by the trained super-resolution segmentation network model. The segmented feature image is visualized to obtain a visualization image corresponding to the segmented feature image, wherein the visualization image includes a coal area, a conveyor belt area and a background area. The coal flow is determined based on the segmented area of ​​the coal area and the conveyor belt area in the visualization image. This solves the current problem of low detection accuracy of coal flow and improves the detection accuracy of coal flow.

[0004] With regard to the above technical solutions, it is obvious that the current coal flow monitoring has the following problems: 1. Image detection is greatly affected by the environment, thus affecting the detection results. When currently monitoring coal flow, the impact of the external environment on image detection is not considered, which increases the complexity of determining the subsequent coal flow and also affects the accuracy of determining the subsequent coal flow.

[0005] 2. The current coal flow laser line template is fixed, and the belt offset is not taken into account. The coal flow accuracy error caused by the up and down floating of the belt leads to deviations in the coal flow measurement results. It also leads to a lack of reliability in the generated data, resulting in erroneous production plans and scheduling, and affecting the efficiency and output of the entire production line. It may also bring safety hazards. Summary of the invention

[0006] In view of this, in order to solve the problems raised in the above background technology, a coal flow dynamic real-time monitoring, analysis and processing system based on image recognition is proposed.

[0007] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides a dynamic real-time monitoring, analysis and processing system for coal flow based on image recognition, including: a coal flow image sampling module, which is used to intercept image samples from the video stream transmitted in real time by the lower computer according to a preset time interval when receiving a sampling instruction sent by the upper computer, and to form an image sample set with each intercepted image sample, and to record the current camera parameters at the same time.

[0008] The environmental information detection module is used to detect the environment in the current image acquisition area through a pre-installed environmental monitoring terminal to obtain the detected environmental information.

[0009] The coal image detection module is used to perform imaging quality detection and analysis and output the imaging quality score. If the imaging quality score is lower than the set value, the coal flow image is collected and processed, and a resampling instruction is sent and returned to the coal flow image sampling module. Otherwise, image content detection and processing are performed, and the original coal flow line image is output and sent to the coal flow template generation module.

[0010] The coal flow template generation module is used to generate the current coal flow laser line template based on the original coal flow line image.

[0011] The coal flow template update module is used to transmit the current coal flow laser line template to the lower computer and update the coal flow laser line template.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention effectively solves the problem of not considering the impact of the external environment on image detection during current coal flow monitoring by performing environmental detection and imaging quality detection and analysis, thereby reducing the tediousness of subsequent coal flow determination and providing a strong guarantee for the accurate determination of subsequent coal flow.

[0013] (2) When the imaging quality score is lower than the set value, the present invention determines the environmental interference and the processing method, determines the processing indicators according to different processing methods, and adjusts the camera, thereby achieving targeted and advanced processing of the camera, ensuring the imaging quality and imaging effect of the camera image, and providing a reliable foundation for determining the coal flow.

[0014] (3) The present invention outputs the original coal flow line image by performing image content detection and processing, and at the same time realizes the dynamic and real-time updating of the coal flow laser line template by generating the current coal flow laser line template and updating the coal flow laser line template. It can better cope with the coal flow accuracy error caused by belt offset and belt up and down floating, ensure the consistency and representativeness of the coal flow measurement results, and also improve the reliability of the generated data, avoid the subsequent erroneous production plan and scheduling, improve the efficiency and output of the entire production line to a certain extent, and reduce safety hazards as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the connection of each module of the system of the present invention.

[0017] Figure 2 It is a schematic diagram of the overall process of dynamic real-time monitoring, analysis and processing of coal flow in the present invention.

[0018] Figure 3 This is a flow chart of image content detection and processing according to the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] See also Figure 1 and Figure 2 As shown, the present invention provides a coal flow dynamic real-time monitoring, analysis and processing system based on image recognition, the system comprising: a coal flow image sampling module, an environmental information detection module, a coal image detection module, a coal flow template generation module and a coal flow template update module.

[0021] In the above, the coal image detection module is respectively connected with the coal flow image sampling module, the environmental information detection module and the coal flow template generation module, and the coal flow template generation module is also connected with the coal flow template update module.

[0022] The coal flow image sampling module is used to intercept image samples from the video stream transmitted in real time by the lower computer according to a preset time interval when receiving a sampling instruction sent by the upper computer, compose each intercepted image sample into an image sample set, and record the current camera parameters at the same time.

[0023] Specifically, the current camera parameters include but are not limited to exposure time, ISO sensitivity and exposure compensation value.

[0024] In a specific embodiment, the preset time interval may be two seconds, and for the convenience of analysis, the number of captured image samples may be 10.

[0025] The environmental information detection module is used to detect the environment in the current image acquisition area through a pre-installed environmental monitoring terminal to obtain detected environmental information.

[0026] Specifically, the detection environment information includes the detection light brightness and dust concentration corresponding to each detection point.

[0027] Among them, the light brightness is detected by a photosensor, and the dust concentration is detected by a dust sensor.

[0028] The coal image detection module is used to perform imaging quality detection and analysis and output an imaging quality score. If the imaging quality score is lower than a set value, the coal flow image is collected and processed, and a resampling instruction is sent and returned to the coal flow image sampling module. Otherwise, image content detection and processing are performed, and the original coal flow line image is output and sent to the coal flow template generation module.

[0029] The embodiments of the present invention effectively solve the problem of not considering the impact of the external environment on image detection during current coal flow monitoring by performing environmental detection and imaging quality detection and analysis, thereby reducing the cumbersomeness of subsequent coal flow determination and providing a strong guarantee for the accurate determination of subsequent coal flow.

[0030] Specifically, the imaging quality detection and analysis includes: A1. For each intercepted image sample in the image sample set, the imaging quality characteristics are calculated, including blur ratio, abnormal noise ratio, contrast difference and contrast deviation ratio, which are respectively recorded as , , and .

[0031] A2. , , and The normalization processing is performed respectively, and the results after normalization are used as variable 1, variable 2, variable 3 and variable 4, respectively marked as , , and .

[0032] A3. Set the degree of fit to variable 1, denoted as ,according to The setting method of is similar to setting the matching degree of variables 2, 3 and 4, which are recorded as , and .

[0033] A4. Define the weights of fuzzy ratio, abnormal noise ratio, contrast difference and contrast deviation ratio, which are respectively denoted as , , and , .

[0034] A4. Statistical imaging quality score , , It is the total score value of the preset imaging quality.

[0035] It should be added that the preset total score of the imaging quality can be a ten-point system, a hundred-point system, or any custom score. For the convenience of subsequent comparison and analysis, for example, Can be set to 10.

[0036] It should also be added that when defining the weights of the fuzzy ratio, abnormal noise ratio, contrast difference and contrast deviation ratio, it is necessary to determine them according to the needs of the actual application scenario. For example, since the fuzzy ratio directly affects the clarity and recognition accuracy of the image, the fuzzy ratio is set to have the largest weight. For example, The value can be 0.4. The abnormal noise ratio will affect the details and recognizability of the image, so the weight of the abnormal noise ratio is set second. For example, The value can be 0.3. Although the contrast difference will affect the image quality, the impact is relatively small and mainly affects the imaging settings. Therefore, the weight of the contrast difference is set to the minimum. For example, The value can be 0.1, and the contrast deviation ratio reflects the overall contrast deviation, which is an evaluation indicator at the deviation breadth level. Therefore, the contrast deviation ratio is set to be slightly larger than the contrast difference weight. For example, The value can be 0.2, that is .

[0037] Furthermore, regarding the calculation of imaging quality features in step A1, it includes: A11, for each clipped image sample, calculating its gradient value through an edge detection operator to obtain the gradient value of each clipped image sample.

[0038] In a specific embodiment, the edge detection operator includes but is not limited to the Laplace operator, the Sobel operator and the Prewitt operator. Specifically, one of them or multiple of them can be used for comprehensive calculation. Calculating the gradient of the image by the edge detection operator is a relatively mature processing method, and the edge detection operator involved is a relatively mature edge detection operator. The specific calculation process and specific calculation formula are not shown here.

[0039] A12. Record the screenshot image samples whose gradient values ​​are less than the set reference gradient value as blurred samples, count the number of blurred samples, and divide it by the number of screenshot image samples to obtain a blur ratio.

[0040] A13. Calculate the standard deviation of the pixel value corresponding to each intercepted image sample as the noise value of the intercepted image sample.

[0041] It should be added that calculating the standard deviation of the pixel value corresponding to each intercepted image sample in the image sample set belongs to a link of noise estimation. For example, the present invention can use a noise variance estimation method based on a local block to perform noise estimation and pixel value standard deviation calculation, which can be done by dividing the image into small blocks, identifying the number of pixels in each small block through image recognition technology, calculating the variance of each small block, and then averaging. Among them, image recognition technology is a relatively mature technology, and its specific recognition process will not be repeated.

[0042] A14. Record the intercepted image samples whose noise values ​​are greater than the reference noise value as abnormal noise image samples, and divide the values ​​by the number of intercepted image samples to obtain an abnormal noise ratio.

[0043] A15. Calculate the contrast of each captured image sample, and calculate the standard deviation of the contrast of each captured image sample, and use the calculation result as the contrast difference.

[0044] A16. The captured image samples whose contrast exceeds the set appropriate contrast range are recorded as contrast deviation samples. The number of contrast deviation samples is counted and divided by the number of captured image samples to obtain the contrast deviation ratio. The blur ratio, abnormal noise ratio, contrast difference and contrast deviation ratio are used as imaging quality features.

[0045] In a specific embodiment, the specific value of the appropriate contrast interval needs to be determined in combination with actual monitoring equipment and technical requirements. For example, the appropriate contrast interval may be 30 to 225, where 30 and 225 represent grayscale values.

[0046] It should be added that the contrast of each captured image sample in the image sample set can be measured by calculating the ratio of the maximum brightness to the minimum brightness of the image, that is, the ratio of the maximum pixel value to the minimum pixel value in the image.

[0047] It should also be added that for dynamic monitoring of coal flow, the image needs to be clear enough to identify the characteristics of coal, such as shape, size, etc. Blurred images may lead to recognition errors, while appropriate contrast helps to highlight the difference between coal and background and improve detection accuracy. It is also necessary to minimize the noise in the image, such as electronic noise and environmental interference. These factors will reduce the quality of the image and thus affect the recognition effect. Therefore, the gradient value, noise value and contrast are selected as the main factors for evaluation in imaging quality detection. Among them, the larger the gradient value, the clearer the image.

[0048] Understandably, in video surveillance, ideally, if the monitoring environment and camera settings remain unchanged, the contrast of multiple image samples captured from the video stream should be relatively consistent. Therefore, when performing contrast analysis, the standard deviation of the contrast corresponding to different captured image samples is calculated as one of the considerations. The gradient value reflects the rate of change of pixel intensity in the image, that is, the steepness of the image edge. When an object moves, the edge in the image will change, causing the gradient value to change. The noise value reflects the degree of random change in the image. The movement of the object may indirectly affect the perception of noise. Therefore, both the noise value and the gradient value will change due to the shooting of dynamic objects, and their consistency in different captured image samples is not considered.

[0049] Further, in step A2, , , and The supplementary processing flow for normalization is as follows: the permitted fuzzy ratio of the preset image sample is Blur ratio of the captured image sample Subtract the difference and add it to Make a comparison and take the ratio as variable 1, that is, .

[0050] Set the allowed abnormal noise ratio of the preset image samples to Abnormal noise ratio of the captured image sample Subtract the difference and add it to Make a comparison and use the ratio as variable 2, that is, .

[0051] Set the allowed contrast difference of the preset image samples The difference between the sample and the captured image Subtract the difference and add it to Make a comparison and use the ratio as variable 3, that is, .

[0052] Set the allowed contrast deviation ratio of the preset image samples to Contrast deviation ratio with the captured image sample Subtract the difference and add it to Make a comparison and use the ratio as variable 4, that is, .

[0053] In a specific embodiment, the allowed blur ratio of the preset image sample is , Permitted abnormal noise ratio Comparison with the permitted deviation ratio The settings are made according to the number of captured image samples. For example, when the number of captured image samples is 100, The value can be 0.05. The value can be 0.1. The value can be 0.2. When the difference between the maximum grayscale value and the minimum grayscale value is used as the contrast, the preset permissible contrast difference of the image sample The specific value can be 0.2.

[0054] Understandably, in image recognition and analysis tasks, the clarity of the image directly affects the accuracy and reliability of recognition. Blurred images can cause unclear edges and difficult to identify features, which seriously affects the recognition results. Therefore, the permitted blur ratio is usually set low, usually lower than the tolerance of the abnormal noise ratio. Although the change in contrast will affect the recognizability of the image, in some cases, even if the contrast is slightly deviated, the main features of the image can still be recognized. Because the tolerance for the permitted contrast deviation ratio is higher than the tolerance for the abnormal noise ratio.

[0055] Specifically, the coal flow image acquisition and processing includes: B1. Extracting the detection light brightness and dust concentration corresponding to each detection point from the detection environment information, and judging whether there is environmental interference. If there is no environmental interference, the processing method is recorded as internal processing. If there is environmental interference, confirm the current environmental interference index, including one or more of dust and light.

[0056] B2. If dust exists in the current environmental interference indicator, the camera self-cleaning instruction is activated to perform automatic camera cleaning. Otherwise, the processing method is recorded as comprehensive processing.

[0057] B3. If the current processing mode is internal processing, confirm the processing indicators, including exposure time, ISO sensitivity and exposure compensation value.

[0058] B4. Extract the exposure time, ISO sensitivity and exposure compensation value from the current camera parameters and record them as , and .

[0059] B5. When the processing indicator is exposure time, As a way to adjust the exposure time and make camera adjustments, is a natural constant, It is the preset camera base exposure time.

[0060] It should be added that ,in, Represents the exposure time adjustment factor, and its value range is (0,1). The purpose is to adjust the exposure time more accurately. The calculation principle of subsequent ISO sensitivity and exposure compensation value is the same, so it will not be repeated.

[0061] B6. When the processing index is ISO sensitivity, As an adjustment of ISO sensitivity, and making camera adjustments.

[0062] B7. When the processing indicator is the exposure compensation value, As an adjustment of exposure compensation value, and camera adjustment, The base exposure compensation value is set.

[0063] B8. If the treatment method is comprehensive treatment, set the environmental interference factor , and correct the adjustment processing index, and make camera adjustments based on the corrected results.

[0064] It should be added that the specific correction example process of correcting the adjustment processing index is as follows: Correct the adjustment exposure time: record the adjustment exposure time as ,Will As a corrected adjusted exposure duration.

[0065] Correct ISO sensitivity: Adjust ISO sensitivity as ,Will As a corrected adjustment of ISO sensitivity, Indicates the set base ISO sensitivity.

[0066] Correct the exposure compensation value: record the adjusted exposure compensation value as ,Will As the corrected adjusted exposure compensation value.

[0067] It is also necessary to add that , and The specific value of is determined by the technical specifications of the shooting equipment and can be provided by the manufacturer of the shooting equipment.

[0068] Furthermore, judging whether there is environmental interference in step B1 includes: calculating the standard deviation of the detection light brightness corresponding to each detection point, and using the calculation result as the light brightness distribution difference.

[0069] The dust concentration at each detection point is averaged to obtain the average dust concentration.

[0070] The light brightness distribution difference and the average dust concentration are compared with the corresponding set reference interference values ​​respectively. If one of them is greater than the corresponding set reference interference value, the existence of environmental interference is judged as the result; if the light brightness distribution difference and the average dust concentration are both less than or equal to the corresponding set reference value, the absence of environmental interference is judged as the result.

[0071] In a specific embodiment, the set reference value corresponding to the light brightness distribution difference can be specifically set to 10 illumination degrees, and the set reference value corresponding to the average dust concentration can be specifically set to 1 mg / m³.

[0072] Furthermore, confirming the current environmental interference index in step B1 includes: if only the light brightness distribution difference is greater than the corresponding reference value, the light is used as the current environmental interference index; if only the average dust concentration is greater than the corresponding reference value, the dust is used as the current environmental interference index; if both the light brightness distribution difference and the average dust concentration are greater than the corresponding reference value, both the light and the dust are used as the current environmental interference indicators.

[0073] Furthermore, in step B3, confirming the processing index includes: , , and Compare with 0 in turn. If , taking clarity processing as the processing indicator.

[0074] like , taking noise processing as the processing indicator.

[0075] like or , taking contrast processing as the processing index.

[0076] Furthermore, in step B8, the environmental interference factor is set, including: recording the light brightness distribution difference and the average dust concentration as and .

[0077] Setting environmental interference factors , , and They respectively represent the reference interference values ​​for setting the light brightness distribution difference and dust concentration.

[0078] When the imaging quality score is lower than the set value, the embodiment of the present invention determines the environmental interference and confirms the processing method, determines the processing indicators according to different processing methods, and performs camera adjustments, thereby achieving targeted and advanced processing of the camera, ensuring the imaging quality and imaging effect of the camera image, and providing a reliable foundation for determining the coal flow.

[0079] More specifically, see Figure 3 As shown, image content detection and processing are performed, including: C1, traversing the image sample set, determining whether the image sample set is static, and if not static, sending a resampling instruction and returning to the coal flow image sampling module.

[0080] C2. If it is still, the image sample set is imported into the deep learning detection model to determine whether there is coal. If there is coal, a resampling instruction is sent and the coal flow image sampling module is returned.

[0081] C3. If there is no coal, a cut image sample is randomly selected from the image sample set as the original coal streamline image.

[0082] Furthermore, the step of judging whether the image sample set is still in step C1 includes: C11, extracting a first clipped image sample from the image sample set, and taking the outline of the first clipped image sample as a standard outline.

[0083] C12. Downsample the contour of each captured image sample, mark the contour sampling points, and record the number of contour sampling points.

[0084] It should be added that downsampling the contour of each captured image sample refers to reducing the number of points on the contour while keeping the basic shape of the contour unchanged. The purpose of downsampling is to reduce the amount of calculation and improve the processing speed, especially when multiple repeated calculations are required, such as contour matching. In a specific embodiment, downsampling can be achieved by a variety of methods, common methods include uniform sampling, non-uniform sampling, and sampling based on contour features. Among them, the method used for downsampling is an existing common method and belongs to the existing technical means. Its specific sampling process is not repeated here.

[0085] C13, through The function calculates the distance between each contour sampling point and the standard contour as the contour distance.

[0086] It should be added that the use The function is used to calculate the distance between the contour sampling point and the standard contour. It is a common image processing and computer vision technology. It is a relatively mature technology. Its specific calculation process will not be repeated here.

[0087] C14. Compare the contour distance of each contour sampling point with a preset reference contour distance. If the contour distance of a contour sampling point is less than or equal to the reference contour distance, the contour sampling point is recorded as a matching sampling point.

[0088] C15. Count the number of matching sampling points and compare it with the number of contour sampling points, and use the ratio as the contour similarity.

[0089] C16. Calculate the mean of the contour similarities of each intercepted image sample to obtain the average contour similarity of the intercepted samples.

[0090] C17. If the average contour similarity of the intercepted samples is greater than the set reference contour similarity, the image sample set is judged to be static; if the average contour similarity of the intercepted samples is less than or equal to the set reference contour similarity, the image sample set is judged to be not static.

[0091] In a specific embodiment, the set reference contour similarity may be specifically set to a value of 0.85.

[0092] Understandably, what needs to be added about step C2 is that the working principle of importing the image sample set into the deep learning detection model to determine whether there is coal includes the following steps: 1. Data preprocessing: preprocess each intercepted image sample in the image sample set, and establish and train the coal flow recognition model.

[0093] It should be added that preprocessing such as scaling and normalization is performed to facilitate subsequent feature extraction of the trained foreign body recognition model.

[0094] 2. Feature extraction: The trained coal flow recognition model uses deep neural network structures such as convolutional neural networks to extract laser line-related features from each intercepted image sample in the preprocessed input image sample set.

[0095] 3. Classification and positioning: Based on the extracted features related to the laser lines, algorithms such as classifiers and bounding box regressors are used to classify and locate the coal flow laser lines. If the model is called and the result returned is 1, it means there is coal. If the model is called and the result returned is 0, it means there is no coal.

[0096] In a specific embodiment, the features related to the laser line are mainly concentrated on the appearance of the laser line in the image. These features can help identify the position, shape, intensity and other information of the laser line. For example, the features related to the laser line include the position, shape, direction, brightness and contrast of the laser line.

[0097] It should be added that extracting features related to laser lines and using algorithms such as classifiers and bounding box regressors to classify and locate coal flow laser lines are relatively mature processing methods, and the specific processing process will not be repeated here.

[0098] The coal flow template generation module is used to generate a current coal flow laser line template based on the original coal flow line image.

[0099] Specifically, generating the current coal flow laser line template includes: D1, converting the original coal flow line image into a grayscale image.

[0100] D2. Read the size of the original coal flow image and generate a blank image of the same size as the output template.

[0101] D3. Adaptively threshold segment the grayscale image using the Otsu method to obtain a binary image. The binary image is used as input, and a contour search algorithm is called to search for contours in the binary image to obtain contours in the binary image.

[0102] It should be added that the contour search function will traverse the pixels in the image, determine the boundaries of the contour based on the black and white changes of the pixels, that is, the edges, and store the contour information in a specific data structure, such as a linked list, array, etc. Each contour is usually composed of a series of coordinate points, indicating the shape and position of the contour.

[0103] It should also be added that the contour search algorithm is mainly used to identify and segment objects in an image. Contour search is usually performed based on the results of edge detection, and its purpose is to find the boundary of the object in the image and represent it as a set of continuous points. Exemplarily, the contour search algorithm includes but is not limited to Canny edge detection, Sobel operator, Laplacian operator and contour tracing algorithm. For the convenience of analysis, Canny edge detection can be selected as the contour search algorithm described in the present invention.

[0104] D4. Extract the area of ​​each contour in the binary image, perform the first filtering on the contours whose area is less than the set area threshold, calculate the minimum circumscribed circle area of ​​each contour remaining after the first filtering, and if the ratio of the minimum circumscribed circle area of ​​a contour to the area of ​​the contour is less than the set coefficient, perform the second filtering on the contour.

[0105] D5. For each contour remaining after the secondary filtering, calculate the particle position of each contour, sort the horizontal coordinates of the particle position of each contour remaining after the secondary filtering, and obtain a sorted particle position sequence.

[0106] It should be added that the position of the particle can be obtained by taking a weighted average of the coordinates of all points on the contour. For example, for the coordinates of each point on the contour, the abscissa of the particle is the sum of the abscissas of each point divided by the number of points on the contour.

[0107] D6. If the distance between a particle and its adjacent particles exceeds the set threshold, the particle is recorded as a prominent point and the contour corresponding to the prominent point is filtered.

[0108] It should be added that the distance calculation can be based on the distance between points on the contour or between particles. Find the position where the distance between each adjacent contour is the smallest.

[0109] In a specific embodiment, the selection of the threshold value corresponding to a particle and adjacent particles needs to consider multiple factors, including but not limited to the resolution, feature scale, noise level of the image, and accuracy requirements of target detection. For example, the threshold value can be set to 7 pixels.

[0110] D7. For the sorted adjacent contours, calculate the distance between them, locate the position with the smallest distance between each adjacent contour, and connect the adjacent contours with line segments in the output template according to the minimum distance between adjacent contours in the sorted order. Draw the connected contours on the output template and use them as the current coal flow laser line template.

[0111] The coal flow template updating module is used to transmit the current coal flow laser line template to the lower computer and update the coal flow laser line template.

[0112] The embodiment of the present invention outputs the original coal flow line image by performing image content detection and processing, and at the same time realizes the dynamic and real-time updating of the coal flow laser line template by generating the current coal flow laser line template and updating the coal flow laser line template. It can better cope with the coal flow accuracy error caused by belt offset and belt up and down floating, ensure the consistency and representativeness of the coal flow measurement results, and also improve the reliability of the generated data, avoid the subsequent erroneous production planning and scheduling, improve the efficiency and output of the entire production line to a certain extent, and reduce the safety hazards as much as possible.

[0113] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A coal flow dynamic real-time monitoring, analysis and processing system based on image recognition, characterized in that: include: The coal flow image sampling module is used to intercept image samples from the video stream transmitted in real time by the lower computer according to a preset time interval when receiving a sampling instruction sent by the upper computer, and to form an image sample set with each intercepted image sample, and to record the current camera parameters at the same time; An environmental information detection module is used to detect the environment in the current image acquisition area through a pre-installed environmental monitoring terminal to obtain detection environment information; The coal image detection module is used to perform imaging quality detection and analysis and output the imaging quality score. If the imaging quality score is lower than the set value, the coal flow image acquisition and processing is performed, and a resampling instruction is sent and returned to the coal flow image sampling module. Otherwise, the image content detection and processing is performed, the original coal flow line image is output, and sent to the coal flow template generation module; A coal flow template generation module is used to generate a current coal flow laser line template based on the original coal flow line image; The coal flow template update module is used to transmit the current coal flow laser line template to the lower computer and update the coal flow laser line template; The imaging quality detection and analysis comprises: For each intercepted image sample in the image sample set, the imaging quality characteristics are calculated, including blur ratio, abnormal noise ratio, contrast difference and contrast deviation ratio, which are respectively denoted as , , and ; Will , , and The normalization processing is performed respectively, and the results after normalization are used as variable 1, variable 2, variable 3 and variable 4, respectively marked as , , and ; Will Set the degree of fit to variable 1, denoted as ,according to The setting method of is similar to setting the matching degree of variables 2, 3 and 4, which are recorded as , and ; The weights of fuzzy ratio, abnormal noise ratio, contrast difference and contrast deviation ratio are defined as , , and , ; Statistical imaging quality score , , It is the total score value of the preset imaging quality; The calculated imaging quality features include: For each intercepted image sample, its gradient value is calculated by edge detection operator to obtain the gradient value of each intercepted image sample; The screenshot image samples whose gradient values ​​are less than the set reference gradient value are recorded as blurred samples, the number of blurred samples is counted, and the number is divided by the number of screenshot image samples to obtain the blur ratio; Calculate the standard deviation of the pixel value corresponding to each intercepted image sample as the noise value of the intercepted image sample; The intercepted image samples whose noise values ​​are greater than the reference noise value are recorded as abnormal noise image samples, and divided by the number of intercepted image samples to obtain the abnormal noise ratio; Calculate the contrast of each intercepted image sample, and calculate the standard deviation of the contrast of each intercepted image sample, and use the calculation result as the contrast difference; The intercepted image samples whose contrast exceeds the set appropriate contrast range are recorded as contrast deviation samples. The number of contrast deviation samples is counted and divided by the number of intercepted image samples to obtain the contrast deviation ratio. The blur ratio, abnormal noise ratio, contrast difference and contrast deviation ratio are used as imaging quality features. The coal flow image acquisition and processing comprises: Extract the detection light brightness and dust concentration corresponding to each detection point from the detection environment information, and determine whether there is environmental interference. If there is no environmental interference, record the processing method as internal processing. If there is environmental interference, confirm the current environmental interference index, including one or more of dust and light. If dust exists in the current environmental interference index, the camera self-cleaning instruction is activated to perform automatic camera cleaning, otherwise, the processing method is recorded as comprehensive processing; If the current processing mode is internal processing, confirm the processing indicators, including exposure time, ISO sensitivity and exposure compensation value; The exposure time, ISO sensitivity and exposure compensation value are extracted from the current camera parameters and recorded as , and ; When the processing indicator is exposure duration, As a way to adjust the exposure time and make camera adjustments, is a natural constant, The preset camera baseline exposure time; When the processing index is ISO sensitivity, As an adjustment of ISO sensitivity, and camera adjustments; When the processing indicator is the exposure compensation value, As an adjustment of exposure compensation value, and camera adjustment, is the set base exposure compensation value; If the processing method is comprehensive processing, set the environmental interference factor, and correct the adjustment processing index, and make camera adjustments based on the corrected results; The determining whether there is environmental interference includes: Calculate the standard deviation of the detection light brightness corresponding to each detection point, and use the calculation result as the light brightness distribution difference; Calculate the dust concentration at each detection point to get the average dust concentration; The light brightness distribution difference and the average dust concentration are compared with the corresponding set reference interference values. If one of them is greater than the corresponding set reference interference value, the existence of environmental interference is regarded as the judgment result. If the light brightness distribution difference and the average dust concentration are both less than or equal to the corresponding set reference value, the absence of environmental interference is regarded as the judgment result. The confirmation processing indicators include: Will , , and Compare with 0 in turn. If , taking clarity processing as the processing index; like , taking noise processing as the processing indicator; like or , taking contrast processing as the processing index; The setting of the environmental interference factor includes: The light brightness distribution difference and the average dust concentration are recorded as and ; Setting environmental interference factors , , and They respectively represent the reference interference values ​​for setting the light brightness distribution difference and dust concentration.

2. A coal flow dynamic real-time monitoring and analysis processing system based on image recognition as claimed in claim 1, characterized in that: The image content detection and processing comprises: Traverse the image sample set to determine whether the image sample set is static. If not, send a resampling instruction and return to the coal flow image sampling module; If it is still, the image sample set is imported into the deep learning detection model to determine whether there is coal. If there is coal, a resampling instruction is sent and the coal flow image sampling module is returned; If there is no coal, a cut image sample is randomly selected from the image sample set as the original coal streamline image.

3. The coal flow dynamic real-time monitoring and analysis processing system based on image recognition as claimed in claim 1, characterized in that: The determining whether the image sample set is still includes: Extracting the first intercepted image sample from the image sample set, and taking the contour of the first intercepted image sample as the standard contour; Down-sample the contour of each intercepted image sample, mark the contour sampling points, and record the number of contour sampling points; pass The function calculates the distance between each contour sampling point and the standard contour as the contour distance; The contour distance of each contour sampling point is compared with the preset reference contour distance. If the contour distance of a contour sampling point is less than or equal to the reference contour distance, the contour sampling point is recorded as a matching sampling point. Count the number of matching sampling points and compare them with the number of contour sampling points, and use the ratio as the contour similarity; The contour similarity of each intercepted image sample is averaged to obtain the average contour similarity of the intercepted samples; If the average contour similarity of the captured samples is greater than the set reference contour similarity, the image sample set is judged to be static; if the average contour similarity of the captured samples is less than or equal to the set reference contour similarity, the image sample set is judged to be not static.

4. The coal flow dynamic real-time monitoring and analysis processing system based on image recognition according to claim 1, characterized in that: The generating of the current coal flow laser line template comprises: Convert the original coal flow line image into a grayscale image; Read the size of the original coal flow image and generate a blank image of the same size as the output template; The grayscale image is segmented by an adaptive threshold using the Otsu method to obtain a binary image, the binary image is used as input, and a contour search algorithm is called to search for contours in the binary image to obtain contours in the binary image; Extract the area of ​​each contour in the binary image, filter the contours whose area is less than the set area threshold for the first time, calculate the minimum circumscribed circle area of ​​each contour remaining after the first filtering, and if the ratio of the minimum circumscribed circle area of ​​a contour to the area of ​​the contour is less than the set coefficient, filter the contour for the second time; For each contour remaining after the secondary filtering, the particle position of each contour is calculated, and the horizontal coordinates of the particle position of each contour remaining after the secondary filtering are sorted to obtain a sorted particle position sequence; If the distance between a certain particle and its adjacent particles exceeds the set threshold, the particle is recorded as a prominent point, and the contour corresponding to the prominent point is filtered; For the sorted adjacent contours, calculate the distance between them, locate the position with the smallest distance between adjacent contours, and connect the adjacent contours with line segments in the output template according to the position of the smallest distance between adjacent contours in the sorted order. Draw the connected contours on the output template and use them as the current coal flow laser line template.

Images