Automatic material stacking and taking method and system for belt conveying materials based on image recognition

Abstract

The invention provides a belt conveying material automatic stacking and taking method and system based on image recognition, and relates to the technical field of image processing, and the method comprises the steps: constructing a material feature grade model; collecting first material accumulation information; performing feature recognition on the first material image information to obtain the humidity grade of the material; calculating a first material taking numerical value of the material to obtain a first material taking sequence; the material taking machine takes materials according to the sequence of the first material taking sequence; collecting second material accumulation information; obtaining the volume grade of the material; calculating a second material taking numerical value of the material to obtain a second material taking sequence; the material taking machine carries out secondary material taking according to the sequence of the second material taking sequence; and a material taking machine arranged at the final sorting opening is used for respectively taking the materials to obtain a plurality of uniform material piles. The technical problems that in the prior art, efficiency is low, potential safety hazards are high, and classification errors are prone to occurring are solved, and the technical effects that efficiency is improved, risks are reduced, materials are accurately classified, and therefore uniform material piles are obtained are achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an automatic stacking and retrieving method and system for belt conveyor materials based on image recognition. Background Art

[0002] In cement plants, materials must be mixed to meet the corresponding humidity and volume standards to avoid affecting the grindability and mill output of the materials, as well as the strength and storage performance of the final cement. Therefore, each material needs to be classified by humidity and volume before mixing, so as to facilitate subsequent drying and crushing operations to obtain standard materials. In order to solve this problem, the traditional classification method sets multiple unloading points on the belt conveyor, and manually unloads materials of different humidity and volume, and then classifies and stacks them. However, this method has significant disadvantages. First, the efficiency of manual operation is low and cannot meet the needs of large-scale raw material classification. Secondly, manual operation has safety hazards, such as working in a high temperature and high dust environment, which is easy to cause occupational diseases. Most importantly, manual operation inevitably introduces subjectivity and instability in the material classification process, which easily leads to classification errors and makes it difficult to ensure the clarity and accuracy of material classification. In this case, the mixed material piles formed have uneven humidity and particle size, which cannot be uniformly processed during the subsequent drying and crushing operations, and cannot be mixed into a uniform material pile, which directly affects the quality of the final product.

[0003] In summary, the existing technology has the following technical problems: low manual operation efficiency, high safety risks, and easy classification errors. It is difficult to ensure the clarity and accuracy of material classification and cannot achieve the goal of uniform mixing, which directly affects the quality of the final product. Therefore, a method is needed to solve the above problems. Summary of the invention

[0004] The present disclosure provides an automatic stacking and retrieving method and system for belt conveyor materials based on image recognition, which is used to solve the technical problems in the prior art that manual operation is inefficient, has high safety hazards, is prone to classification errors, is difficult to ensure the clarity and accuracy of material classification, cannot achieve the goal of uniform mixing, and directly affects the quality of the final product.

[0005] According to a first aspect of the present disclosure, there is provided an automatic stacking and reclaiming method for belt conveyed materials based on image recognition, comprising: Constructing a material characteristic grade model based on historical material production record data, wherein the material characteristic grade model includes a material moisture grade table and a material volume grade table; Collecting first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stocker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; The material reclaimer provided at the first sorting port obtains the first material accumulation information, performs feature recognition on the first material image information, and obtains the moisture levels of the M positioning marked materials in combination with the material feature level model; Calculate the first material picking value of the M positioning mark materials according to the material humidity level and position information, the first material picking value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material picker moves to the positioning mark, the order of magnitude of the humidity value is one hundred times the first time value, integrate the M first material picking values, arrange the first material picking values ​​from large to small, and obtain the first material picking sequence; The material reclaimer reclaims materials according to the order of the first reclaiming sequence, and respectively takes materials of different humidity to Q different belts, and transmits them to the second sorting port through the Q belts; The second material accumulation information on the Q belts is collected by Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes second material position information and second material image information, and the second material position information includes N positioning marks; The material reclaimer provided at the second sorting port obtains the second material accumulation information, calculates the material volume in combination with the volume information in the characteristic grade model, and obtains the volume grades of the N positioning marked materials; Calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information, the second material fetching value is the sum of the volume value and the second time value, the volume value is determined according to the volume level, the second time value is obtained in the same way as the first time value, integrate the N second material fetching values, arrange the second material fetching values ​​from large to small, and obtain the second material fetching sequence; The reclaimer performs secondary reclaiming according to the arrangement of the second reclaiming sequence, and respectively takes materials of different volumes to P different belts, and transports them to the final sorting port through the P belts; P reclaimers arranged at the final sorting port respectively reclaim materials to obtain multiple uniform material piles.

[0006] According to a second aspect of the present disclosure, there is provided an automatic stacking and reclaiming system for belt conveyed materials based on image recognition, comprising: A material characteristic grade model construction module, which is used to construct a material characteristic grade model according to the material historical production record data, wherein the material characteristic grade model includes a material moisture grade table and a material volume grade table; A first material accumulation information acquisition module, the first material accumulation information acquisition module is used to collect first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stacker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; A humidity level acquisition module, which is used to obtain the first material accumulation information through a material reclaimer arranged at the first sorting port, perform feature recognition on the first material image information, and obtain the humidity levels of M positioning marked materials in combination with the material feature level model; The first material fetching sequence acquisition module is used to calculate the first material fetching value of M positioning mark materials according to the material humidity level and position information, the first material fetching value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material fetcher moves to the positioning mark, the order of magnitude of the humidity value is one hundred times of the first time value, and the M first material fetching values ​​are integrated and arranged from large to small to obtain the first material fetching sequence; The first material picking module is used for the material picking machine to pick up materials according to the first material picking sequence, and to pick up materials with different humidity to Q different belts respectively, and to transmit them to the second sorting port through the Q belts; A second material accumulation information acquisition module, the second material accumulation information acquisition module is used to collect second material accumulation information on Q belts through Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes second material position information and second material image information, and the second material position information includes N positioning marks; A volume level acquisition module, which is used to obtain the second material accumulation information from a material reclaimer arranged at the second sorting port, calculate the material volume in combination with the volume information in the characteristic level model, and obtain the volume levels of N positioning marked materials; The second material fetching sequence acquisition module is used to calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information. The second material fetching value is the sum of the volume value and the second time value. The volume value is determined according to the volume level. The second time value is obtained in the same way as the first time value. The N second material fetching values ​​are integrated and arranged from large to small to obtain the second material fetching sequence; The midway material picking module is used for the material picking machine to perform secondary material picking according to the arrangement of the second material picking sequence, and to pick materials of different volumes to P different belts respectively, and to transport them to the final sorting port through the P belts; The final material taking module is used to obtain multiple uniform material piles by P material taking machines arranged at the final sorting port.

[0007] One or more technical solutions provided in this disclosure have at least the following technical effects or advantages: A material characteristic grade model is constructed according to the historical production record data of the material, and the material characteristic grade model includes a material humidity grade table and a material volume grade table; the first material accumulation information on the target belt is collected by a first visual sensor, wherein the first visual sensor is arranged above the stacker, and the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; the first material accumulation information is acquired by a reclaimer arranged at the first sorting port, and the first material image information is feature-recognized, and the humidity grade of the M positioning marked materials is acquired in combination with the material characteristic grade model; the first reclaiming value of the M positioning marked materials is calculated according to the material humidity grade and position information, and the first reclaiming value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity grade, the first time value is determined according to the time when the reclaimer moves to the positioning mark, and the order of magnitude of the humidity value is one hundred times of the first time value, and the M first reclaiming values ​​are integrated, and the first reclaiming values ​​are arranged from large to small to obtain a first reclaiming sequence; the reclaimer reclaims materials according to the arrangement of the first reclaiming sequence, and reclaims materials of different humidity to the first reclaiming sequence respectively. Q different belts are used, and the second material is transmitted to the second sorting port through the Q belts; the second material accumulation information on the Q belts is collected through Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes the second material position information and the second material image information, and the second material position information includes N positioning marks; the second material accumulation information is obtained by the reclaimer arranged at the second sorting port, and the material volume is calculated in combination with the volume information in the feature grade model to obtain the volume grade of the N positioning mark materials; the second reclaiming value of the N positioning mark materials is calculated according to the material volume grade and position information, the second reclaiming value is the sum of the volume value and the second time value, the volume value is determined according to the volume grade, the second time value is obtained in the same way as the first time value, the N second reclaiming values ​​are integrated, the second reclaiming values ​​are arranged from large to small, and the second reclaiming sequence is obtained; the reclaimer performs secondary reclaiming according to the arrangement of the second reclaiming sequence, and materials of different volumes are respectively taken to P different belts, and transported to the final sorting port through the P belts; the P reclaimers arranged at the final sorting port respectively take materials to obtain multiple uniform material piles. The technical problems in the prior art, such as low manual operation efficiency, high safety risks, easy classification errors, difficulty in ensuring the clarity and accuracy of material classification, inability to achieve the goal of uniform mixing, and direct impact on the quality of the final product, are solved. The technical effect of enhancing sorting efficiency, effectively reducing safety risks, accurately classifying materials, and thus obtaining a uniform material pile is achieved.

[0008] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0010] Figure 1 A schematic diagram of a process flow of an automatic stacking and reclaiming method for belt conveyor materials based on image recognition provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of an automatic stacking and reclaiming system for belt conveyor materials based on image recognition provided in an embodiment of the present application.

[0011] Explanation of the accompanying drawings: material characteristic level model construction module 11, first material accumulation information collection module 12, humidity level acquisition module 13, first material picking sequence acquisition module 14, initial material picking module 15, second material accumulation information collection module 16, volume level acquisition module 17, second material picking sequence acquisition module 18, intermediate material picking module 19, final material picking module 20. DETAILED DESCRIPTION

[0012] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0013] Embodiment 1: The automatic stacking and retrieving method of belt conveyor materials based on image recognition provided by the embodiment of the present disclosure is referred to Figure 1 For illustration, the methods include: S1: constructing a material characteristic grade model according to the historical production record data of the material, wherein the material characteristic grade model includes a material moisture grade table and a material volume grade table; Furthermore, a material characteristic level model is constructed, including: Extracting the type, humidity, and volume data from the historical production records of the material to obtain a basic feature set of the material, and preprocessing the basic feature set of the material, wherein the preprocessing includes supplementing missing data values ​​and removing abnormal data values; Traverse the basic feature set of materials, use the material type as the row index, and the humidity data and volume data corresponding to the material as the column, to generate the material humidity statistics table and material volume statistics table; Based on the K-means clustering algorithm, the material humidity statistics table and the material volume statistics table are classified into grades to obtain the material humidity grade table and the material volume grade table; A material characteristic grade model is constructed based on the material moisture grade table and the material volume grade table.

[0014] Specifically, obtain the historical production records of cement plant materials and collect key information related to the materials. Conduct stratified sampling of materials from different batches, different production workshops, and different types to ensure that the samples cover all production situations. Use appropriate measuring tools to measure the humidity and volume of the samples, for example, use a humidity sensor or drying weighing method to measure the sample humidity, and use a measuring tool or drainage method to measure the sample volume. Accurately record the measured humidity and volume data and transfer them to the basic feature set of the material. Sort out the data and check the missing values ​​in the humidity and volume data. If the humidity or volume data of any material is missing, re-test the material data to supplement it. Check for outliers in the data. Determine a reasonable range based on statistical analysis of historical data. For example, by calculating the mean μ and standard deviation σ of historical data, data outside the range of μ±3σ is judged as an outlier. If the data of a certain material deviates significantly from the normal range, further check whether there is an error in the measurement process. If it is a measurement error, re-measure it. If it is due to special reasons, consider correcting it or excluding the outlier during data analysis to ensure the accuracy of subsequent statistical analysis.

[0015] Traverse the basic feature set of materials, use the material type as the row index, and use the corresponding humidity data and volume data as columns. For example, limestone, clay and other material types are each taken as a row, the humidity data obtained by sampling and measuring each material constitutes one column, and the volume data constitutes another column, thereby generating a material humidity statistics table and a material volume statistics table. The statistics table will provide the basic data structure for subsequent grade classification.

[0016] The K-means clustering algorithm is used to obtain the humidity grade table and volume grade table. Since the measured values ​​of different types of materials may have different dimensions and numerical ranges, in order to make the clustering results more accurate, the data in the statistical table can be standardized. For example, standard deviation standardization is used, and the formula is z=(x-μ) / σ, where z is the standard score, representing the dimensionless value after data standardization, reflecting its degree of deviation from the mean, x is the original data, μ is the mean, and σ is the standard deviation. The appropriate number of clusters k is determined by the elbow rule. The elbow rule is to find the inflection point of the error sum square curve under different k values ​​by drawing the error sum square curve under different k values. The k value corresponding to the inflection point is the number of more appropriate clusters. Randomly select k measured values ​​as the initial centroids, calculate the distance from each measured value to the k centroids, and assign them to the cluster with the nearest centroid. For each cluster, recalculate its centroid, that is, the average value of the measured values ​​of all material types in the cluster. Iterate optimization and repeat the steps of assigning clusters and updating centroids until the centroid no longer changes or the preset number of iterations is reached. According to the clustering results, the material types are divided into k levels, and the humidity level table and volume level table are obtained.

[0017] According to the moisture grade table and volume grade table, trace back the samples, select the material samples corresponding to the median data under the grade, take pictures to obtain image information, and extract features from the images to obtain the texture features, color features and volume features of materials of each grade. Link the texture features and color features to the moisture grade table, and link the texture features and volume features to the volume grade table to build a material feature grade model. Specifically, taking limestone as an example, the humidity level table includes three levels, where the first level represents low humidity, the second level represents medium humidity, and the third level represents high humidity. The humidity value of the first level humidity is mainly between 0-0.5%, which is the ideal humidity of the material. The material within this range is not easy to agglomerate during storage, and can also maintain good fluidity during transportation, which is conducive to subsequent grinding and calcination processes. The humidity value of the second level humidity is mainly between 0.5%-1.5%. In this humidity range, although the material may be damp to a certain extent, it can still be handled normally in the production process through some auxiliary means. The humidity value of the third level humidity is greater than 1.5%. At this time, the material may have obvious agglomeration phenomenon, which is easy to accumulate during storage, which is not conducive to subsequent production links. This kind of material must be dried, otherwise it cannot be normally ground and calcined. Similarly, the volume level table of limestone materials also includes three levels, where the first level represents small volume, the second level represents medium volume, and the third level represents large volume. Small volume materials can be quickly dispersed and evenly mixed with other materials. Medium-volume materials can be mixed with other materials more quickly and evenly. They are not as difficult to disperse as large-volume materials, nor are they as easily overblown as small-volume materials. This helps improve the mixing effect during cement production. The feeding uniformity of large-volume materials is difficult to control. For example, when using a crusher or mill, if the material volume is too large, it may cause uneven feeding and affect the crushing or grinding efficiency of the equipment. Large-volume materials may block the feed port or cause material accumulation inside the equipment, making the equipment unable to work properly, so subsequent crushing operations are required.

[0018] S2: collecting first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stocker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; Furthermore, the first material accumulation information on the target belt is collected, including: Acquire image information of the material through a first visual sensor; Preprocessing the acquired image, wherein the preprocessing includes grayscale conversion, filtering, enhancement and histogram equalization; Based on the SSD target detection algorithm, identify the material position in the positioning image and generate M positioning marks; The image is divided into M area images according to the M positioning marks, and the images are summarized, compressed and stored in the first material accumulation information.

[0019] Specifically, the first sensor is placed above the stacker to ensure that the sensor can completely cover the target belt area and avoid blind spots. The stacker piles the same type of materials onto the conveyor belt to obtain multiple evenly laid material piles. The trigger device of the first sensor is set to trigger the sensor at a fixed time according to the belt running speed, and collect images at fixed time intervals.

[0020] Get the image data and use the weighted average method to convert the RGB image into a grayscale image. The formula is: Gray = 0.299R + 0.587G + 0.114B, where Gray represents the grayscale value, and R, G, and B represent the values ​​of the red, green, and blue channels, respectively.

[0021] Perform Gaussian filtering on image data based on Gaussian function. Generate Gaussian kernel according to the required filter size and standard deviation, and perform border processing on the image, adding extra pixel rows and columns at the edge of the image to achieve zero padding. Align each pixel in the image with the Gaussian kernel, perform weighted summation according to the weight of the filter, perform convolution calculation, and generate new pixel values. According to the above convolution operation, start from the upper left corner pixel of the image, process each pixel in the image row and column by row until the lower right corner pixel, complete the image traversal, and generate a smooth image.

[0022] The generated smooth image is enhanced again, including contrast stretching and sharpening. Contrast stretching is mainly achieved by adjusting the grayscale value range of the image. For example, the grayscale value range in the original image is expanded from a narrow range to the entire grayscale value range. Assuming that the grayscale values ​​of the original image are concentrated between 50 and 150, through contrast stretching, the pixels with a grayscale value of 50 are mapped to 0, and the pixels with a grayscale value of 150 are mapped to 255. The grayscale values ​​of the pixels in the middle are redistributed according to a linear or nonlinear mapping relationship, thereby enhancing the contrast of the image. Sharpening processing requires calculating the gradient of the image and superimposing the gradient value on the original image, so as to make the edges in the image more obvious, so as to facilitate the subsequent distinction between the boundaries between materials and the boundaries between materials and backgrounds.

[0023] Get the filtered and enhanced image and perform histogram equalization. Calculate the histogram of the original image, count the number of times each gray value appears in the image, and calculate the cumulative distribution function based on the histogram. The cumulative distribution function represents the proportion of pixels less than or equal to a certain gray value in the image. According to the cumulative distribution function, the gray value of each pixel in the original image is mapped to a new gray value, so that the histogram of the new image is close to uniform distribution, and the dark and bright details in the image are clarified.

[0024] Based on the SSD target detection algorithm, the preprocessed image data is positionally identified to obtain M positioning marks. The specific steps include: generating multi-layer feature maps through multiple convolutional layers and pooling layers of the VGG16 convolutional neural network to obtain features of different scales; setting multiple anchor frames of different sizes and aspect ratios to cover the pixels in each feature map; predicting the target appearance area by obtaining the target category probability and the position offset of the anchor frame relative to the real target frame; performing non-maximum suppression on the prediction results to remove redundant detection frames and obtain the actual location of the material; setting positioning marks according to the actual location of the material, and marking the position of each material pile with numbers from 1 to M, M < 100.

[0025] According to the distance relationship between the identification numbers, a clustering algorithm is used to determine the range of each area, so as to divide the image into M area images, and the area images are arranged in the order of the identification numbers and stored in the first material accumulation information.

[0026] S3: The material reclaimer provided at the first sorting port obtains the first material accumulation information, performs feature recognition on the first material image information, and obtains the humidity levels of M positioning marked materials in combination with the material feature level model; Further, the humidity levels of M positioning marked materials are obtained, including: Obtaining first material accumulation information, and extracting key features from M regional images respectively, wherein the key features mainly include texture features and color features; Input the texture features of the M area images into the material feature grade model to determine the type of material to be transported this time and the humidity grade table corresponding to the type; The color information of the M area images is compared with the humidity level table to obtain the humidity levels of the M positioning marking materials.

[0027] Specifically, the reclaimer set at the first sorting port obtains M regional images in the first material accumulation information. The regional image is converted into a grayscale image by the weighted average method, and the grayscale image is subjected to feature extraction, including texture feature extraction and color feature extraction. Texture feature extraction is obtained according to the grayscale co-occurrence matrix GLCM. The GLCM parameters are set, including the distance d and the direction θ, and a grayscale co-occurrence matrix is ​​constructed based on each pixel in the image. The distance d is set to 1, representing the adjacent pixel points, and the direction θ is set to 0°, 45°, 90° and 135°, representing the adjacent pixel points in the directions of 0°, 45°, 90° and 135° of the selected pixel point. For each pixel in the image, according to the set distance and direction, the grayscale value of the pixel and the adjacent pixel points are set as the row and column index of the matrix respectively, and the co-occurrence matrix is ​​constructed by increasing the count of the position in the matrix. For example, if the grayscale value of a pixel is i and the grayscale value of its adjacent pixel is j, the count in the i-th row and j-th column of the GLCM is increased by 1. The value of each element is divided by the total number of pixel pairs, and the counts in the co-occurrence matrix are converted into probabilities, realizing the conversion of GLCM values ​​to probability distribution, which is convenient for subsequent feature extraction.

[0028] Extract GLCM texture feature parameters, the calculation formulas are: Contrast ratio: ; Where P(i,j) represents the value of element (i,j) in GLCM, and n represents the number of gray levels. Contrast reflects the clarity and local variation of texture in an image, and high contrast indicates drastic texture variation.

[0029] energy: ; Energy represents the uniformity of image texture. The higher the energy value, the more uniform the texture.

[0030] entropy: ; Entropy reflects the randomness of image texture. The higher the entropy value, the more complex and random the texture.

[0031] Relevance: ; where μ i and μ j denote the mean of the row and column respectively, σ i and σ j Denotes the standard deviation of rows and columns respectively. Correlation reflects the linear relationship between pixels of different gray levels in the image.

[0032] The texture feature parameters such as contrast, energy, entropy and correlation calculated above are combined into a texture feature vector V = [Contrast, Energy, Entropy, Correlation] for comparison with the texture features in the material feature grade model.

[0033] Calculate the mean of the image on the R, G, and B channels. The formula is: R channel mean: ; Where I represents the image, (x, y) is the pixel coordinate, M and N are the height and width of the image respectively, I R (x,y) is the R channel value of the pixel at coordinate (x,y). Similarly, the mean of the G and B channels can be calculated.

[0034] Calculate the variance of each channel to reflect the degree of color discreteness. The formula is: R channel variance: ; Among them, μ R is the mean of the R channel, and the other symbols are the same as above. The variance of the G and B channels can be calculated in the same way.

[0035] Divide the R, G, and B channel value ranges into several intervals. For example, if each channel is divided into 16 intervals, there will be a total of 16×16×16=4096 intervals. Count the number of pixels in each interval to form a color histogram. This histogram can reflect the distribution of different colors in the image. Construct the feature vector V RGB =[μ R ,σ R ,μ G ,σ G ,μ B ,σ B ,h R ,h G ,h B ], where h R 、h G 、h B It is a certain statistic of the color histogram of the R, G, and B channels, such as the peak value of the histogram. The color feature vector is used to compare with the color features in the humidity level table.

[0036] Obtain the material feature grade model and retrieve the texture feature data of different types of materials in the model. Compare the texture features based on the Euclidean distance. The Euclidean distance formula is: , where xi is the i-th element in the regional image texture feature vector, yi is the i-th element of the texture feature vector of each material in the material feature grade model, and n is the dimension of the feature vector. In this formula, the value of n is 4, corresponding to the four features of contrast, energy, entropy and correlation mentioned above. For each material in the material feature model, the Euclidean distance between the two texture feature vectors is calculated, and the minimum distance is determined to determine the type of material. According to the determined type, the humidity level table corresponding to the material type is called to obtain the color features corresponding to each humidity level. The regional image color features are compared using the same method of calculating the Euclidean distance to obtain the humidity levels of the M identification points.

[0037] S4: Calculate the first material picking value of the M positioning mark materials according to the material humidity level and position information, the first material picking value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material picker moves to the positioning mark, the order of magnitude of the humidity value is one hundred times the first time value, integrate the M first material picking values, arrange the first material picking values ​​from large to small, and obtain the first material picking sequence; Further, a first material fetching sequence is obtained, including: Set the humidity value H, the humidity value formula is H=humidity level×100; Get M positioning marks, calculate the time required for the reclaimer to move to each positioning mark, obtain M reclaiming times, sort the reclaiming times, and assign values ​​1 to M in sequence to generate a reclaiming time sequence; Set a first time value T1, wherein the formula of the first time value is T1=material taking sequence×1; Calculate the first material fetching value, which is the sum of the humidity value H and the first time value T1, integrate M first material fetching values ​​for sorting, trace back the location identifier corresponding to each value, and obtain the first material fetching sequence.

[0038] Specifically, obtain M positioning marks and the humidity level of the materials at each position. Set a humidity value H, which is determined according to the humidity level of the material. The higher the level, the larger the value. The formula is H=humidity level×100. For example, if the humidity level of the material under the positioning mark is 3, then its humidity value is 300. According to the material position, the initial position of the reclaimer and the moving speed of the reclaimer, calculate the time required for the reclaimer to move to each material position to obtain M reclaiming times. Sort the M reclaiming times in order from large to small, generate a reclaiming time sequence table, and assign values ​​1 to M in order, overwriting the original reclaiming time values. The reclaiming time sequence table describes the positional relationship of each material to the reclaimer from far to near. Calculate the first time value T1, which reflects the reclaiming time. The shorter the time, the larger the value. The formula is T1=reclaiming time sequence×1.

[0039] Calculate the first material picking value and obtain the first material picking sequence. By adding the humidity value of each positioning mark material and the first time value, M first material picking values ​​are obtained, such as 301, 106 and other specific values. Re-sort these M first material picking values ​​in order of scores from large to small to obtain the first material picking sequence. Given that the order of magnitude of the humidity value is a hundred times that of the first time value, the leading indicator used for sorting is the humidity level of the material, and the subsequent indicator is the position relationship of the material, so that materials of different humidity can be divided and each humidity material can be grabbed in order from near to far.

[0040] S5: The reclaimer reclaims materials according to the order of the first reclaiming sequence, and respectively takes materials with different humidity to Q different belts, and transmits them to the second sorting port through the Q belts; Specifically, the reclaimer classifies and reclaims materials according to the first reclaiming order of the materials, and reclaims materials of the same humidity level from near to far to the same belt to obtain evenly spread materials. For example, the first reclaiming value is detected, and materials with a reclaiming value of 300 or more are collected from near to far to the same conveyor belt. When all materials with a reclaiming value of 300 or more are sorted, the reclaimer replaces the belt carrying the materials, and recollects and reclaims materials with a reclaiming value of 200 or more, and so on to complete all reclaiming, and the materials are divided into Q belts for transmission.

[0041] S6: collecting second material accumulation information on the Q belts through Q second visual sensors, where the second visual sensors are located above the Q belts at the first sorting port, the second material accumulation information includes second material position information and second material image information, and the second material position information includes N positioning marks; Specifically, a second sensor is set above the first sorting port reclaimer to collect image information. The setting requirements and triggering device of the second sensor are the same as those of the first sensor. Since the materials have been divided into Q belts for transmission at this time, the number of second sensors is also Q. The subsequent volume classification of materials also occurs on Q belts. Here, only one example is given to illustrate the process.

[0042] The image data collected by the second sensor is obtained, and the image is preprocessed by grayscale, filtering, enhancement, and histogram equalization to obtain clear and smooth image information. The SSD target detection algorithm is applied to the preprocessed image data to identify the position and obtain N positioning marks, and the value of N is set to be less than 100. Based on the clustering algorithm, the image is divided into N regional images, and the regional images are arranged in the order of the identification numbers and stored in the second material accumulation information.

[0043] S7: The material reclaimer provided at the second sorting port obtains the second material accumulation information, calculates the material volume in combination with the volume information in the characteristic grade model, and obtains the volume grades of the N positioning marked materials; Furthermore, the volume levels of N positioning marked materials are obtained, including: Obtaining the second material accumulation information, and extracting key features from the N regional images respectively, wherein the key features mainly include texture features and shape features; Combining the texture features of the N regional images and the material feature grade model, determine the type of material being transported this time and the volume grade table corresponding to the type; The size of the object is estimated by calculating the size of the bounding box, and the size information of the N region images is obtained; The size information of the M area images is compared with the volume grade table to obtain the volume grades of the N positioning marked materials.

[0044] Specifically, N regional images in the second material accumulation information are obtained, and features are extracted from each image, including texture feature extraction and shape feature extraction. The texture feature extraction and comparison are the same as those in step S3, including obtaining a texture feature vector according to the gray level co-occurrence matrix GLCM, calculating the Euclidean distance between the feature vector and the texture feature data in the material feature grade model, determining the minimum distance, and determining the material type. The difference is that this comparison is based on the material type to obtain the volume grade table corresponding to the material type, rather than the humidity grade table.

[0045] Based on the Canny edge detection algorithm, the shape and contour of the object are determined. The gradient amplitude and direction of the image are calculated by using the first-order partial derivative finite difference to obtain a preliminary edge response map. Then non-maximum suppression is performed, and the pixel points with the local maximum gradient amplitude are retained as edge points, so as to obtain more accurate edges. Finally, double threshold detection and regional connectivity are used to distinguish strong edges, weak edges and non-edges, and weak edges are connected with strong edges to obtain complete and accurate edges.

[0046] For the extracted complex contour, a polygonal approximation method is used to simplify it into a polygon. For example, the Douglas-Peucker algorithm is used to merge the points on the contour that are close to each other by setting a distance threshold, so as to obtain a polygon that can approximate the original contour. The possible geometric shape of the object is judged based on the number of sides, side length ratio, angle and other features of the polygon. If the object is judged to be approximately a cuboid, the pixel lengths lp and wp in the length and width directions of the object are obtained, and the actual length l=lp×s and width w=wp×s are calculated based on the known image scale s. For the height h, since the shape of the material conveyed by the belt is roughly circular or block-shaped, its simplified three-dimensional shape is mostly a cube or a cuboid. In order to reduce the amount of calculation, the height of each material block is directly set to the mean of the length and width of the material block to avoid excessive errors in the final volume calculation. The volume V=l×w×h of the cuboid is calculated, and the size information of the N regional images is finally obtained. The volume level table is retrieved, the difference between the image size information and each volume level is calculated, the minimum difference is obtained, and the volume level of the N identification points is obtained.

[0047] S8: Calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information, the second material fetching value is the sum of the volume value and the second time value, the volume value is determined according to the volume level, the second time value is obtained in the same way as the first time value, integrate the N second material fetching values, arrange the second material fetching values ​​from large to small, and obtain the second material fetching sequence; Further, a second material extraction sequence is obtained, including: Set a volume value V, the volume value formula is V=volume level×100; Get N positioning marks, calculate the time required for each mark to fetch materials, sort and assign values ​​to generate the fetching time sequence, and then obtain the second time value T2, the formula is T2=fetching time sequence × 1; Calculate the second material fetching value, integrate N second material fetching values ​​and sort them, trace back the location identifier corresponding to each value, and obtain the second material fetching sequence.

[0048] Specifically, obtain N positioning marks and the volume levels corresponding to the materials. Set a volume value V, which is determined according to the material volume level. The higher the level, the larger the value. The formula is V=volume level×100. For example, if the volume level of the material under the positioning mark is 2, then its volume value is 200. According to the material position, the initial position of the reclaimer and the moving speed of the reclaimer, calculate the time required for the reclaimer to move to each material position to obtain N reclaiming times. Sort the N reclaiming times in order from large to small, generate a reclaiming timing table, and assign values ​​1 to N in order, overwriting the original reclaiming time values. Calculate the second time value T2, and the formula is T2=reclaiming timing×1.

[0049] Calculate the second material picking value, which is the sum of the volume value and the second time value. Sort the second material picking values ​​from large to small to obtain the second material picking sequence. Divide the materials of different volumes, and grab the materials of each volume in order from near to far.

[0050] S9: the reclaimer performs secondary reclaiming according to the arrangement of the second reclaiming sequence, and respectively takes materials of different volumes to P different belts, and transports them to the final sorting port through the P belts; Specifically, the reclaimer classifies and reclaims materials according to the second reclaiming order of the materials, and reclaims materials of the same volume from near to far to the same belt. Since the materials have been classified by humidity at the first sorting port and divided into Q belts for transportation, this reclaiming is a secondary classification of the materials transported by Q belts. Assuming that each belt will be divided into R belts for transportation after volume classification, after this sorting, there will eventually be P belts transporting materials, where P=Q×R.

[0051] S10: P reclaimers arranged at the final sorting port respectively reclaim materials to obtain multiple uniform material piles.

[0052] By arranging P reclaimers at the final sorting port, the materials on the P belts are reclaimed, and finally P piles of materials with consistent moisture and volume are obtained, thus completing the classification of the materials and ensuring the consistency of the properties of the material piles.

[0053] Embodiment 2, based on the same inventive concept as the belt conveyor material automatic stacking and reclaiming method based on image recognition in the previous embodiment, the present application also provides a belt conveyor material automatic stacking and reclaiming system based on image recognition, please refer to the attached Figure 2 , the system comprising: A material characteristic grade model construction module 11, the material characteristic grade model construction module 11 is used to construct a material characteristic grade model according to the material historical production record data, and the material characteristic grade model includes a material moisture grade table and a material volume grade table; A first material accumulation information acquisition module 12, the first material accumulation information acquisition module 12 is used to collect first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stacker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; The humidity level acquisition module 13 is used to obtain the first material accumulation information by the material reclaimer arranged at the first sorting port, perform feature recognition on the first material image information, and obtain the humidity levels of M positioning marked materials in combination with the material feature level model; The first material fetching sequence acquisition module 14 is used to calculate the first material fetching value of the M positioning mark materials according to the material humidity level and position information, the first material fetching value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material fetcher moves to the positioning mark, the order of magnitude of the humidity value is one hundred times of the first time value, and the M first material fetching values ​​are integrated and arranged from large to small to obtain the first material fetching sequence; The first material taking module 15 is used for the material taking machine to take materials according to the first material taking sequence, take materials with different humidity to Q different belts respectively, and transmit them to the second sorting port through the Q belts; The second material accumulation information acquisition module 16 is used to collect the second material accumulation information on the Q belts through Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes the second material position information and the second material image information, and the second material position information includes N positioning marks; The volume level acquisition module 17 is used to obtain the second material accumulation information by the material reclaimer arranged at the second sorting port, calculate the material volume in combination with the volume information in the characteristic level model, and obtain the volume level of N positioning marked materials; The second material fetching sequence acquisition module 18 is used to calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information. The second material fetching value is the sum of the volume value and the second time value. The volume value is determined according to the volume level. The second time value is obtained in the same way as the first time value. The N second material fetching values ​​are integrated and arranged from large to small to obtain the second material fetching sequence; The midway material taking module 19 is used for the material taking machine to perform secondary material taking according to the arrangement of the second material taking sequence, taking materials of different volumes to P different belts respectively, and transporting them to the final sorting port through the P belts; The final material taking module 20 is used for P material taking machines arranged at the final sorting port to take materials respectively, so as to obtain a plurality of uniform material piles.

[0054] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic stacking and reclaiming method for belt conveyor materials based on image recognition, characterized in that: The method comprises: Constructing a material characteristic grade model based on historical material production record data, wherein the material characteristic grade model includes a material moisture grade table and a material volume grade table; Collecting first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stocker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; The material reclaimer provided at the first sorting port obtains the first material accumulation information, performs feature recognition on the first material image information, and obtains the moisture levels of the M positioning marked materials in combination with the material feature level model; Calculate the first material picking value of the M positioning mark materials according to the material humidity level and position information, the first material picking value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material picker moves to the positioning mark, the order of magnitude of the humidity value is one hundred times the first time value, integrate the M first material picking values, arrange the first material picking values ​​from large to small, and obtain the first material picking sequence; The material reclaimer reclaims materials according to the order of the first reclaiming sequence, and respectively takes materials of different humidity to Q different belts, and transmits them to the second sorting port through the Q belts; The second material accumulation information on the Q belts is collected by Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes second material position information and second material image information, and the second material position information includes N positioning marks; The material reclaimer provided at the second sorting port obtains the second material accumulation information, calculates the material volume in combination with the volume information in the characteristic grade model, and obtains the volume grades of the N positioning marked materials; Calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information, the second material fetching value is the sum of the volume value and the second time value, the volume value is determined according to the volume level, the second time value is obtained in the same way as the first time value, integrate the N second material fetching values, arrange the second material fetching values ​​from large to small, and obtain the second material fetching sequence; The reclaimer performs secondary reclaiming according to the arrangement of the second reclaiming sequence, and respectively takes materials of different volumes to P different belts, and transports them to the final sorting port through the P belts; P reclaimers arranged at the final sorting port respectively reclaim materials to obtain multiple uniform material piles.

2. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition according to claim 1, characterized in that: Construct material characteristic level model, including: Extracting the type, humidity, and volume data from the historical production records of the material to obtain a basic feature set of the material, and preprocessing the basic feature set of the material, wherein the preprocessing includes supplementing missing data values ​​and removing abnormal data values; Traverse the basic feature set of materials, use the material type as the row index, and the humidity data and volume data corresponding to the material as the column, to generate the material humidity statistics table and material volume statistics table; Based on the K-means clustering algorithm, the material humidity statistics table and the material volume statistics table are classified into grades to obtain the material humidity grade table and the material volume grade table; A material characteristic grade model is constructed based on the material moisture grade table and the material volume grade table.

3. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition as claimed in claim 1, characterized in that: Collect the first material accumulation information on the target belt, including: Acquire image information of the material through a first visual sensor; Preprocessing the acquired image, wherein the preprocessing includes grayscale conversion, filtering, enhancement and histogram equalization; Based on the SSD target detection algorithm, identify the material position in the positioning image and generate M positioning marks; The image is divided into M area images according to the M positioning marks, and the images are summarized, compressed and stored in the first material accumulation information.

4. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition according to claim 1, characterized in that: Get the humidity levels of M positioning marked materials, including: Obtaining first material accumulation information, and extracting key features from M regional images respectively, wherein the key features mainly include texture features and color features; Input the texture features of the M area images into the material feature grade model to determine the type of material to be transported this time and the humidity grade table corresponding to the type; The color information of the M area images is compared with the humidity level table to obtain the humidity levels of the M positioning marking materials.

5. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition according to claim 1, characterized in that: Obtain the first material taking sequence, including: Set the humidity value H, the humidity value formula is H=humidity level×100; Get M positioning marks, calculate the time required for the reclaimer to move to each positioning mark, obtain M reclaiming times, sort the reclaiming times, and assign values ​​1 to M in sequence to generate a reclaiming time sequence; Set a first time value T1, wherein the formula of the first time value is T1=material taking sequence×1; Calculate the first material fetching value, which is the sum of the humidity value H and the first time value T1, integrate M first material fetching values ​​for sorting, trace back the location identifier corresponding to each value, and obtain the first material fetching sequence.

6. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition as claimed in claim 1, characterized in that: Get the volume level of N positioning marked materials, including: Obtaining the second material accumulation information, and extracting key features from the N regional images respectively, wherein the key features mainly include texture features and shape features; Combining the texture features of the N regional images and the material feature grade model, determine the type of material being transported this time and the volume grade table corresponding to the type; The size of the object is estimated by calculating the size of the bounding box, and the size information of the N region images is obtained; The size information of the M area images is compared with the volume grade table to obtain the volume grades of the N positioning marked materials.

7. The automatic stacking and reclaiming method for belt conveyor materials based on image recognition as claimed in claim 1, characterized in that: The second material taking sequence is obtained, including: Set a volume value V, the volume value formula is V=volume level×100; Get N positioning marks, calculate the time required for each mark to fetch materials, sort and assign values ​​to generate the fetching time sequence, and then obtain the second time value T2, the formula is T2=fetching time sequence × 1; Calculate the second material fetching value, integrate N second material fetching values ​​and sort them, trace back the location identifier corresponding to each value, and obtain the second material fetching sequence.

8. The automatic stacking and reclaiming system for belt conveyor materials based on image recognition is characterized by: The system comprises: A material characteristic grade model construction module, which is used to construct a material characteristic grade model according to the material historical production record data, wherein the material characteristic grade model includes a material moisture grade table and a material volume grade table; A first material accumulation information acquisition module, the first material accumulation information acquisition module is used to collect first material accumulation information on the target belt through a first visual sensor, wherein the first visual sensor is arranged above the stacker, the first material accumulation information includes first material position information and first material image information, and the first material position information includes M positioning marks; A humidity level acquisition module, which is used to obtain the first material accumulation information through a material reclaimer arranged at the first sorting port, perform feature recognition on the first material image information, and obtain the humidity levels of M positioning marked materials in combination with the material feature level model; The first material fetching sequence acquisition module is used to calculate the first material fetching value of M positioning mark materials according to the material humidity level and position information, the first material fetching value is the sum of the humidity value and the first time value, the humidity value is determined according to the humidity level, the first time value is determined according to the time when the material fetcher moves to the positioning mark, the order of magnitude of the humidity value is one hundred times of the first time value, and the M first material fetching values ​​are integrated and arranged from large to small to obtain the first material fetching sequence; The first material picking module is used for the material picking machine to pick up materials according to the first material picking sequence, and to pick up materials with different humidity to Q different belts respectively, and to transmit them to the second sorting port through the Q belts; A second material accumulation information acquisition module, the second material accumulation information acquisition module is used to collect second material accumulation information on Q belts through Q second visual sensors, the second visual sensors are located above the Q belts of the first sorting port, the second material accumulation information includes second material position information and second material image information, and the second material position information includes N positioning marks; A volume level acquisition module, which is used to obtain the second material accumulation information from a material reclaimer arranged at the second sorting port, calculate the material volume in combination with the volume information in the characteristic level model, and obtain the volume levels of N positioning marked materials; The second material fetching sequence acquisition module is used to calculate the second material fetching values ​​of N positioning marked materials according to the material volume level and position information. The second material fetching value is the sum of the volume value and the second time value. The volume value is determined according to the volume level. The second time value is obtained in the same way as the first time value. The N second material fetching values ​​are integrated and arranged from large to small to obtain the second material fetching sequence; The midway material picking module is used for the material picking machine to perform secondary material picking according to the arrangement of the second material picking sequence, and to pick materials of different volumes to P different belts respectively, and to transport them to the final sorting port through the P belts; The final material taking module is used to obtain multiple uniform material piles by P material taking machines arranged at the final sorting port.

Images