Control method and system of intelligent sorting robot

Abstract

The invention belongs to the technical field of robot control, and particularly relates to a control method and system for an intelligent sorting robot, and the method comprises the steps: determining a to-be-recognized region according to a transmission image of to-be-sorted raw coal and an edge in a first gray level image; according to the gray level change of the local position of the edge of the area to be identified in the first gray level image in the transmission image, determining the edge gray level gradient degree of the area to be identified; correcting the probability that the to-be-identified area is gangue according to the edge gray scale gradient degree, and further screening a gangue area; positioning the gangue according to the gangue area, and constructing a three-dimensional model of the gangue; predicting the quality of the gangue according to the three-dimensional model; the pushing force of the sorting robot on the gangue is determined according to the mass; and the sorting robot is controlled to push the gangue to the sorting channel according to the pushing force. According to the invention, accurate identification and positioning of gangue are realized, accurate control of the sorting robot is realized, and the sorting efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control. More specifically, the present invention relates to a control method and system for an intelligent sorting robot. Background Art

[0002] During the coal mining process, the rock fragments (i.e., gangue) on the rock wall generated along with coal mining inevitably mix into the raw coal. The efficient sorting of these associated minerals is a key technical link to improve the utilization rate of coal mine resources and reduce transportation costs.

[0003] There are significant differences in the physical properties between gangue and coal, mainly reflected in parameters such as density, hardness, and surface morphology, which provides a theoretical basis for the sorting operation. However, the sorting process in actual production still faces multiple technical challenges: First, the particularity of the mine operation environment poses a severe test to the detection system. Factors such as high dust concentration and unstable lighting conditions seriously interfere with the performance of traditional optical detection equipment, resulting in a decrease in the accuracy of gangue recognition. The size and weight of individual gangue vary significantly, ranging from several centimeters to dozens of centimeters, further increasing the difficulty of accurately identifying gangue, and thus affecting the reliability of subsequent automated sorting.

[0004] At the same time, most of the existing robot sorting technologies use a unified pushing force to sort the identified gangue. This single sorting strategy has obvious technical limitations: for gangue with a relatively large mass, the fixed pushing force may not be sufficient to make it completely enter the sorting channel, resulting in incomplete sorting; for gangue with a relatively light mass, the same pushing force will cause its kinetic energy to be excessive, leading to safety problems such as gangue rushing out of the sorting channel and colliding with equipment, and may even cause equipment damage or secondary pollution and other problems. Summary of the Invention

[0005] To solve the above technical problems of insufficient accuracy in gangue recognition and the fact that the fixed pushing force of the robot for gangue may lead to incomplete gangue sorting or gangue rushing out of the sorting channel, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a control method for an intelligent sorting robot, including: Determine the corresponding regions to be recognized in the transmission image and the first grayscale image based on the edges in the raw coal transmission image to be sorted and the first grayscale image; determine the probability that the region to be recognized is gangue according to the average gray value of the corresponding pixel points in the transmission image of the region to be recognized; determine the degree of edge gray-scale gradient of the region to be recognized according to the gray-scale change of the edge included in the region to be recognized in the first grayscale image at the local position in the transmission image; correct the probability that the region to be recognized is gangue according to the degree of edge gray-scale gradient, and screen the gangue regions according to the size of the corrected probability; locate the gangue based on the gangue regions and construct a three-dimensional model of the gangue; predict the quality of the gangue according to the three-dimensional model of the gangue; determine the pushing force of the sorting robot for the gangue according to the quality of the gangue; and control the sorting robot to push the gangue into the sorting channel according to the pushing force.

[0007] The present invention determines the regions to be recognized through the edges in the transmission image and the first grayscale image, and can completely segment individual coal or gangue, providing a basis for subsequent gangue recognition; the present invention adopts a dual verification mechanism of the gray-scale characteristics and edge gradient characteristics of the regions to be recognized in the transmission image, making the corrected probability of gangue more accurate and improving the accuracy of gangue recognition; the present invention determines the pushing force of the sorting robot for the gangue according to the quality of the gangue, and can dynamically adjust the pushing intensity, which not only avoids problems such as energy waste and even causing the gangue to rush out of the sorting channel and collide with equipment due to using a large pushing force for relatively light gangue, but also ensures the accurate delivery of the sorting channel, and can also make relatively heavy gangue completely enter the sorting channel to avoid incomplete sorting, achieving precise mechanical control of the sorting robot based on the pushing force.

[0008] Preferably, the determining the corresponding regions to be recognized in the transmission image and the first grayscale image includes: obtaining the overlapping edge pixel points in the transmission image and the first grayscale image, taking the overlapping edge pixel points as the boundary, respectively dividing corresponding closed regions in the transmission image and the first grayscale image, and taking each closed region as an independent region to be recognized.

[0009] The present invention determines independent regions to be recognized according to the overlapping edge pixel points, providing basic data for subsequent feature extraction and recognition. This method ensures the consistency of the regions to be recognized in terms of spatial position, which is beneficial to improving the accuracy and reliability of image analysis.

[0010] Preferably, the determining the probability that the region to be recognized is gangue according to the average gray value of the corresponding pixel points in the transmission image of the region to be recognized includes: taking the average value of the gray values of the corresponding pixel points in the transmission image of the region to be recognized as the overall gray value of the region to be recognized; obtaining the ratio of the overall gray value of the region to be recognized to the maximum value of the overall gray values of all regions to be recognized, and performing a negative correlation mapping on the ratio to obtain the probability that the region to be recognized is gangue.

[0011] In view of the physical characteristic that the density of gangue is relatively large, the present invention calculates the ratio of the gray-scale mean value of the area to be recognized to the global maximum gray scale, and performs a negative correlation mapping, ensuring that the gangue area, which usually exhibits low gray-scale characteristics, can obtain a higher probability value.

[0012] Preferably, the method for obtaining the gray-scale gradual change degree of the edge of the area to be recognized is as follows: project the edge within the target area of the first gray-scale image onto the transmission image. For any projection pixel point on the projected edge within the target area of the transmission image, obtain the normal line of the projected edge at this projection pixel point, and respectively use a plurality of pixel points adjacent to this projection pixel point in the positive direction and negative direction of the normal line within the target area as the positive local pixel points and negative local pixel points of this projection pixel point; respectively use the absolute value of the mean value of the differences of all adjacent positive local pixel points and the absolute value of the mean value of the differences of all adjacent negative local pixel points as the gray-scale gradual change amounts of the corresponding projection pixel point in the positive direction and negative direction of its normal line; determine the gray-scale gradual change degree of the edge of the target area according to the gray-scale gradual change amounts in the positive direction and negative direction.

[0013] The present invention takes into account that the structural change of gangue with a relatively large density can bring more significant gray-scale changes in the transmission image, while the structural change of coal with a relatively small density has insignificant gray-scale changes in the transmission image, and the projected edge in the first gray-scale image is the position of the structural change of coal or gangue. Therefore, the present invention combines the gray-scale change situation around the projected edge within the target area of the transmission image to obtain the gray-scale gradual change degree of the edge of the target area, providing a mathematical basis for subsequent differentiation between coal and gangue, and enabling better differentiation between thicker coal and thinner gangue.

[0014] Preferably, the gray-scale gradual change degree satisfies the expression: ; where represents the gray-scale gradual change degree of the edge of the target area; represents the th projected edge in the target area and the th gray-scale gradual change amount of the projection pixel point in the positive direction of its normal line, represents the th projected edge in the target area and the th gray-scale gradual change amount of the projection pixel point in the negative direction of its normal line; represents the number of projection pixel points on the th projected edge in the target area; represents the number of projected edges in the target area; is the first hyperparameter; represents the hyperbolic tangent function.

[0015] Preferably, the corrected probability satisfies the expression: ; wherein, represents the correction probability that the target area is gangue; represents the probability that the target area is gangue; represents the degree of edge gray-scale gradient of the target area; is the second hyperparameter; is the third hyperparameter.

[0016] Preferably, the method for positioning gangue according to the gangue area and constructing a three-dimensional model of gangue includes: determining the depth information of the gangue surface in the gangue area in the first grayscale image through a stereo vision algorithm based on the first grayscale image and the second grayscale image of the gangue, and constructing a three-dimensional model of the gangue by using point cloud reconstruction and surface fitting techniques; the second grayscale image and the first grayscale image are images of the same area from different perspectives; determining the real-time coordinate position of the gangue according to the running speed of the coal conveyor.

[0017] Preferably, the method for predicting the quality of gangue according to the three-dimensional model of gangue includes: predicting the quality of gangue by using a neural network.

[0018] Preferably, the pushing force satisfies the expression: ; wherein, represents the pushing force; represents the quality of the gangue; represents the acceleration due to gravity; represents the dynamic friction coefficient of the gangue on the coal conveyor; represents the acceleration required to push the gangue to the sorting channel; represents the distance from the gangue to the sorting channel.

[0019] The present invention dynamically adjusts the pushing intensity according to the quality of the gangue and the distance from the gangue to the sorting channel, which not only avoids energy waste but also ensures accurate delivery to the sorting channel, and can achieve precise mechanical control during the sorting of the sorting robot.

[0020] In a second aspect, the present invention provides a control system for an intelligent sorting robot, including a processor and a memory, where the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above control method for an intelligent sorting robot is implemented.

[0021] By adopting the above technical solution, the above control method for an intelligent sorting robot is generated into a computer program and stored in the memory to be loaded and executed by the processor, so as to manufacture a terminal device according to the memory and the processor, which is convenient to use.

[0022] The beneficial effects of the present invention are as follows: The present invention can completely separate individual coal or gangue, providing a basis for subsequent gangue identification; the present invention adopts a dual verification mechanism of the gray - scale feature and the edge gradient characteristic of the area to be identified in the transmission image, improving the accuracy of gangue identification; the present invention can dynamically adjust the pushing intensity, which not only avoids problems such as energy waste caused by using a large pushing force for relatively light gangue, or even causing the gangue to rush out of the sorting channel and collide with equipment, but also ensures the accurate delivery of the sorting channel. For relatively heavy gangue, it can also be completely fed into the sorting channel, realizing the precise mechanical control of the sorting robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart schematically showing a control method of an intelligent sorting robot in the present invention; Figure 2 is a flowchart schematically showing step S2 of the control method of an intelligent sorting robot in the present invention; Figure 3 is a schematic diagram showing the normal line. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0025] Next, the specific embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0026] An embodiment of the present invention discloses a control method of an intelligent sorting robot. Referring to Figure 1 , it includes steps S1 - step S6: S1. Collect the transmission image and the gray - scale image of the raw coal to be sorted on the coal mine conveyor.

[0027] Specifically, an imaging system is installed on the raw coal conveyor line. The imaging system is composed of a high - resolution industrial camera and an X - ray generating device, and is in linkage control with the coal mine conveyor. To ensure the accuracy of image acquisition, a plurality of positioning points are set in the imaging area.

[0028] When the raw coal on the conveyor passes through the first positioning point in the imaging area, the imaging system synchronously performs the following operations: First, use the industrial camera to collect the gray - scale image of the raw coal; at the same time, use the X - ray generating device to collect the transmission image of the raw coal. The collected gray - scale image and transmission image contain the information of the first positioning point set in advance. The collected gray - scale image is called the first gray - scale image.

[0029] When the raw coal on the conveyor passes through the second positioning point in the imaging area, an industrial camera is used to collect the grayscale image of the raw coal again, and the grayscale image collected this time is called the second grayscale image. The first grayscale image and the second grayscale image are images of the same area from different perspectives.

[0030] Based on the positioning points in the first grayscale image and the transmission image, an image registration algorithm is used to perform spatial alignment processing on the first grayscale image and the transmission image to ensure the consistency of the spatial positions of the first grayscale image and the transmission image.

[0031] According to the registration results of the first grayscale image and the transmission image, the first grayscale image and the transmission image are regionally cropped so that the first grayscale image and the transmission image correspond to the same observation area. Through image processing techniques such as interpolation or downsampling, the cropped transmission image is adjusted to the same size as the first grayscale image.

[0032] S2. Identify the gangue areas in the transmission image and the first grayscale image.

[0033] The flowchart of step S2 is referred to Figure 2 , including steps S201 to S205, specifically: S201. Determine the corresponding areas to be identified in the transmission image and the first grayscale image according to the edges in the transmission image and the first grayscale image.

[0034] Specifically, the Canny edge detection algorithm is used to perform edge detection on the transmission image and the first grayscale image respectively to obtain the edges in the transmission image and the edges in the first grayscale image. In other embodiments, the implementer can select an edge detection algorithm according to the actual implementation situation, such as the Sobel operator, the Prewitt operator, etc.

[0035] It should be noted that the edges in the first grayscale image include the edges of coal or gangue, and also include the texture edges of the structural changes on the surface of coal or gangue. The edges in the transmission image include the edges of coal or gangue, and also include the internal structural information of coal or gangue. Therefore, the present invention obtains the areas to be identified according to the corresponding edges in the first grayscale image and the transmission image.

[0036] Furthermore, the overlapping edge pixel points in the transmission image and the first grayscale image are obtained. Taking the overlapping edge pixel points as the boundary, corresponding closed areas are respectively divided in the transmission image and the first grayscale image, and each closed area is used as an independent area to be identified. It should be noted that the areas to be identified in the transmission image and the first grayscale image are in one-to-one correspondence, and have the same position and the same shape.

[0037] S202. Take any area to be recognized as the target area, and determine the probability that the area to be recognized is gangue according to the average gray value of the corresponding pixel points of the target area in the transmission image.

[0038] It should be noted that coal and gangue have different densities and compositions, and also have different absorption characteristics for rays. Coal is mainly composed of carbon, with a lower density and weaker absorption of rays; gangue contains various mineral components, with a higher density and stronger absorption of rays. When rays penetrate an object, the part with stronger absorption will block more rays, resulting in a decrease in the intensity of the transmitted rays, and the ray signal received by the detector is weaker. Then, in the generated transmission image, the area with stronger absorption will be displayed as a darker gray scale (low gray value). Due to its high density and strong absorption, gangue is displayed as a darker area in the transmission image, while coal has a low density and weak absorption, and is displayed as a brighter area in the transmission image. Therefore, in the present invention, the probability that the target area is gangue is determined according to the average gray value of the corresponding pixel points of the target area in the transmission image.

[0039] Specifically, the probability that the target area is gangue satisfies the expression: ; In the formula, represents the probability that the target area is gangue; represents the average value of the gray values of all the pixel points corresponding to the target area in the transmission image; represents the maximum value among the average values of the gray values of all the pixel points corresponding to each area to be recognized in the transmission image; is an exponential function with the natural constant as the base, used for performing a negative correlation mapping on .

[0040] In the transmission image, the average gray value of the pixel points corresponding to the target area can effectively represent the overall gray feature of the target area. According to the principle of ray transmission, when the overall gray of the target area is larger, it indicates that the intensity of the transmitted rays is higher, and the target area is more likely to be a coal area, and the probability that the target area is gangue is smaller; on the contrary, when the overall gray of the target area is smaller, it indicates that the intensity of the transmitted rays is lower, and the target area is more likely to be a gangue area, and the probability that the target area is gangue is larger. Considering that coal is the main component in raw coal and the proportion of gangue is relatively small, there must be coal areas in the transmission image. Then, the maximum value among the average values of the gray values of all the pixel points corresponding to each area to be recognized in the transmission image can be used as the typical gray feature of the coal area. Therefore, in the present invention, by obtaining the ratio of the overall gray of the target area to the maximum value, the overall gray of the target area is normalized, avoiding the problem that the result of the negative correlation mapping is always too small due to the large overall gray of the target area, thus affecting the accuracy of gangue recognition.

[0041] S203. Determine the edge gray-scale gradient degree of the target area according to the gray-scale change of the edge included in the first gray-scale image in the transmission image at the local position.

[0042] It should be noted that the brightness of the target area in the transmission image is not only related to the material of the object corresponding to the target area, but also related to the thickness of the object. When the thickness of the object is larger, more rays will be blocked, resulting in a decrease in the intensity of the transmitted rays, so that the ray signal received by the detector is weak, and the corresponding area in the transmission image is displayed as a darker gray scale. When the thickness of the object is smaller, the intensity of the transmitted rays is larger, and the corresponding area in the transmission image is displayed as a brighter gray scale. Therefore, the overall gray scale of thicker coal and thinner gangue in the transmission image may be close, which affects the recognition accuracy of gangue. When the structure of gangue or coal changes, in the transmission image, the local area where the structure changes will have a gray-scale gradient. However, due to the larger density of gangue and the smaller density of coal, the corresponding gray-scale changes of the same structural changes in gangue and coal are different in the transmission image. The structural changes in gangue correspond to more obvious gray-scale gradients in the transmission image, while the gray-scale gradients corresponding to the structural changes in coal are not obvious in the transmission image. And the edges inside each area to be recognized in the first gray-scale image represent the positions of the structural changes of the coal or gangue corresponding to the area to be recognized. Therefore, the present invention determines the edge gray-scale gradient degree of the area to be recognized according to the gray-scale change of the edge included in the first gray-scale image in the transmission image at the local position.

[0043] Specifically, project the edge within the target area of the first gray-scale image onto the transmission image as the projected edge within the target area of the transmission image.

[0044] For any projection pixel point on the projected edge, obtain the normal line of the projected edge at this projection pixel point, and use the pixel points adjacent to this projection pixel point in the positive direction of this normal line within the target area as the positive local pixel points of this projection pixel point. Use the pixel points adjacent to this projection pixel point in the negative direction of this normal line within the target area as the negative local pixel points of this projection pixel point. Among them, is a preset first quantity, and the implementer can set it according to the actual implementation situation. For example, . When the number of pixel points adjacent to this projection pixel point in the positive direction of the normal line of the projection pixel point in the target area of the transmission image is less than , count with the actual quantity. Figure 3 is a schematic diagram of the normal line. Figure 3 In it, the curve is the projected edge, the dotted line is the tangent line at the projection pixel point A, the straight line perpendicular to the dotted line at the projection pixel point A is the normal line at the projection pixel point A, the arrow B is the positive direction of the normal line, and the arrow C is the negative direction of the normal line.

[0045] Determine the gray-scale gradient of each projection pixel point in the positive direction of its normal line and the gray-scale gradient of each projection pixel point in the negative direction of its normal line according to the gray-scale changes of the positive local pixel points and the gray-scale changes of the negative local pixel points on each projection edge in the target area of the projection image: ; ; wherein, represents the gray-scale gradient of the th projection edge in the target area of the projection image at the th projection pixel point in the positive direction of its normal line, represents the gray-scale gradient of the th projection edge in the target area of the projection image at the th projection pixel point in the negative direction of its normal line; represents the number of positive local pixel points of the th projection edge in the target area of the projection image at the th projection pixel point; represents the number of negative local pixel points of the th projection edge in the target area of the projection image at the th projection pixel point; represents the gray-scale value of the th projection edge in the target area of the projection image at the th projection pixel point at the th positive local pixel point, represents the gray-scale value of the th projection edge in the target area of the projection image at the th projection pixel point at the th positive local pixel point; represents the gray-scale value of the th projection edge in the target area of the projection image at the th projection pixel point at the th negative local pixel point, represents the gray-scale value of the th projection edge in the target area of the projection image at the th projection pixel point at the th negative local pixel point; represents the absolute value symbol.

[0046] It should be noted that since the thickness changes on both sides of the edge may not be consistent, the present invention calculates the gray-scale gradient separately for the positive and negative directions of the normal. At some projection pixel points, there are no pixel points in the target area in the positive direction of the normal. In this case, the gray-scale gradient of this projection pixel point in the positive direction of its normal is not calculated, and the gray-scale gradient of this projection pixel point in the positive direction of its normal is directly marked as 0; at some projection pixel points, there are no pixel points in the target area in the negative direction of the normal. In this case, the gray-scale gradient of this projection pixel point in the negative direction of its normal is not calculated, and the gray-scale gradient of this projection pixel point in the negative direction of its normal is directly marked as 0.

[0047] According to the gray-scale gradients in the positive direction and the negative direction of the normal of each projection pixel point on each projection edge in the target area of the projection image, determine the edge gray-scale gradient degree of the target area: ; Wherein, represents the edge gray-scale gradient degree of the target area; represents the gray-scale gradient in the positive direction of the normal of the th projection pixel point on the th projection edge in the target area, represents the gray-scale gradient in the negative direction of the normal of the th projection pixel point on the th projection edge in the target area; represents the number of projection pixel points on the th projection edge in the target area; represents the number of projection edges in the target area; is the first hyperparameter, used to avoid being too large, resulting in always being close to 1, the empirical value of is , and the implementer can set the value of according to the actual implementation situation; represents the hyperbolic tangent function, used to normalize

[0048] It should be noted that when there is no edge in the target area of the first gray-scale image, there is no projection edge in the target area of the transmission image. In this case, it is stipulated that the edge gray-scale gradient degree of the target area is 0.

[0049] S204. According to the edge gray-scale gradient degree, correct the probability that the target area is gangue to obtain the corrected probability that the target area is gangue.

[0050] Specifically, the corrected probability satisfies the expression: ; Among them, represents the correction probability that the target area is gangue; represents the probability that the target area is gangue; represents the degree of edge gray - level gradient of the target area; is the second hyper - parameter, used to judge the size of the edge gray - level gradient , with an empirical value of 0.75, and implementers can set it according to the actual implementation situation. is the third hyper - parameter, used to prevent the denominator from being 0 when , with an empirical value of 0.0001, and implementers can set it according to the actual implementation situation.

[0051] When is less than or equal to , the edge gray - level gradient is small, indicating that the structural change of the object corresponding to the target area has little influence on the intensity of the transmitted ray, and the target area is more likely to be coal. Therefore, take as the exponent, and use the gamma transformation method to reduce the probability that the target area is gangue. When the edge gray - level gradient is smaller, is larger, and the degree of reduction of the probability that the target area is gangue is greater; when is greater than , the edge gray - level gradient is large, indicating that the structural change of the object corresponding to the target area has a great influence on the intensity of the transmitted ray, and the target area is more likely to be gangue. Therefore, take as the exponent, and use the gamma transformation method to increase the probability that the target area is gangue. When the edge gray - level gradient is larger, is smaller, and the degree of increase of the probability that the target area is gangue is greater.

[0052] It should be noted that the present invention combines the overall gray - level size of the target area and the edge gray - level gradient, taking into account both the influence of the density of coal and gangue on the intensity of the transmitted ray and the influence of the structural changes of coal and gangue on the intensity of the transmitted ray, making the correction probability that the target area is gangue more accurate, and can avoid the problem of inaccurate identification of gangue caused by the overall gray - level of relatively thick coal and relatively thin gangue being close in the transmission image.

[0053] S205. Screen the gangue area according to the size of the correction probability.

[0054] Specifically, in response to the correction probability that the target area is gangue If it is greater than a preset probability threshold, the target area is the gangue area; otherwise, the target area is the coal area.

[0055] The probability threshold can be set by the implementer according to the actual implementation situation, for example, 0.5.

[0056] Thus, the gangue area is obtained.

[0057] S3. Locate the gangue on the coal conveyor according to the gangue area and construct a three-dimensional model of the gangue.

[0058] According to the first grayscale image and the second grayscale image, the surface depth information of the gangue in the gangue area in the first grayscale image is determined by the stereo vision algorithm, and the point cloud reconstruction and surface fitting techniques are used to construct a three-dimensional model of the gangue.

[0059] According to the running speed of the coal conveyor, the real-time coordinate position of the gangue is determined.

[0060] S4. Predict the quality of the gangue according to the three-dimensional model of the gangue.

[0061] Specifically, the present invention predicts the quality of the gangue through a neural network. The input of the neural network is the three-dimensional model of the gangue, and the output is the quality of the gangue; the data set is a data set composed of three-dimensional models of gangues with different shapes, and the label is the quality of the gangue; the loss function of the neural network is the mean square error loss.

[0062] Input the three-dimensional model of the gangue into the neural network, and the weight of the gangue is output.

[0063] S5. Determine the pushing force of the sorting robot for the gangue according to the quality of the gangue.

[0064] Specifically, the pushing force satisfies the expression: ; Where represents the pushing force; represents the quality of the gangue; represents the acceleration due to gravity; represents the dynamic friction coefficient of the gangue on the coal conveyor; represents the acceleration required to push the gangue into the sorting channel; represents the distance from the gangue to the sorting channel. Among them, the dynamic friction coefficient is set by the implementer according to experience and is related to the surface roughness of the gangue and the material of the conveyor belt. Experience shows that when is set to 0.6, most working conditions can be covered.

[0065] Furthermore, the acceleration required to push the gangue into the sorting channel satisfies the expression: ; wherein, represents the distance from the gangue to the sorting channel, is the target sorting time for a single piece of gangue, which is set by real-time personnel according to the actual real-time situation, such as 1 second.

[0066] S6. Control the sorting robot to push the gangue towards the sorting channel according to the pushing force.

[0067] Specifically, according to the real-time coordinate position of the gangue, the motion trajectory of the end effector of the sorting robot is accurately adjusted through a closed-loop control system, and with the pushing force obtained in step S5, the gangue is accurately guided to the sorting channel on the side of the coal conveyor.

[0068] Through the above method, efficient and accurate sorting of gangue in coal mines can be achieved, which can significantly improve the sorting efficiency, reduce manual intervention, and lower the operation cost.

[0069] The embodiment of the present invention also discloses a control system for an intelligent sorting robot, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a control method for an intelligent sorting robot according to the present invention is implemented.

[0070] The above system also includes other components well-known to those skilled in the art such as a communication bus and a communication interface, and their settings and functions are known in the art, so they will not be elaborated here.

Claims

1. A control method for an intelligent sorting robot, characterized in that, Including: Determine the corresponding regions to be recognized in the transmission image and the first grayscale image according to the transmission image of the raw coal to be sorted and the edges in the first grayscale image; Determine the probability that the region to be recognized is gangue according to the average gray value of the corresponding pixel points of the region to be recognized in the transmission image; determine the degree of edge gray-scale gradual change of the region to be recognized according to the gray-scale change of the edge included in the region to be recognized in the first grayscale image at its local position in the transmission image; Correct the probability that the region to be recognized is gangue according to the degree of edge gray-scale gradual change, and screen the gangue regions according to the size of the corrected probability; Locate the gangue according to the gangue region, construct a three-dimensional model of the gangue; predict the quality of the gangue according to the three-dimensional model of the gangue; determine the pushing force of the sorting robot for the gangue according to the quality of the gangue; Control the sorting robot to push the gangue into the sorting channel according to the pushing force.

2. The control method of an intelligent sorting robot according to claim 1, characterized in that, The determination of the corresponding regions to be recognized in the transmission image and the first grayscale image includes: Obtain the overlapping edge pixel points in the transmission image and the first grayscale image. Taking the overlapping edge pixel points as the boundary, divide the corresponding closed regions in the transmission image and the first grayscale image respectively, and take each closed region as an independent region to be recognized.

3. The control method of an intelligent sorting robot according to claim 1, characterized in that The determination of the probability that the region to be recognized is gangue according to the average gray value of the corresponding pixel points of the region to be recognized in the transmission image includes: Take the average value of the gray values of the corresponding pixel points of the region to be recognized in the transmission image as the overall gray value of the region to be recognized; obtain the ratio of the overall gray value of the region to be recognized to the maximum value of the overall gray values of all regions to be recognized, and perform a negative correlation mapping on the ratio to obtain the probability that the region to be recognized is gangue.

4. The control method of an intelligent sorting robot according to claim 1, wherein The method for obtaining the degree of edge gray-scale gradual change of the region to be recognized is: Project the edge in the target region of the first grayscale image onto the transmission image. For any projection pixel point on the projection edge in the target region of the transmission image, obtain the normal line of the projection edge at this projection pixel point, and take several pixel points adjacent to this projection pixel point in the positive direction and negative direction of this normal line in the target region as the positive local pixel points and negative local pixel points of this projection pixel point respectively; take the absolute value of the average value of the differences of all adjacent positive local pixel points and the absolute value of the average value of the differences of all adjacent negative local pixel points as the gray-scale gradual change amounts of the corresponding projection pixel point in the positive direction and negative direction of its normal line respectively; determine the degree of edge gray-scale gradual change of the target region according to the gray-scale gradual change amounts in the positive direction and negative direction.

5. The control method of an intelligent sorting robot according to claim 4, characterized in that, The degree of gray-scale gradual change satisfies the expression: ； Among them, represents the edge gray-scale gradient degree of the target area; represents the gray-scale gradient variable in the positive direction of the normal of the th projection edge in the target area at the th projection pixel point; represents the gray-scale gradient variable in the negative direction of the normal of the th projection edge in the target area at the th projection pixel point; represents the number of projection pixel points on the th projection edge in the target area; represents the number of projection edges in the target area; is the first hyperparameter; represents the hyperbolic tangent function.

6. The control method of an intelligent sorting robot according to claim 1, wherein The corrected probability satisfies the expression: ； Among them, represents the correction probability that the target area is gangue; represents the probability that the target area is gangue; represents the degree of edge gray-scale gradient of the target area; is the second hyperparameter; is the third hyperparameter.

7. The control method of an intelligent sorting robot according to claim 1, characterized in that, The positioning of the gangue according to the gangue region and the construction of the three-dimensional model of the gangue include: Determine the surface depth information of the gangue in the gangue region in the first grayscale image through the stereo vision algorithm according to the first grayscale image and the second grayscale image of the gangue, and construct a three-dimensional model of the gangue by using the point cloud reconstruction and surface fitting technology; the second grayscale image and the first grayscale image are images of the same region from different perspectives; determine the real-time coordinate position of the gangue according to the running speed of the coal conveyor.

8. The control method of an intelligent sorting robot according to claim 1, wherein, The prediction of the quality of the gangue according to the three-dimensional model of the gangue includes: Predict the quality of gangue using a neural network.

9. The control method of an intelligent sorting robot according to claim 1, characterized in that The pushing force satisfies the expression: ； Among them, represents the pushing force; represents the mass of the gangue; represents the acceleration due to gravity; represents the dynamic friction coefficient of the gangue on the coal conveyor; represents the acceleration required to push the gangue to the sorting channel; represents the distance from the gangue to the sorting channel.

10. A control system for an intelligent sorting robot, characterized in that, Including: A processor and a memory, the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a control method of an intelligent sorting robot according to any one of claims 1-9 is implemented.

Images