A control method and system for an intelligent sorting robot

Abstract

The present invention belongs to the field of robot control technology, and specifically relates to a control method and system for an intelligent sorting robot. The method includes: determining a to-be-identified area based on a transmission image of the raw coal to be sorted and edges in a first grayscale image; determining the degree of grayscale gradient of the edge of the to-be-identified area based on the grayscale change in the transmission image of the local position of the edge contained in the first grayscale image of the to-be-identified area; correcting the probability that the to-be-identified area is gangue based on the degree of grayscale gradient, thereby screening the gangue area; locating the gangue based on the gangue area and constructing a three-dimensional model of the gangue; predicting the quality of the gangue based on the three-dimensional model; determining the pushing force of the sorting robot on the gangue based on the quality; and controlling the sorting robot to push the gangue into a sorting channel based on the pushing force. The present invention achieves accurate identification and positioning of gangue, accurate control of the sorting robot, and improved sorting efficiency.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology. More specifically, the present invention relates to a control method and system for an intelligent sorting robot. Background Art

[0002] During the coal mining process, rock wall debris (i.e. gangue) produced by coal mining is inevitably mixed into the raw coal. The efficient sorting of these associated minerals is a key technical link in improving the utilization rate of coal resources and reducing transportation costs.

[0003] Gangue and coal exhibit significant physical differences, primarily in parameters such as density, hardness, and surface morphology, which provide a theoretical basis for sorting operations. However, the actual sorting process still faces multiple technical challenges: First, the unique nature of the mine operating environment presents a severe test for the detection system. Factors such as high dust concentrations and unstable lighting conditions severely interfere with the performance of traditional optical detection equipment, resulting in a decrease in the accuracy of gangue identification. Individual gangue pieces vary significantly in size and weight, ranging from a few centimeters to tens of centimeters, further increasing the difficulty of accurate gangue identification and, in turn, affecting the reliability of subsequent automated sorting.

[0004] At the same time, existing robot sorting technologies mostly use a unified pushing force to sort the identified gangue. This single sorting strategy has obvious technical limitations: for gangue with heavier mass, a fixed pushing force may not be enough to make it fully enter the sorting channel, resulting in incomplete sorting; and for gangue with lighter mass, the same pushing force will cause its excess kinetic energy, thereby causing safety problems such as gangue rushing out of the sorting channel and colliding with equipment, and may even cause equipment damage or secondary pollution. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems of insufficient accuracy in gangue identification and the possibility that the robot's fixed pushing force on the gangue may lead to incomplete gangue sorting or gangue rushing out of the sorting channel, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a control method for an intelligent sorting robot, comprising:

[0007] The corresponding area to be identified in the transmission image and the first grayscale image is determined based on the transmission image of the raw coal to be sorted and the edges in the first grayscale image; the probability that the area to be identified is gangue is determined based on the grayscale mean value of the corresponding pixel points of the area to be identified in the transmission image; the degree of edge grayscale gradient of the area to be identified is determined based on the grayscale change of the local position of the edge contained in the first grayscale image of the area to be identified in the transmission image; the probability that the area to be identified is gangue is corrected according to the degree of edge grayscale gradient, and the gangue area is screened according to the size of the corrected probability; the gangue is located according to the gangue area, and a three-dimensional model of the gangue is constructed; the quality of the gangue is predicted based on the three-dimensional model of the gangue; the pushing force of the sorting robot on the gangue is determined based on the quality of the gangue; and the sorting robot is controlled to push the gangue to the sorting channel according to the pushing force.

[0008] The present invention determines the area to be identified by the edges in the transmission image and the first grayscale image, and can completely segment separate coal or gangue, providing a basis for subsequent identification of gangue; the present invention adopts a dual verification mechanism of the grayscale characteristics of the area to be identified in the transmission image and the edge gradient characteristics, so that the correction probability of gangue is more accurate, and the accuracy of gangue identification is improved; the present invention determines the pushing force of the sorting robot on the gangue by the mass of the gangue, and can dynamically adjust the pushing strength, which not only avoids the use of a larger pushing force for lighter gangue, resulting in energy waste and even causing the gangue to rush out of the sorting channel, collide with equipment, etc., but also ensures the accurate delivery of the sorting channel, and can make the gangue with larger mass enter the sorting channel completely, avoiding incomplete sorting, and realizing precise mechanical control of the sorting robot based on the pushing force.

[0009] Preferably, determining the corresponding areas to be identified in the transmission image and the first grayscale image includes: obtaining the overlapping edge pixel points in the transmission image and the first grayscale image, and using the overlapping edge pixel points as boundaries to divide corresponding closed areas in the transmission image and the first grayscale image respectively, and treating each closed area as an independent area to be identified.

[0010] The present invention determines independent areas to be identified based on overlapping edge pixels, providing basic data for subsequent feature extraction and identification. This method ensures the consistency of the spatial position of the areas to be identified, which is conducive to improving the accuracy and reliability of image analysis.

[0011] Preferably, the method of determining the probability that the area to be identified is gangue based on the grayscale mean of the corresponding pixel points of the area to be identified in the transmission image includes: taking the mean of the grayscale values ​​of the corresponding pixel points of the area to be identified in the transmission image as the overall grayscale of the area to be identified; obtaining the ratio of the overall grayscale of the area to be identified to the maximum value of the overall grayscale of all areas to be identified, performing negative correlation mapping on the ratio, and obtaining the probability that the area to be identified is gangue.

[0012] The present invention takes into account the physical characteristic of high density of gangue, calculates the ratio of the grayscale mean of the area to be identified to the global maximum grayscale, and performs negative correlation mapping, thereby ensuring that the gangue area that usually presents low grayscale characteristics can obtain a higher probability value.

[0013] Preferably, the method for obtaining the edge grayscale gradient degree of the area to be identified is: projecting the edge within the target area of ​​the first grayscale image into the transmission image, and for any projected pixel point on the projected edge within the target area in the transmission image, obtaining the normal of the projected edge at the projected pixel point, and taking several pixel points adjacent to the projected pixel point in the positive direction and negative direction of the normal in the target area as the positive local pixel point and negative local pixel point of the projected pixel point respectively; taking the absolute value of the difference mean of all adjacent positive local pixel points and the absolute value of the difference mean of all adjacent negative local pixel points as the grayscale gradient amount of the corresponding projected pixel point in the positive direction and negative direction of its normal; and determining the edge grayscale gradient degree of the target area according to the grayscale gradient amounts in the positive direction and negative direction.

[0014] The present invention takes into account that the structural changes of gangue with higher density can bring more significant grayscale changes in the transmission image, while the structural changes of coal with lower density do not bring obvious grayscale changes in the transmission image. The projection edge in the first grayscale image is the location of the structural change of coal or gangue. Therefore, the present invention combines the grayscale changes around the projection edge in the target area in the transmission image to obtain the edge grayscale gradient of the target area, providing a mathematical basis for the subsequent distinction between coal and gangue, and can better distinguish thicker coal and thinner gangue.

[0015] Preferably, the grayscale gradient degree satisfies the expression: ;in, Indicates the grayscale gradient of the edge of the target area; Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the positive direction of its normal, Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the negative direction of its normal; Indicates the target area The number of projected pixels on the edge of the projection strip; Indicates the number of projected edges in the target area; is the first hyperparameter; represents the hyperbolic tangent function.

[0016] Preferably, the modified probability satisfies the expression: ;in, Indicates the corrected probability that the target area is gangue; Indicates the probability that the target area is gangue; Indicates the grayscale gradient of the edge of the target area; is the second hyperparameter; is the third hyperparameter.

[0017] Preferably, the method of locating the gangue according to the gangue area and constructing a three-dimensional model of the gangue includes: determining the surface depth information of the gangue in the gangue area in the first grayscale image through a stereo vision algorithm based on the first grayscale image and the second grayscale image of the gangue, and constructing a three-dimensional model of the gangue using point cloud reconstruction and surface fitting technology; the second grayscale image and the first grayscale image are images of the same area from different perspectives; and determining the real-time coordinate position of the gangue according to the operating speed of the coal conveyor.

[0018] Preferably, predicting the quality of the gangue based on the three-dimensional model of the gangue includes: predicting the quality of the gangue using a neural network.

[0019] Preferably, the pushing force satisfies the expression: ;in, Indicates the pushing force; Indicates the quality of gangue; represents the acceleration due to gravity; Indicates the dynamic friction coefficient of gangue on coal conveyor; Indicates the acceleration required to push the gangue to the sorting channel; Indicates the distance from the gangue to the sorting channel.

[0020] The present invention dynamically adjusts the pushing strength according to the quality of the gangue and the distance from the gangue to the sorting channel, which not only avoids energy waste but also ensures accurate delivery to the sorting channel, and can realize precise mechanical control of the sorting robot during sorting.

[0021] In a second aspect, the present invention provides a control system for an intelligent sorting robot, comprising a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the control method of the intelligent sorting robot described above is implemented.

[0022] By adopting the above technical solution, the control method of the above-mentioned intelligent sorting robot is generated into a computer program and stored in a memory to be loaded and executed by a processor, so that a terminal device is made based on the memory and the processor for easy use.

[0023] The beneficial effects of the present invention are:

[0024] The present invention can completely separate separate coal or gangue, providing a basis for subsequent gangue identification; the present invention adopts a dual verification mechanism of grayscale characteristics of the area to be identified in the transmission image and edge gradient characteristics, thereby improving the accuracy of gangue identification; the present invention can dynamically adjust the pushing intensity, which not only avoids the use of a large pushing force on lighter gangue, resulting in energy waste and even causing gangue to rush out of the sorting channel, collide with equipment, etc., but also ensures accurate delivery of the sorting channel, and can also make heavier gangue completely enter the sorting channel, thereby realizing precise mechanical control of the sorting robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart schematically illustrating a control method of an intelligent sorting robot in the present invention;

[0026] Figure 2 is a flow chart schematically illustrating step S2 of a control method for an intelligent sorting robot in the present invention;

[0027] Figure 3 Schematic diagram showing normal lines. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0029] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] The embodiment of the present invention discloses a control method for an intelligent sorting robot, referring to Figure 1 , including steps S1 to S6:

[0031] S1. Collect transmission images and grayscale images of raw coal to be sorted on a coal mine conveyor.

[0032] Specifically, an imaging system was installed on the raw coal conveyor line. The system, consisting of a high-resolution industrial camera and an X-ray generator, was linked to the coal mine conveyor and controlled. To ensure accurate image acquisition, multiple positioning points were set within the imaging area.

[0033] When the raw coal on the conveyor passes the first positioning point in the imaging area, the imaging system performs the following operations simultaneously: First, the industrial camera captures a grayscale image of the raw coal; simultaneously, the X-ray generator captures a transmission image of the raw coal. The captured grayscale image and transmission image contain information about the pre-set first positioning point. This captured grayscale image is called the first grayscale image.

[0034] When the raw coal on the conveyor passes the second positioning point in the imaging area, the industrial camera captures another grayscale image of the raw coal. This grayscale image is called the second grayscale image. The first grayscale image and the second grayscale image are images of the same area from different perspectives.

[0035] Based on the positioning points in the first grayscale image and the transmission image, an image registration algorithm is used to perform spatial alignment processing on the first grayscale image and the transmission image to ensure consistency in spatial position between the first grayscale image and the transmission image.

[0036] Based on the registration results of the first grayscale image and the transmission image, the first grayscale image and the transmission image are cropped so that they correspond to the same observation area. The cropped transmission image is resized to the same size as the first grayscale image using image processing techniques such as interpolation or downsampling.

[0037] S2. Identify the gangue area in the transmission image and the first grayscale image.

[0038] See the flowchart of step S2 Figure 2 , including steps S201 to S205, specifically:

[0039] S201 : Determine corresponding areas to be identified in the transmission image and the first grayscale image according to edges in the transmission image and the first grayscale image.

[0040] Specifically, the Canny edge detection algorithm is used to perform edge detection on the transmission image and the first grayscale image, respectively, to obtain edges in the transmission image and edges in the first grayscale image. In other embodiments, implementers can select an edge detection algorithm based on actual implementation conditions, such as a Sobel operator, a Prewitt operator, etc.

[0041] It should be noted that the edges in the first grayscale image include the edges of the coal or gangue, as well as the texture edges representing structural changes on the surface of the coal or gangue. The edges in the transmission image include the edges of the coal or gangue, as well as structural information within the coal or gangue. Therefore, the present invention obtains the area to be identified based on the corresponding edges in the first grayscale image and the transmission image.

[0042] Furthermore, overlapping edge pixels in the transmission image and the first grayscale image are obtained. Using these overlapping edge pixels as boundaries, corresponding closed regions are demarcated in each of the transmission image and the first grayscale image, with each closed region serving as a separate region to be identified. It should be noted that the regions to be identified have a one-to-one correspondence in the transmission image and the first grayscale image, and are located and shaped identically.

[0043] S202 , taking any area to be identified as a target area, and determining the probability that the area to be identified is gangue according to the grayscale mean value of the corresponding pixel points of the target area in the transmission image.

[0044] It should be noted that coal and gangue have different densities and compositions, and their absorption characteristics for radiation are also different. Coal is mainly composed of carbon, has a low density, and absorbs radiation weakly; gangue contains a variety of mineral components, has a high density, and absorbs radiation strongly. When radiation penetrates an object, the part with stronger absorption will block more radiation, resulting in a decrease in the intensity of the radiation after transmission, making the radiation signal received by the detector weaker. In the generated transmission image, the area with stronger absorption will appear as a darker grayscale (low grayscale value). Due to its high density and strong absorption, gangue appears as a darker area in the transmission image, while coal has a low density and weak absorption, and appears as a brighter area in the transmission image. Therefore, the present invention determines the probability that the target area is gangue based on the grayscale mean value of the pixel points corresponding to the target area in the transmission image.

[0045] Specifically, the probability that the target area is gangue satisfies the expression:

[0046] ;

[0047] Where, Indicates the probability that the target area is gangue; Represents the mean grayscale value of all pixels corresponding to the target area in the transmission image; Represents the maximum value among the mean values ​​of the grayscale values ​​of all pixels corresponding to each area to be identified in the transmission image; is an exponential function with a natural constant as the base, used to Perform negative correlation mapping.

[0048] In the transmission image, the mean grayscale value of the pixels corresponding to the target area can effectively characterize the overall grayscale characteristics of the target area. According to the principle of ray transmission, the larger the overall grayscale of the target area, the higher the intensity of the transmitted ray, the more likely the target area is a coal area, and the lower the probability of the target area being gangue. Conversely, the smaller the overall grayscale of the target area, the lower the intensity of the transmitted ray, the more likely the target area is a gangue area, and the higher the probability of the target area being gangue. Considering that coal is the main component of raw coal, while gangue accounts for a relatively small proportion, coal areas must exist in the transmission image. The maximum value among the mean grayscale values ​​of all pixels corresponding to each area to be identified in the transmission image can be used as the typical grayscale characteristic of the coal area. Therefore, the present invention normalizes the overall grayscale of the target area by obtaining the ratio of the overall grayscale of the target area to the maximum value, avoiding the problem that the result of the negative correlation mapping is always small due to the large overall grayscale of the target area, thereby affecting the accuracy of gangue identification.

[0049] S203 : Determine a grayscale gradient degree of an edge of the target area according to a grayscale change of a local position of an edge of the target area contained in the first grayscale image in the transmission image.

[0050] It should be noted that the brightness of a target area in a transmission image depends not only on the material of the object corresponding to the target area but also on the object's thickness. A thicker object blocks more radiation, resulting in a lower intensity of the transmitted radiation and a weaker radiation signal received by the detector. The corresponding area appears darker in the transmission image. A thinner object, however, increases the intensity of the transmitted radiation and results in a brighter grayscale. Therefore, thicker coal and thinner gangue may appear similar in overall grayscale in the transmission image, affecting gangue identification accuracy. When the structure of either gangue or coal changes, the localized area of ​​the structural change will exhibit a grayscale gradient in the transmission image. However, due to the higher density of gangue and the lower density of coal, the same structural change in gangue and coal will result in different grayscale changes in the transmission image. Structural changes in gangue will result in a more pronounced grayscale gradient in the transmission image, while structural changes in coal will not. The edges within each to-be-identified region in the first grayscale image represent the locations of structural changes in the coal or gangue corresponding to the to-be-identified region. Therefore, the present invention determines the degree of grayscale gradient at the edge of the to-be-identified region based on the grayscale changes in the transmission image at the local locations of the edges contained in the first grayscale image.

[0051] Specifically, the edge in the target area of ​​the first grayscale image is projected into the transmission image as the projected edge in the target area in the transmission image.

[0052] For any projected pixel point on the projection edge, obtain the normal of the projection edge at the projection pixel point, and convert the normal line adjacent to the projection pixel point in the positive direction of the normal line in the target area into the normal line. Pixels are taken as the positive local pixels of the projected pixel. The negative direction of the normal in the target area adjacent to the projected pixel pixel points as the negative local pixel points of the projected pixel point. This is the preset first quantity, which can be set by the implementer according to the actual implementation situation, for example When the number of pixels adjacent to the projected pixel in the positive direction of the normal line of the projected pixel in the target area of ​​the transmission image is insufficient When counting, the actual number is counted. Figure 3 is a normal diagram, Figure 3 The middle curve is the projected edge, the dotted line is the tangent line at the projected pixel point A, the straight line perpendicular to the dotted line at the projected pixel point A is the normal line at the projected pixel point A, arrow B is the positive direction of the normal line, and arrow C is the negative direction of the normal line.

[0053] According to the grayscale changes of the positive local pixels and the negative local pixels of each projection pixel on each projection edge in the target area of ​​the projected image, the grayscale gradient of each projection pixel in the positive direction of its normal and the grayscale gradient of each projection pixel in the negative direction of its normal are determined:

[0054] ;

[0055] ;

[0056] in, Indicates the first The first projection edge The grayscale gradient of the projected pixel in the positive direction of its normal, Indicates the first The first projection edge The grayscale gradient of the projected pixel in the negative direction of its normal; Indicates the first The first projection edge The number of positive local pixels of the projected pixel; Indicates the first The first projection edge The number of negative local pixels of the projected pixel; Indicates the first The first projection edge The projected pixel The gray value of the positive local pixel, Indicates the first The first projection edge The projected pixel Gray value of positive local pixel; Indicates the first The first projection edge The projected pixel Gray value of negative local pixel, Indicates the first The first projection edge The projected pixel Gray value of negative local pixel; Indicates the absolute value symbol.

[0057] It should be noted that, since the thickness changes on both sides of the edge may be inconsistent, the present invention calculates the grayscale gradient separately for the positive and negative directions of the normal. If there is no pixel in the target area in the positive direction of the normal of some projected pixel points, the grayscale gradient of the projected pixel point in the positive direction of its normal is not calculated, and the grayscale gradient of the projected pixel point in the positive direction of its normal is directly marked as 0; if there is no pixel in the target area in the negative direction of the normal of some projected pixel points, the grayscale gradient of the projected pixel point in the negative direction of its normal is not calculated, and the grayscale gradient of the projected pixel point in the negative direction of its normal is directly marked as 0.

[0058] The grayscale gradient degree of the edge of the target area is determined according to the grayscale gradient amount of each projected pixel point on each projected edge in the target area in the projected image in the positive direction of its normal and the grayscale gradient amount in the negative direction:

[0059] ;

[0060] in, Indicates the grayscale gradient of the edge of the target area; Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the positive direction of its normal, Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the negative direction of its normal; Indicates the target area The number of projected pixels on the edge of the projection strip; Indicates the number of projected edges in the target area; is the first hyperparameter, used to avoid Too large, resulting in Always close to 1, The experience value is , the implementation personnel can set according to the actual implementation situation The value of Represents the hyperbolic tangent function, which is used to Perform normalization.

[0061] It should be noted that when there is no edge in the target area of ​​the first grayscale image, there is no projected edge in the target area of ​​the transmission image. In this case, the grayscale gradient degree of the edge of the target area is set to 0.

[0062] S204 , correcting the probability that the target area is gangue according to the grayscale gradient of the edge to obtain a corrected probability that the target area is gangue.

[0063] Specifically, the modified probability satisfies the expression:

[0064] ;

[0065] in, Indicates the corrected probability that the target area is gangue; Indicates the probability that the target area is gangue; Indicates the grayscale gradient of the edge of the target area; Is the second hyperparameter, used to adjust the edge grayscale gradient The empirical value is 0.75, and the implementer can set it according to the actual implementation situation. is the third hyperparameter, used to prevent When the denominator is 0, the empirical value is 0.0001, and the implementer can set it according to the actual implementation situation.

[0066] when Less than or equal to When , the grayscale gradient of the edge is small, which means that the structural change of the object corresponding to the target area has little effect on the intensity of the transmitted ray. The target area is more likely to be coal, so As an index, the probability of the target area being gangue is calculated by using the gamma change method. To reduce, when the edge gray gradient The smaller the time, The larger the value, the higher the probability that the target area is gangue. The greater the degree of reduction; Greater than When , the grayscale gradient of the edge is large, which means that the structural change of the object corresponding to the target area has a greater impact on the intensity of the transmitted ray. The target area is more likely to be gangue, so As an index, the probability of the target area being gangue is calculated by using the gamma change method. Increase, when the edge gray gradient The bigger it is, The more filial, the probability that the target area is gangue The greater the increase.

[0067] It should be noted that the present invention combines the overall grayscale size of the target area and the degree of grayscale gradient at the edge, taking into account both the influence of the density of coal and gangue on the intensity of the rays after transmission and the influence of the structural changes of coal and gangue on the intensity of the rays after transmission, so that the corrected probability that the target area is gangue is more accurate, and can avoid the problem of inaccurate identification of gangue caused by the overall grayscale of thicker coal and thinner gangue in the transmission image being close.

[0068] S205. Filter the waste rock area according to the size of the corrected probability.

[0069] Specifically, in response to the modified probability that the target area is gangue If the probability is greater than the preset threshold, the target area is the gangue area; otherwise, the target area is the coal area.

[0070] The probability threshold can be set by the implementer according to the actual implementation situation, for example, 0.5.

[0071] At this point, the waste rock area is obtained.

[0072] S3. Locate the gangue on the coal conveyor according to the gangue area and construct a three-dimensional model of the gangue.

[0073] According to the first grayscale image and the second grayscale image, the surface depth information of the gangue in the gangue area in the first grayscale image is determined by a stereo vision algorithm, and a three-dimensional model of the gangue is constructed by adopting point cloud reconstruction and surface fitting technology.

[0074] According to the running speed of the coal conveyor, the real-time coordinate position of the gangue is determined.

[0075] S4. Predict the quality of the gangue based on the three-dimensional model of the gangue.

[0076] Specifically, the present invention predicts the mass of gangue through a neural network, the input of the neural network is a three-dimensional model of the gangue, and the output is the mass of the gangue; the data set is a data set composed of three-dimensional models of gangue of different shapes, and the label is the mass of the gangue; the loss function of the neural network is the mean square error loss.

[0077] The three-dimensional model of the gangue is input into the neural network, and the weight of the gangue is output.

[0078] S5. Determine the pushing force of the sorting robot on the gangue according to the mass of the gangue.

[0079] Specifically, the pushing force satisfies the expression:

[0080] ;

[0081] in, Indicates the pushing force; Indicates the quality of gangue; represents the acceleration due to gravity; Indicates the dynamic friction coefficient of gangue on coal conveyor; Indicates the acceleration required to push the gangue to the sorting channel; Indicates the distance from the gangue to the sorting channel. Among them, the dynamic friction coefficient It is set by the implementers based on experience, and is related to the surface roughness of the gangue and the material of the conveyor belt. Experience shows that when When set to 0.6, it can cover most working conditions.

[0082] Furthermore, the acceleration required to push the gangue into the sorting channel Satisfies the expression:

[0083] ;

[0084] in, Indicates the distance from gangue to the sorting channel, The target sorting time for a single piece of gangue is set by real-time personnel based on the actual real-time situation, for example, 1 second.

[0085] S6. Control the sorting robot to push the gangue to the sorting channel according to the pushing force.

[0086] Specifically, according to the real-time coordinate position of the gangue, the motion trajectory of the end effector of the sorting robot is accurately adjusted through the closed-loop control system, and the pushing force obtained in step S5 is used to accurately guide the gangue to the sorting channel on the side of the coal conveyor.

[0087] Through the above method, efficient and accurate sorting of gangue in coal mines can be achieved, which can significantly improve sorting efficiency, reduce manual intervention and reduce operating costs.

[0088] An embodiment of the present invention further discloses a control system for an intelligent sorting robot, comprising a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a control method for an intelligent sorting robot according to the present invention is implemented.

[0089] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and will not be described in detail here.

Claims

1. A control method for an intelligent sorting robot, characterized in that: include: determining corresponding areas to be identified in the transmission image and the first grayscale image according to the transmission image of the raw coal to be sorted and the edges in the first grayscale image; Determine the probability that the area to be identified is gangue based on the grayscale mean of the corresponding pixel points of the area to be identified in the transmission image; determine the degree of grayscale gradient of the edge of the area to be identified based on the grayscale change of the local position of the edge contained in the first grayscale image of the area to be identified in the transmission image; Correcting the probability that the area to be identified is gangue according to the degree of edge grayscale gradient, and screening the gangue area according to the magnitude of the corrected probability; Locate the gangue according to the gangue area and construct a three-dimensional model of the gangue; predict the quality of the gangue based on the three-dimensional model of the gangue; determine the pushing force of the sorting robot on the gangue based on the quality of the gangue; The sorting robot is controlled according to the pushing force to push the gangue to the sorting channel.

2. The control method of an intelligent sorting robot according to claim 1, characterized in that: The determining of the corresponding areas to be identified in the transmission image and the first grayscale image includes: Obtain the edge pixel points that overlap in the transmission image and the first grayscale image, and use the overlapping edge pixel points as boundaries to divide corresponding closed areas in the transmission image and the first grayscale image, respectively, and treat each closed area as an independent area to be identified.

3. The control method of an intelligent sorting robot according to claim 1, characterized in that: The method of determining the probability that the area to be identified is gangue according to the grayscale mean of the pixel points corresponding to the area to be identified in the transmission image includes: The average of the grayscale values ​​of the corresponding pixel points of the area to be identified in the transmission image is taken as the overall grayscale of the area to be identified; the ratio of the overall grayscale of the area to be identified to the maximum value of the overall grayscale of all areas to be identified is obtained, and the ratio is negatively correlated with each other to obtain the probability that the area to be identified is gangue.

4. The control method of an intelligent sorting robot according to claim 1, characterized in that: The method for obtaining the grayscale gradient degree of the edge of the area to be identified is: The edge within the target area of ​​the first grayscale image is projected into the transmission image. For any projected pixel point on the projected edge within the target area in the transmission image, the normal of the projected edge at the projected pixel point is obtained, and several pixel points adjacent to the projected pixel point in the positive direction and negative direction of the normal within the target area are respectively used as the positive local pixel point and negative local pixel point of the projected pixel point; the absolute value of the difference mean of all adjacent positive local pixel points and the absolute value of the difference mean of all adjacent negative local pixel points are respectively used as the grayscale gradient amount of the corresponding projected pixel point in the positive direction and negative direction of its normal; and the degree of grayscale gradient of the edge of the target area is determined according to the grayscale gradient amounts in the positive direction and negative direction.

5. The control method of an intelligent sorting robot according to claim 4, characterized in that: The grayscale gradient degree satisfies the expression: ； in, Indicates the grayscale gradient of the edge of the target area; Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the positive direction of its normal, Indicates the target area The first projection edge The grayscale gradient of the projected pixel in the negative direction of its normal; Indicates the target area The number of projected pixels on the edge of the projection strip; Indicates the number of projected edges in the target area; is the first hyperparameter; represents the hyperbolic tangent function.

6. The control method of an intelligent sorting robot according to claim 1, characterized in that: The modified probability satisfies the expression: ； in, Indicates the corrected probability that the target area is gangue; Indicates the probability that the target area is gangue; Indicates the grayscale gradient of the edge of the target area; is the second hyperparameter; is the third hyperparameter.

7. The control method of an intelligent sorting robot according to claim 1, characterized in that: Positioning the gangue according to the gangue area and constructing a three-dimensional model of the gangue includes: Based on the first grayscale image and the second grayscale image of the gangue, the surface depth information of the gangue in the gangue area in the first grayscale image is determined by a stereo vision algorithm, and a three-dimensional model of the gangue is constructed using point cloud reconstruction and surface fitting technology; the second grayscale image and the first grayscale image are images of the same area from different perspectives; the real-time coordinate position of the gangue is determined according to the operating speed of the coal conveyor.

8. The control method of an intelligent sorting robot according to claim 1, characterized in that: The method of predicting the quality of the gangue based on the three-dimensional model of the gangue includes: Using neural networks to predict the quality of waste rock.

9. The control method of an intelligent sorting robot according to claim 1, characterized in that: The pushing force satisfies the expression: ； in, Indicates the pushing force; Indicates the quality of gangue; represents the acceleration due to gravity; Indicates the dynamic friction coefficient of gangue on coal conveyor; Indicates the acceleration required to push the gangue to the sorting channel; Indicates the distance from the gangue to the sorting channel.

10. A control system for an intelligent sorting robot, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a control method for an intelligent sorting robot according to any one of claims 1 to 9 is implemented.

Images