A mineral ball identification, positioning and sorting system and method for a ball forming machine

Abstract

The present invention discloses a mineral ball identification, positioning and sorting system for a ball forming machine, comprising: a camera, a computer, a light source component, a mechanical component and a communication link. The present invention also discloses a mineral ball identification, positioning and sorting method for a ball forming machine, which obtains image data collected by a camera, sends brightness adjustment requirement information according to the image data, identifies and locates the over-standard spheres in the image data, determines the running track of the over-standard spheres, and controls the mechanical component to grab the over-standard spheres according to their running tracks. Compared with the existing manual patrol method, it has the advantages of high automation and strong safety, and compared with the existing identification and sorting method, it has the advantages of high image quality and dynamic identification, and is more suitable for harsh working environments, while improving the accuracy and real-time performance of the identification, positioning and sorting of over-standard spheres.

Description

Technical Field

[0001] The present invention relates to the technical field of mineral ball sorting, and in particular to a mineral ball identification, positioning and sorting system and method for a ball forming machine. Background Art

[0002] The ball forming machine is used to make mineral powder into a spherical shape with a certain hardness. When the diameter of the spherical particles reaches a certain degree, they are screened and graded through the screen in the ball forming machine. Spherical particles of different sizes and qualities are collected separately, and unqualified particles can be returned for reprocessing.

[0003] During the ball-making process, the ball-forming machine will produce individual spherical particles with diameters exceeding the threshold. Spherical particles with diameters exceeding the threshold are called over-standard spheres. Since the over-standard spheres will swing back and forth in a relatively fixed small area, they cannot flow into the screen according to the planned path of the ball-forming machine. In addition, since the over-standard spheres have a large mass, they will collide with other particles during the swinging process, causing varying degrees of damage to other particles, thereby reducing the yield rate of the ball-forming machine.

[0004] At present, the above technical problems are mainly solved manually, that is, through regular patrols by production workers, the spheres that exceed the standard are manually removed. The disadvantages are: first, it is impossible to accurately control the timing of the appearance of the spheres that exceed the standard, and there is often a time lag in dealing with the spheres that exceed the standard, resulting in poor timeliness of manual processing; second, since the ball forming machine generally works 24 hours a day, it takes at least 2-3 people to maintain the normal operation of the ball forming machine, resulting in high labor costs; third, manual processing may lead to unexpected situations caused by human factors such as forgetfulness, misjudgment and omission, and the processing efficiency and accuracy are low. Summary of the invention

[0005] To solve the above technical problems, the present invention provides a system and method for identifying, locating and sorting mineral balls for a ball forming machine. In order to have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a general review, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0006] The present invention adopts the following technical solution:

[0007] The present invention provides a mineral ball identification, positioning and sorting system for a ball forming machine, comprising:

[0008] Camera, used to shoot the running environment of the mineral balls in the ball forming machine;

[0009] A computer is used to obtain the image data collected by the camera, issue brightness adjustment requirement information according to the image data, identify and locate the excessive sphere in the image data, determine the running track of the excessive sphere, and control the mechanical component to grab the excessive sphere according to the running track;

[0010] The light source assembly is composed of a visible light source and an infrared light source, and is used to adjust the brightness of the light source provided by this end for the operating environment of the mineral balls in the ball forming machine according to the brightness adjustment request information sent by the computer.

[0011] Wherein, the image data is a video stream with a frame rate greater than or equal to 15 Hz; the computer comprises:

[0012] An image enhancement module, used for performing contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image;

[0013] A target recognition module is used to identify the mineral balls in the enhanced image and estimate the diameter of the mineral balls based on the object recognition technology of the deep neural network, and screen out the mineral balls with diameters exceeding the threshold as the exceeding-standard spheres;

[0014] The target tracking module is used to determine the target search area of ​​the next enhanced image frame that is adjacent in time sequence, and to determine whether there is an over-standard sphere in the target search area. If so, the next target search area is determined; if not, the next target search area is expanded, and when an over-standard sphere is screened out, the estimated coordinates of the over-standard sphere are calculated;

[0015] The result output module is used to convert the estimated coordinates of the out-of-standard sphere into the space-time coordinates in the world coordinate system of the material surface, and send the space-time coordinates to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the out-of-standard sphere.

[0016] Wherein, the target tracking module includes:

[0017] A motion domain determination unit is used to determine the motion domain of the over-standard sphere in the enhanced image of the current frame, where the motion domain is an irregular two-dimensional region that conforms to the Gaussian distribution;

[0018] A candidate search area determination unit is used to determine a candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as a target search area of ​​the next frame enhanced image that is adjacent in time sequence;

[0019] The range adjustment unit is used to expand the Gaussian radius of the candidate search area to 1.5 times when it is determined that there is no exceeding sphere in the target search area.

[0020] Wherein, the target tracking module also includes:

[0021] An image coordinate determination unit, used to obtain the image coordinates of the over-standard sphere in the current enhanced image according to the rectangular center position of the detection frame;

[0022] A moving time determination unit is used to determine the time t required for the manipulator in the mechanical assembly to move from the start to the target position after the exceeding spheres are screened out;

[0023] The estimated coordinate determination unit is used to calculate the estimated coordinates of the exceeding standard sphere at time t+1 according to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t.

[0024] Among them, the mineral ball identification, positioning and sorting system for a ball forming machine also includes: a communication link for providing a communication channel between the computer and other computer systems.

[0025] Among them, the mechanical assembly includes: a first hydraulic push rod, a second hydraulic push rod, and a horizontal turntable, a mechanical arm, a mechanical arm and a bidirectional bucket that are articulated in sequence; the fixed end of the first hydraulic push rod is connected to the mechanical arm, and the movable end of the first hydraulic push rod is connected to the mechanical arm; the fixed end of the second hydraulic push rod is connected to the mechanical arm, and the movable end of the second hydraulic push rod is connected to the bidirectional bucket through a bucket connecting rod; the bidirectional bucket includes: a bucket body and a screen, both sides of the bucket body are provided with feed ports, the bottom of the bucket body is provided with a through opening, and the screen is arranged in the through opening.

[0026] The present invention also provides a method for identifying, positioning and sorting mineral balls for a ball forming machine, comprising:

[0027] Acquire image data obtained when a camera shoots the operating environment of the mineral balls in the ball forming machine, issue brightness adjustment requirement information according to the image data, identify and locate the balls exceeding the standard in the image data, determine the running track of the balls exceeding the standard, and control the mechanical components to grab the balls exceeding the standard according to the running track;

[0028] According to the brightness adjustment requirement information, the brightness of the light source provided for the mineral ball operating environment in the ball forming machine is adjusted.

[0029] The image data is a video stream with a frame rate greater than or equal to 15 Hz; the process of identifying and locating the excessive sphere in the image data, determining the running trajectory of the excessive sphere, and controlling the mechanical component to capture the excessive sphere according to the running trajectory of the excessive sphere includes:

[0030] Performing contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image;

[0031] Based on the object recognition technology of deep neural network, the mineral balls in the enhanced image are distinguished and the diameter of the mineral balls is estimated, and the mineral balls with diameters exceeding the threshold are screened out as the exceeding-standard spheres;

[0032] Determine the target search area of ​​the next enhanced image frame that is adjacent in time sequence, and judge whether there is an over-standard sphere in the target search area. If so, continue to determine the next target search area. If not, expand the scope of the next target search area, and calculate the estimated coordinates of the over-standard sphere when the over-standard sphere is screened out;

[0033] The estimated coordinates of the out-of-standard sphere are converted into space-time coordinates in the world coordinate system of the material surface, and the space-time coordinates are sent to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the out-of-standard sphere.

[0034] The process of determining the target search area of ​​the next enhanced image frame that is adjacent in time sequence includes:

[0035] Determine the motion domain of the over-standard sphere in the enhanced image of the current frame, where the motion domain is an irregular two-dimensional region that conforms to the Gaussian distribution;

[0036] Determine a candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as the target search area of ​​the next frame enhanced image that is adjacent in time sequence;

[0037] When it is determined that there is no exceeding sphere in the target search area, the Gaussian radius of the candidate search area is enlarged to 1.5 times.

[0038] The process of calculating the estimated coordinates of the spheres exceeding the standard when the spheres exceeding the standard are screened out includes:

[0039] According to the rectangular center position of the detection frame, the image coordinates of the over-standard sphere in the current enhanced image are obtained;

[0040] After the exceeding spheres are screened out, the time t required for the manipulator in the mechanical assembly to move to the target position from the start is determined;

[0041] According to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t, the estimated coordinates of the exceeding standard sphere at time t+1 are calculated.

[0042] The beneficial effects brought by the present invention: Compared with the existing manual patrol method, the present invention has the advantages of high automation and strong safety. Compared with the existing identification and sorting method, it has the advantages of high image quality and dynamic identification. It is more suitable for harsh working environments and improves the accuracy and real-time performance of the identification, positioning and sorting of over-standard spheres. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a schematic diagram of a mineral ball identification, positioning and sorting system for a ball forming machine of the present invention;

[0045] Figure 2 is a schematic diagram of the structure of the mechanical assembly of the present invention;

[0046] Figure 3 It is a structural schematic diagram of a bidirectional bucket of the present invention;

[0047] Figure 4 The image is a display image of the processing result after the image is processed using the image contrast enhancement method based on smoke modeling in the present invention;

[0048] Figure 5 The image is displayed as a result of the present invention processing the overexposed spots. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It should be clear that the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, in some illustrative embodiments, a mineral ball identification, positioning and sorting system for a ball forming machine is provided, comprising: a camera, a computer, a light source component, a mechanical component, and a communication link.

[0051] The light source component is used to adjust the brightness of the light source provided by this end for the ball forming machine's operating environment according to the brightness adjustment requirement information sent by the computer. Specifically, the light source component is composed of a visible light source and an infrared light source, which provides the ball forming machine's operating environment light source when the camera is collecting images, and provides a visible light and infrared light fill light source with adjustable brightness according to the brightness adjustment requirement information under the computer.

[0052] The camera is an image acquisition device used to capture the operating environment of the mineral balls in the ball forming machine, that is, to capture real-time images of the operation of the mineral balls in the ball forming machine.

[0053] The computer is used to obtain image data collected by the camera, send brightness adjustment requirement information according to the image data, identify and locate the excessive sphere in the image data, determine the movement trajectory of the excessive sphere, and control the mechanical component to grab the excessive sphere according to the movement trajectory.

[0054] Communication link is used to provide a communication channel between the computer and other computer systems. Other computer systems refer to management servers, etc. The communication link can be selected according to the on-site environment but is not limited to communication technologies such as RS485, LoRa, Zigbee, 4G, IoT or Ethernet.

[0055] The image data is a video stream with a frame rate greater than or equal to 15 Hz.

[0056] The computer includes: an image enhancement module, a target recognition module, a target tracking module and a result output module.

[0057] The image enhancement module is used to perform contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image.

[0058] The contrast enhancement processing method is to first use an image contrast enhancement method based on smoke modeling to remove dust from the real-time image, and then process the overexposed points in the real-time image based on a contrast enhancement method based on second-order spectral balance.

[0059] First, the image contrast enhancement method based on smoke modeling is explained:

[0060] The light source assembly used in the present invention can be considered to have uniform illumination within the image range. Therefore, the agitated mineral smoke can be considered to have a similar imaging model to haze, as shown in Formula 1:

[0061] Formula 1: O(x) = J(x)d(x) + I(1-d(x));

[0062] Among them, O(x) is the observation image, J(x) is the dust removal image, d(x) is the dust model, and I is the illumination model. Therefore, if d(x) and I can be obtained, the dust removal image can be obtained. Transforming formula 1 can obtain formula 2:

[0063] Formula 2:

[0064] d(x) satisfies d(x)∈[0,1], the value range of O(x), J(x) and I is [0,255]. Assuming that the uniform illumination model I is not affected by object occlusion and dust, we can determine an inequality 3:

[0065] Inequality 3: 0≤O(x), J(x)≤I.

[0066] According to Formula 2 and Inequality 3, another inequality is derived as Formula 4:

[0067] Formula 4:

[0068] Therefore, d(x) satisfies Formula 5:

[0069] Formula 5: 1-O(x) / I≤d(x)≤1.

[0070] To model d(x), Equation 5 is rewritten as Equation 6:

[0071] Formula 6: d(x)=α(1-O(x) / I);

[0072] Among them, 0≤α≤1, α=0 when and only when there is no dust influence in the image, that is, O(x)=J(x).

[0073] Through actual experimental observations, if a mean filter is used, the dust image is very similar to the severely blurred image after filtering, so Formula 8 is used to simulate the illumination model I:

[0074] Formula 8: min(Ave(img(x)))≤I≤max(Ave(img(x)));

[0075] Here, Ave() refers to a 5×5 averaging filter. Figure 4 Results of the image contrast enhancement method based on smoke modeling are shown.

[0076] Then, the contrast enhancement method based on second-order spectral balance is explained:

[0077] The real-time image contains dark areas caused by mineral absorption characteristics and diffuse reflection, and overexposed spots caused by local specular reflection. The dynamic range of the dark area is narrow, making the boundaries of adjacent spherical minerals unclear, while the overexposed spots may cause errors in mineral ball segmentation / detection. For example, a spot has a significant brightness difference with the adjacent parts, making it likely to be detected as an independent individual even though it belongs to a part of a sphere. Therefore, it is necessary to deal with this optical imbalance between the same target and different targets, which includes two sub-problems: expanding the dynamic range of the dark area and suppressing the high-brightness part at the same time.

[0078] Overexposed spots can be regarded as randomly distributed, small high-brightness areas, which can be regarded as a randomly distributed salt-and-pepper noise with different sizes. Therefore, the present invention does not use a fixed-size sliding window for filtering, but uses several windows of adaptive sizes. First, the overexposed spots are located by image binarization, and then filters of different sizes are used for each connection domain. When the overexposed spots belong to the nearby regular part, the median filter is used for processing. The processing results are shown in FIG. Figure 5 shown.

[0079] First, find the median δ in the M×M window at position (i, j), where the value of M is determined by Equation 9:

[0080] Formula 9: M = ^ M+λ;

[0081] in, ^ The side length of the square of M is limited by a specific connection domain, λ is an integer, and the present invention uses λ=3.

[0082] Then, all pixel values ​​in the filter centered at (i, j) are redistributed according to Equation 10:

[0083] Formula 10:

[0084] Where k is a constant used to adjust the overall brightness of the image, (m,n) represents each pixel in the filter window centered at (i,j), and Ave(m,n) is the average value obtained by the 3×3 sub-window centered at (m,n), so the filtering process changes all pixel values.

[0085] After removing the overexposed spots, the dynamic range of the dark area needs to be expanded. Therefore, the present invention proposes a two-stage image transformation, such as formula 11. Formula 11 is a piecewise function, which consists of an exponential transformation applied to the dark area and a linear transformation applied to other parts. The exponential transformation can expand the dynamic range of the image, while the linear transformation can limit the high-brightness part. Formula 11 is as follows:

[0086]

[0087] During the production and transportation of minerals, a large amount of dust is generated, which increases the difficulty and error rate of identifying mineral balls. The contrast enhancement processing method of the present invention processes the real-time image into an enhanced image, which can improve the accuracy of subsequent identification of excessive balls in the image and effectively enhance the image quality collected in harsh industrial and mining environments.

[0088] The target recognition module is used to identify the mineral balls in the enhanced image and estimate their diameter based on deep neural network object recognition technology, and screen out the mineral balls with diameters exceeding the threshold as excessive spheres.

[0089] The present invention uses SSD network for mineral ball detection, and deep neural network can automatically extract and abstract the correlation of high-dimensional data layer by layer. Since mineral balls lack sufficient distinguishable features except contour features, object recognition technology based on deep neural network is more suitable for target detection of mineral balls.

[0090] For the object detectors in the SSD network, they have feature maps and receptive fields of different sizes, so an adaptive detection box can be designed so that the deep neural network can find mineral balls of different diameters and estimate their actual size in an image.

[0091] Since there are k feature maps to detect objects, the default size Si in the feature map is as follows:

[0092] Si=α+(i-1)(β-α) / (k-1),i∈[1,k];

[0093] Among them, α=0.30 and β=0.95 are the minimum and maximum ratios of the box.

[0094] The present invention uses multiple αr to meet the requirements of boxes of different shapes. In this way, the default width wi and default height hi of the i-th box can be calculated by the following formula:

[0095]

[0096] The position of the target is regarded as a plane, and the real size of the target can be transformed by the number of pixels. In the present invention, the outline of the target is generally not a perfect square, so the diagonal of the outline is used as the diameter of the mineral ball for transformation to obtain a similar value of the actual diameter of the mineral ball.

[0097] The target tracking module is used to determine the target search area of ​​the next frame enhanced image that is closely adjacent in time sequence, and to judge whether there is an excessive sphere in the target search area. If so, the next target search area is determined; if not, the range of the next target search area is expanded, and when an excessive sphere is screened out, the estimated coordinates of the excessive sphere are calculated.

[0098] The result output module is used to convert the estimated coordinates of the over-standard sphere into the space-time coordinates of the material surface in the world coordinate system, and send the space-time coordinates to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the over-standard sphere. The material surface refers to the surface of the mineral material in the ball forming machine.

[0099] The target tracking module includes: a motion domain determination unit, a candidate search area determination unit, a range adjustment unit, an image coordinate determination unit, a moving time determination unit, and an estimated coordinate determination unit.

[0100] The motion domain determination unit is used to determine the motion domain of the over-standard sphere in the enhanced image of the current frame, and the motion domain is an irregular two-dimensional area that conforms to the Gaussian distribution.

[0101] The candidate search area determination unit is used to determine the candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as the target search area of ​​the next frame enhanced image that is adjacent in time sequence.

[0102] Although there are factors such as collision and extrusion, the general movement direction of the mineral balls can be determined due to the regular movement of the ball forming machine.

[0103] After determining the rotational angular velocity of the ball forming machine, a certain margin of motion domain can be determined based on the distance between the location of the exceeding ball and the rotation center of the ball forming machine. Due to the difference in centrifugal force, the closer to the rotation center of the ball forming machine, the smaller the motion domain coverage. Since the impact of collision, friction, etc. of each mineral ball can be regarded as a random independent event, the present invention sets the motion domain of the mineral ball and the candidate search area in the image as an irregular two-dimensional area that conforms to the Gaussian distribution according to the location of the mineral ball and the movement direction of the ball forming machine.

[0104] Let r1 be the Gaussian radius of the candidate search area of ​​the ore ball that is closer to the rotation center of the ball forming machine, and r2 be the Gaussian radius of the candidate search area of ​​the ore ball that is farther from the rotation center of the ball forming machine, then r1 and r2 satisfy: r2 = α·r1,1≤α, where α is positively correlated with the rotation angular velocity of the ball forming machine. The determined candidate search area is used as the target search area of ​​the next frame of enhanced image that is adjacent in time sequence, which narrows the search range when screening the target in the next frame of enhanced image, not only reducing the amount of calculation to ensure real-time performance, but also making the results more reliable.

[0105] It is worth noting that the purpose of the ball forming machine is to mix the mineral powder into spheres, and then transport the finished mineral balls to the belt conveyor through the finished product screen. Due to the uncertainty of the movement process, if the spheres that reach the finished product diameter are not filtered by the finished product screen, the diameter of the spheres will expand to the detection threshold and continue to increase. When the spheres expand to a certain size, due to the segregation of the fluid, the movement of the target spheres will no longer rotate rapidly with the rotation of the ball forming machine, but will stay in a small area on the periphery and oscillate back and forth with a small amplitude. This makes it unnecessary for the manipulator of the mechanical component to quickly track or predict the position of the target ball while retaining a certain space margin.

[0106] The range adjustment unit is used to expand the Gaussian radius of the candidate search area to 1.5 times when it is determined that there is no over-standard sphere in the target search area. The candidate search area after the Gaussian radius is expanded is the target search area of ​​the next enhanced image frame that is adjacent in time sequence.

[0107] When it is determined that there is an over-standard sphere in the target search area, the process of determining the target search area of ​​the next frame of enhanced image that is adjacent in time sequence is continued, that is, the tracking of the over-standard sphere is realized. When it is determined that there is no over-standard sphere in the target search area, that is, when the tracking target is not found in the subsequent frame, an expansion factor greater than or equal to 1 is set for the Gaussian radius of the candidate search area to expand the search range. In the present invention, the expansion factor is selected as 1.5 to ensure a high matching success rate without reducing credibility. If the target to be tracked is still not found after the candidate search area is expanded, it is determined that the over-standard sphere has lost the need to be tracked due to reasons such as coverage, fragmentation, and transportation out of the detection area.

[0108] The image coordinate determination unit is used to obtain the image coordinates of the over-standard sphere in the current enhanced image according to the rectangular center position of the detection frame.

[0109] The moving time determination unit is used to determine the time t required for the manipulator in the mechanical assembly to move from starting to the target position after screening out the exceeding spheres.

[0110] The estimated coordinate determination unit is used to calculate the estimated coordinates of the exceeding standard sphere at time t+1 according to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t.

[0111] The image coordinates of the target in the image are obtained according to the rectangular center position of the detection frame. After finding the target, it takes time t for the robot arm used for sorting to start and move to the target position. Therefore, it is necessary to determine the image coordinates at time t and the estimated coordinates at time t+1. Time t+1 is the time when the robot arm grabs.

[0112] Since the image data is a video stream with a frame rate greater than or equal to 15Hz, the moving distance and moving direction of the over-standard sphere in the X and Y directions in adjacent frames can be considered to be basically consistent. Therefore, the estimated coordinates of the over-standard sphere at time t+1 can be calculated based on the moving distance and moving direction of the over-standard sphere in the X and Y directions from time t-1 to time t. After the estimated coordinates are obtained, the estimated coordinates of the over-standard sphere can be converted into the space-time coordinates of the world coordinate system of the material surface.

[0113] Since the camera position is fixed, the transformation relationship between the estimated coordinates and the time-space coordinates can be obtained. Since the amount of material carried by the ball forming machine is basically the same when it is running, the position of the camera is taken as the physical origin, that is, the origin of the world coordinate system, and its distance to the top layer of the material of the ball forming machine, that is, the z-axis distance can be considered fixed. Moreover, since only the top layer of the material that can be visually detected needs to be detected, in the physical space, it is only necessary to convert the estimated coordinates of the position of the excessive sphere in the enhanced image to the world coordinate system of the material surface. This conversion only requires knowing the intrinsic and extrinsic matrix provided by the camera, establishing the rotation and translation relationship, and obtaining it through matrix operations.

[0114] While successfully grasping the out-of-standard sphere, the space margin of the manipulator should not be too high, otherwise it will cause waste of material. Therefore, it is necessary to locate the position of the manipulator in the world coordinate system while positioning the out-of-standard sphere. The approximate position of the manipulator can be obtained through the control instructions such as extension and translation sent to the mechanical components.

[0115] On this basis, a high-pass position indicator light is set at the center of the manipulator. The light-emitting component can effectively penetrate the agitated dust and form a point-like light spot in the camera, so as to accurately image in real-time and enhanced images. Therefore, the result output module is also used to map the coordinates of the manipulator in the enhanced image to the world coordinate system, calculate the position difference between the manipulator and the over-standard sphere, and issue a control instruction for indicating the next position movement direction and distance of the mechanical component. More importantly, since there is no time difference between the position of the over-standard sphere and the position of the manipulator in one frame of image, this relative distance is accurate, which helps to improve the grasping success rate while tightening the size margin of the manipulator.

[0116] The computer is also used to organize and store the key information of the balls exceeding the standard after each identification, and transmit it to other computer systems through communication links. After other computer systems collect the key information of the balls exceeding the standard and analyze the data, they can be used to support people or other automated equipment to control the operating parameters of the ball forming machine, such as the amount of water injected, the rotation speed and tilt angle of the ball forming machine's ball disc, etc., to reduce the frequency of balls exceeding the standard produced by the ball forming machine and improve the uniformity of balls produced by the ball forming machine.

[0117] For video streams with a frame rate greater than or equal to 15Hz, in each frame of real-time images, the positioning and diameter identification of mineral balls are completed through image enhancement and spherical material identification based on deep neural networks, and targets of non-compliant sizes are searched and screened. In the enhanced images of temporal adjacency, similarity judgment and matching are performed in the target search area through motion trend discrimination to achieve target matching of continuous frames, and target tracking and positioning tasks are completed according to the continuous motion trajectory of the target, providing the location information of the target for subsequent mechanical component operations. Compared with the existing manual patrol method, it has the advantages of high degree of automation and strong safety, and compared with the existing identification and sorting methods, it has the advantages of high image quality and dynamic identification, which is more suitable for harsh working environments, while improving the accuracy and real-time performance of the identification, positioning and sorting of over-standard spheres.

[0118] like Figure 2-3 As shown, the mechanical assembly includes: a first hydraulic push rod 1, a second hydraulic push rod 2, a horizontal turntable 3, a mechanical arm 4, a mechanical arm 5, a bidirectional bucket 6, and a connecting rod 7.

[0119] The horizontal turntable 3 is used to realize the rotation of the mechanical component in the horizontal direction. The mechanical arm 4 is hinged to the horizontal turntable 3, the mechanical arm 5 is hinged at the front end of the mechanical arm 4, and the two-way bucket 6 is hinged at the front end of the mechanical arm 5. The fixed end of the first hydraulic push rod 1 is connected to the mechanical arm 4, and the movable end of the first hydraulic push rod 1 is connected to the mechanical arm 5, so that the mechanical arm 5 can swing at the front end of the mechanical arm 4. The fixed end of the second hydraulic push rod 2 is connected to the mechanical arm 5, and the movable end of the second hydraulic push rod 2 is connected to the two-way bucket 6 through the bucket connecting rod 7, so that the two-way bucket 6 can swing at the front end of the mechanical arm 5. The mechanical arm 5 is the manipulator mentioned in this article.

[0120] The mechanical assembly is installed in conjunction with the mechanical fixing device just below the ball forming machine ball disc body, and has the advantages of simple design, low manufacturing cost and high reliability.

[0121] The bidirectional bucket 6 includes a bucket body 601 and a screen 602 .

[0122] Both sides of the bucket body 601 are provided with feed ports 603, the bottom of the bucket body 601 is provided with a through opening 604, and a screen 602 is arranged in the through opening 604. The screen 602 is used for screening the over-standard spheres in practical applications, and a screen with a suitable sieve hole size can be selected and replaced according to the size of the over-standard spheres.

[0123] After the ball forming machine forms an oversized sphere during operation, the oversized sphere usually swings in a relatively fixed small area of ​​the ball making disc, but will not approach the disc wall. The traditional one-way bucket can only dig from a single direction during digging, which may cause the bucket to collide with the disc wall, resulting in mechanical damage. Compared with the traditional bucket, the bidirectional bucket 6 of the present invention uses openings in two directions to replace the traditional one opening direction. It can be used in conjunction with a mechanical arm, a connector and a hydraulic device to achieve digging in both left and right directions, thereby effectively avoiding the collision problem between the one-way bucket and the disc wall. At the same time, when the bidirectional bucket 6 digs the oversized sphere, it will also dig up some normal particles and mineral powder. The design of the screen 602 can quickly screen out the normal particles and mineral powder during digging, leaving only the oversized spheres, and because the screen is detachable, it can be replaced with a screen of appropriate aperture according to the size of the proposed oversized sphere.

[0124] A method for identifying, positioning and sorting mineral balls for a ball forming machine is also provided, comprising the following steps:

[0125] The computer obtains the image data obtained when the camera shoots the running environment of the mineral balls in the ball forming machine, sends brightness adjustment requirement information according to the image data, identifies and locates the balls exceeding the standard in the image data, determines the running trajectory of the balls exceeding the standard, and controls the mechanical components to grab the balls exceeding the standard according to the running trajectory;

[0126] According to the brightness adjustment requirement information, adjust the brightness of the light source provided for the ball ball operating environment in the ball ball forming machine. The light source component is used to adjust the brightness of the light source provided by the local end for the ball ball operating environment in the ball ball forming machine according to the brightness adjustment requirement information sent by the computer. Specifically, the light source component is composed of a visible light source and an infrared light source.

[0127] The image data is a video stream with a frame rate greater than or equal to 15 Hz.

[0128] The process of identifying and locating the over-standard sphere in the image data, determining the running trajectory of the over-standard sphere, and controlling the mechanical component to grasp the over-standard sphere according to the running trajectory includes:

[0129] Performing contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image;

[0130] Based on the object recognition technology of deep neural network, the mineral balls in the enhanced image are distinguished and the diameter of the mineral balls is estimated, and the mineral balls with diameters exceeding the threshold are screened out as the exceeding-standard spheres;

[0131] Determine the target search area of ​​the next enhanced image frame that is adjacent in time sequence, and judge whether there is an over-standard sphere in the target search area. If so, continue to determine the next target search area. If not, expand the scope of the next target search area, and calculate the estimated coordinates of the over-standard sphere when the over-standard sphere is screened out;

[0132] The estimated coordinates of the out-of-standard sphere are converted into space-time coordinates in the world coordinate system of the material surface, and the space-time coordinates are sent to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the out-of-standard sphere.

[0133] The process of determining the target search area of ​​the next enhanced image frame that is adjacent in time sequence includes:

[0134] Determine the motion domain of the over-standard sphere in the enhanced image of the current frame, where the motion domain is an irregular two-dimensional region that conforms to the Gaussian distribution;

[0135] Determine a candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as the target search area of ​​the next frame enhanced image that is adjacent in time sequence;

[0136] When it is determined that there is no exceeding sphere in the target search area, the Gaussian radius of the candidate search area is enlarged to 1.5 times.

[0137] When the excess sphere is screened out, the process of calculating the estimated coordinates of the excess sphere includes:

[0138] According to the rectangular center position of the detection frame, the image coordinates of the over-standard sphere in the current enhanced image are obtained;

[0139] After the exceeding spheres are screened out, the time t required for the manipulator in the mechanical assembly to move to the target position from the start is determined;

[0140] According to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t, the estimated coordinates of the exceeding standard sphere at time t+1 are calculated.

[0141] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A mineral ball identification, positioning and sorting system for a ball forming machine, characterized in that: include: Camera, used to shoot the running environment of the mineral balls in the ball forming machine; A computer is used to obtain image data collected by the camera, wherein the image data is a video stream with a frame rate greater than or equal to 15 Hz; issue brightness adjustment requirement information according to the image data, identify and locate the excessive sphere in the image data, determine the running trajectory of the excessive sphere, and control the mechanical component to grab the excessive sphere according to the running trajectory; The light source assembly is composed of a visible light source and an infrared light source, and is used to adjust the brightness of the light source provided by the local end for the operating environment of the mineral balls in the ball forming machine according to the brightness adjustment requirement information sent by the computer; The computer comprises: An image enhancement module, used for performing contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image; A target recognition module is used to identify the mineral balls in the enhanced image and estimate the diameter of the mineral balls based on the object recognition technology of the deep neural network, and screen out the mineral balls with diameters exceeding the threshold as the exceeding-standard spheres; The target tracking module is used to determine the target search area of ​​the next enhanced image frame that is adjacent in time sequence, and to determine whether there is an over-standard sphere in the target search area. If so, the next target search area is determined; if not, the next target search area is expanded, and when an over-standard sphere is screened out, the estimated coordinates of the over-standard sphere are calculated; The result output module is used to convert the estimated coordinates of the out-of-standard sphere into the space-time coordinates in the world coordinate system of the material surface, and send the space-time coordinates to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the out-of-standard sphere.

2. The mineral ball identification, positioning and sorting system for a ball forming machine according to claim 1 is characterized in that: The target tracking module comprises: A motion domain determination unit, used to determine the motion domain of the over-standard sphere in the enhanced image of the current frame, wherein the motion domain is an irregular two-dimensional region conforming to the Gaussian distribution; A candidate search area determination unit is used to determine a candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as a target search area of ​​the next frame enhanced image that is adjacent in time sequence; The range adjustment unit is used to expand the Gaussian radius of the candidate search area to 1.5 times when it is determined that there is no exceeding sphere in the target search area.

3. The mineral ball identification, positioning and sorting system for a ball forming machine according to claim 2 is characterized in that: The target tracking module also includes: An image coordinate determination unit, used to obtain the image coordinates of the over-standard sphere in the current enhanced image according to the rectangular center position of the detection frame; A moving time determination unit is used to determine the time t required for the manipulator in the mechanical assembly to move from the start to the target position after the exceeding spheres are screened out; The estimated coordinate determination unit is used to calculate the estimated coordinates of the exceeding standard sphere at time t+1 according to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t.

4. The mineral ball identification, positioning and sorting system for a ball forming machine according to claim 3 is characterized in that: Also includes: The communication link is used to provide a communication channel between the computer and other computer systems.

5. The mineral ball identification, positioning and sorting system for a ball forming machine according to claim 4 is characterized in that: The mechanical assembly includes: a first hydraulic push rod, a second hydraulic push rod, and a horizontal turntable, a mechanical arm, a mechanical arm and a bidirectional bucket articulated in sequence; the fixed end of the first hydraulic push rod is connected to the mechanical arm, and the movable end of the first hydraulic push rod is connected to the mechanical arm; the fixed end of the second hydraulic push rod is connected to the mechanical arm, and the movable end of the second hydraulic push rod is connected to the bidirectional bucket through a bucket connecting rod; the bidirectional bucket includes: a bucket body and a screen, both sides of the bucket body are provided with feed ports, the bottom of the bucket body is provided with a through opening, and the screen is arranged in the through opening.

6. A sorting method based on a mineral ball identification, positioning and sorting system for a ball forming machine according to any one of claims 1 to 5, characterized in that: include: Acquire image data obtained when a camera shoots the operating environment of the mineral balls in the ball forming machine, issue brightness adjustment requirement information according to the image data, identify and locate the balls exceeding the standard in the image data, determine the running track of the balls exceeding the standard, and control the mechanical components to grab the balls exceeding the standard according to the running track; According to the brightness adjustment requirement information, the brightness of the light source provided for the mineral ball operating environment in the ball forming machine is adjusted; The image data is a video stream with a frame rate greater than or equal to 15 Hz; The process of identifying and locating the over-standard sphere in the image data, determining the running trajectory of the over-standard sphere, and controlling the mechanical component to grasp the over-standard sphere according to the running trajectory includes: Performing contrast enhancement processing on each frame of real-time image in the video stream to obtain an enhanced image; Based on the object recognition technology of deep neural network, the mineral balls in the enhanced image are distinguished and the diameter of the mineral balls is estimated, and the mineral balls with diameters exceeding the threshold are screened out as the exceeding-standard spheres; Determine the target search area of ​​the next enhanced image frame that is adjacent in time sequence, and judge whether there is an over-standard sphere in the target search area. If so, continue to determine the next target search area. If not, expand the scope of the next target search area, and calculate the estimated coordinates of the over-standard sphere when the over-standard sphere is screened out; The estimated coordinates of the out-of-standard sphere are converted into space-time coordinates in the world coordinate system of the material surface, and the space-time coordinates are sent to the mechanical component to control the manipulator in the mechanical component to move to the space-time coordinates to grab the out-of-standard sphere.

7. The sorting method according to claim 6, characterized in that: The process of determining the target search area of ​​the next enhanced image frame that is adjacent in time sequence includes: Determine the motion domain of the over-standard sphere in the enhanced image of the current frame, where the motion domain is an irregular two-dimensional region that conforms to the Gaussian distribution; Determine a candidate search area according to the motion domain of the over-standard sphere in the current frame enhanced image and the rotation angular velocity of the ball forming machine, and use the determined candidate search area as the target search area of ​​the next frame enhanced image that is adjacent in time sequence; When it is determined that there is no exceeding sphere in the target search area, the Gaussian radius of the candidate search area is enlarged to 1.5 times.

8. The sorting method according to claim 7, characterized in that: The process of calculating the estimated coordinates of the spheres exceeding the standard when the spheres exceeding the standard are screened out includes: According to the rectangular center position of the detection frame, the image coordinates of the over-standard sphere in the current enhanced image are obtained; After the exceeding spheres are screened out, the time t required for the manipulator in the mechanical assembly to move to the target position from the start is determined; According to the moving distance and moving direction of the exceeding standard sphere in the X and Y directions from time t-1 to time t, the estimated coordinates of the exceeding standard sphere at time t+1 are calculated.

Images