Automatic unloading device for lower layer of belt conveyor and intelligent control method

Abstract

The invention relates to the technical field of conveyors, in particular to a lower-layer automatic unloading device of a belt conveyor and an intelligent control method. The method comprises the steps that real-time collection is conducted through a multi-mode sensor array, corresponding sensor data are formed, and according to material RGB images collected in real time, corresponding materials in the material RGB images are recognized; outputting material identification information corresponding to the material, and performing spatial alignment on each contour key point and the point cloud data; extracting a point cloud corresponding to the material from the aligned point cloud data, constructing a corresponding material three-dimensional model, and calculating a material volume corresponding to the material three-dimensional model; constructing a corresponding space-time diagram, and generating a grabbing priority queue; generating grabbing path point coordinate data corresponding to the materials according to the material information corresponding to the materials in the generated grabbing priority queue; and according to the grabbing path point coordinate data, a clamping device of the parallel robot is controlled to grab the materials on the conveyor.

Description

Technical Field

[0001] The present invention relates to the technical field of conveyors, and in particular to an automatic unloading device for the lower layer of a belt conveyor and an intelligent control method. Background Art

[0002] In the field of mine production, the transportation of production auxiliary materials for the excavation team is crucial and accounts for a large proportion of the overall work. At present, double-layer bidirectional belt conveyors are often used to transport production auxiliary materials on the lower layer. However, this transportation method has many problems that need to be solved.

[0003] On the one hand, double-layer belt conveyors have the characteristics of long transportation distance and high speed. In addition, the space in coal mines is limited, and manual unloading is very likely to cause material stacking. Material stacking will not only seriously affect the normal operation of the belt conveyor and reduce transportation efficiency, but also increase the risk of equipment failure. At the same time, during manual unloading, workers face a high risk of being caught in the belt conveyor, posing a major threat to their lives and safety, which undoubtedly increases the project risk. In addition, manual unloading requires a lot of manpower, resulting in high transportation costs.

[0004] On the other hand, although the traditional mechanical structure unloading method can realize the unloading function to a certain extent, it will cause additional wear on the conveyor belt. Frequent wear will significantly shorten the service life of the conveyor belt, thereby increasing the equipment maintenance cost, including the cost of conveyor belt replacement and maintenance labor costs, etc., and will also affect the normal production progress due to equipment downtime and maintenance, causing indirect economic losses.

[0005] Based on this, there is an urgent need for an automatic unloading device for the lower layer of a belt conveyor and an intelligent control method, which can realize efficient, safe and accurate unloading on the belt conveyor, and greatly provide intelligence, automation and reliability of unloading. Summary of the invention

[0006] One of the purposes of the present invention is to provide an automatic unloading device for the lower layer of a belt conveyor and an intelligent control method, which can realize efficient, safe and accurate unloading on the belt conveyor, and greatly improve the intelligence, automation and reliability of unloading.

[0007] In order to achieve the above object, a belt conveyor lower layer automatic unloading intelligent control method is provided, comprising the following steps: S1. Real-time acquisition is performed by a multimodal sensor array arranged on a belt conveyor to form corresponding sensor data, wherein the sensor data includes RGB image data of materials collected by an industrial camera and point cloud data collected by a laser radar; S2. Based on the real-time collected RGB images of the materials, identify the corresponding materials in the RGB images of the materials in the preset improved YOLOv7-tiny model, and output the material identification information corresponding to the materials. The material identification information includes a material bounding box, a material type, and several contour key points; S3. According to the collected point cloud data and several contour key points in the material identification information, spatially align each contour key point with the point cloud data; S4. According to the material bounding box, extract the point cloud corresponding to the material from the aligned point cloud data, construct a corresponding 3D model of the material, and calculate the volume of the material corresponding to the 3D model of the material based on the constructed 3D model of the material; S5. Construct a corresponding spatio-temporal graph according to the material identification information and the material volume , the node features of the spatio-temporal graph include position , speed , material volume, material type; the calculation formula for the edge weights of the spatio-temporal graph is:

[0008] In the formula, is a weight coefficient that can be adjusted through experiments to balance the influence of different factors on the edge weights; is the Euclidean norm of the difference between the feature vectors of nodes i and j, measuring the degree of difference between the features of two nodes, is the material compatibility coefficient, is the cosine similarity of the feature vectors of nodes i and j; S6. Based on the node features of the spatio-temporal graph and a preset grasping priority strategy, generate a grasping priority queue; S7. According to the material information corresponding to a certain material in the generated grasping priority queue, generate the coordinate data of the grasping path points corresponding to the material based on a preset grasping path prediction strategy; S8. According to the coordinate data of the grasping path points corresponding to a certain material, control the clamping device of the parallel robot to grasp the material on the conveyor based on a preset material grasping strategy.

[0009] Technical principle and effect of this solution: The RGB image data and point cloud data of the material are collected in real time through a multi-modal sensor array (industrial camera and lidar) installed on the belt conveyor. The industrial camera obtains the visual appearance information of the material, and the lidar obtains the spatial position and depth information of the material. The two types of data complement each other, providing rich raw data for subsequent processing. The preset improved YOLOv7-tiny model is used to process the RGB image of the material to identify the bounding box of the material (determine the position and range of the material in the image), the material type (distinguish different types of materials), and several contour key points (used to describe the shape characteristics of the material). The YOLOv7-tiny model is an object detection model that can quickly and accurately identify target objects in images.

[0010] Align the information obtained from the point cloud data with the contour key points in the material recognition information in space. This step is to fuse the image information and point cloud information in the same spatial coordinate system, enabling more accurate analysis based on the fused data in the subsequent steps.

[0011] Extract the point cloud corresponding to the material from the aligned point cloud data according to the material bounding box, and then construct a three-dimensional model of the material. Through the three-dimensional model, the volume of the material can be calculated, providing important physical parameters for subsequent grasping decisions.

[0012] Construct a spatio-temporal graph based on the material recognition information (position, material type, etc.) and the material volume. The node features in the graph include information such as the position, speed, volume, and material type of the material. The calculation formula of the edge weight comprehensively considers the degree of difference in node features, material compatibility, and cosine similarity of feature vectors. By adjusting the weight coefficient to balance the influence of different factors. Such a spatio-temporal graph can comprehensively describe the relationships and characteristics between materials.

[0013] Generate a grasping priority queue based on the node features of the spatio-temporal graph and the preset grasping priority strategy. This strategy determines the order of material grasping according to various features of the material (such as position, volume, material, etc.) to facilitate more efficient discharging operations.

[0014] Based on the material information and volume of a certain material in the grasping priority queue, generate the coordinate data of the grasping path points corresponding to the material based on the preset grasping path prediction strategy. This step provides specific motion path guidance for the parallel robot.

[0015] Based on the generated coordinate data of the grasping path points, control the parallel robot to grasp the material on the conveyor according to the preset material grasping strategy. The parallel robot features high precision and high speed, enabling accurate grasping of materials and realizing automatic discharging.

[0016] Traditional unloading methods often lead to grasping errors due to deviations in the judgment of the position and shape of materials. However, this control method constructs a three-dimensional information network of materials by using the RGB images and point cloud data collected by a multi-modal sensor array. The improved YOLOv7-tiny model accurately identifies the bounding box, material, and contour key points of the material, and then perfectly integrates the two-dimensional image information and three-dimensional space information of the material through the spatial alignment of the point cloud data. This collaborative processing of multi-source data enables the parallel robot to accurately locate the material, and its grasping error can be controlled within a very small range. Compared with the traditional method, the unloading accuracy is greatly improved, effectively avoiding problems such as material spillage and equipment damage caused by grasping errors, and ensuring the stable operation of the unloading process.

[0017] In the scenario of continuous operation of the belt conveyor, the unloading efficiency is crucial. By constructing a spatio-temporal graph and combining a grasping priority strategy, this method can dynamically plan the optimal grasping order based on multi-dimensional features such as the position, speed, volume, and material of the material. Materials that are close to the unloading port, have a small volume, or are easy to roll are processed first, avoiding time waste and congestion caused by improper grasping order.

[0018] Further, the S2 includes: S20. Input the real-time collected RGB image of the material into the backbone network of the improved YOLOv7-tiny model, and under the action of the convolutional layer of the backbone network, gradually generate a feature map ; S21. According to the generated feature map , based on a preset sparse factor calculation formula, calculate the sparse factor of the feature map ; The sparse factor calculation formula is:

[0019] In the formula, is the activation value of the k-th channel at the position (i, j) in the input feature map, H represents the height of the feature map F, W represents the width of the feature map F, C represents the number of channels of the feature map F, is the maximum value of all elements in the input feature map F, is the activation threshold, is the indicator function, which judges whether the condition holds. If , then return 1, otherwise return 0; S22. According to the calculated sparse factor of the feature map , based on a preset convolutional kernel structure switching strategy, adaptively switch the convolutional kernel structure in the improved YOLOv7-tiny model, and switch the convolutional kernel structure of the backbone network of the improved YOLOv7-tiny model in S20 to the convolutional kernel structure corresponding to this adaptive switch; The convolutional kernel structure switching strategy is as follows: When , use a complete 3×3 convolutional kernel to retain detailed features; When , decompose the 3×3 convolutional kernel into an asymmetric cascade structure.

[0020] Beneficial effects: In the automatic unloading scenario of a belt conveyor, real-time processing of a large number of material RGB images requires extremely high computing resources. Through the sparse factor calculation formula, the sparsity degree of the feature map can be quantified. When the sparse factor S≥Z, decompose the 3×3 convolutional kernel into an asymmetric cascade structure, significantly reducing the number of parameters and the amount of calculation in convolutional calculations. Taking the actual application scenario as an example, when processing high-resolution material images, this structure switch can reduce the amount of calculation in a single forward propagation by about 40%, effectively alleviating the hardware computing pressure. By dynamically adjusting the convolutional kernel structure, while maintaining the accuracy, the model significantly improves the computing speed and reduces the computing cost as the spatial resolution increases. When S≥Z, the asymmetric cascade structure can grasp the macroscopic features of the material as a whole while reducing the amount of calculation. For materials with regular shapes and single textures, such as bagged cement, it can efficiently extract their key contour and position information.

[0021] Furthermore, the preset grasping priority strategy is as follows: Step 1, initialize the key parameters corresponding to the parallel robot, and the key parameters include the starting point coordinates , the target point coordinates , the sampling space range, the maximum number of iterations, the step size, and the neighborhood radius, and create an empty node set and an edge set, and add the starting point coordinates as the root node to the node set; Step 2, within the sampling space range, generate a random point through a random number generator , and the corresponding coordinates are: ; Step 3, in the node set, based on the node distance calculation formula, calculate the node distances between each node in the node set and the random point, and select the node with the closest node distance ; The node distance calculation formula is:

[0022] In the formula, is the node and random points the node distance between; Step 4, according to the node closest to the selected node distance , from node to node the new node extended in the direction of , and based on the preset path cost calculation formula, calculate the estimated path cost corresponding to the new node ; ; Step 5, according to the extended new node and node , judge whether the connection line between the new node and node collides with the obstacle. If not, add the extended new node to the node set, and record the corresponding path cost , otherwise, re-execute Step 2; Step 6, centered on the extended new node , search for the neighborhood node set within the preset neighborhood radius , for each node in the neighborhood node set , calculate the path cost from the extended new node to node ; , and compare the path cost from the new node to node with the path cost from the parent node of node to . If , then update the parent node of node to , and at the same time update the path cost of node to , and update the corresponding edge in the edge set. Otherwise, the parent node and path cost of node remain unchanged; ; ; Step 7, when the target point is found, starting from the target point, backtrack to the starting point through the parent node to generate the coordinate data of the grasping path points .

[0023] Beneficial effects: Under the condition of continuous operation of the belt conveyor, quickly planning a reasonable grasping path is the key to improving the unloading efficiency. This strategy generates random points in the sampling space through a random number generator, and combines the node distance calculation to quickly determine the expansion direction. Compared with the traditional heuristic search algorithm, it reduces a large number of redundant search steps.

[0024] In step 5, this strategy determines the feasibility of path expansion by judging whether the connection line between the new node and the neighboring nodes collides with obstacles. Once the collision risk is detected, the path is immediately replanned, effectively avoiding the collision between the parallel robot and obstacles during the grasping process and reducing the probability of equipment damage.

[0025] By comparing and updating the path costs of neighborhood nodes in step 6, this strategy can continuously optimize the path to ensure that the generated initial path is a local optimal solution. Compared with simple shortest path planning, it comprehensively considers factors such as path length and turning angle, making the movement of the parallel robot smoother during the grasping process and reducing frequent acceleration, deceleration, and sharp turns.

[0026] Furthermore, the preset material grasping strategy is as follows: When determining the coordinate data of the grasping path points corresponding to a certain material, based on the preset material mass calculation formula, the estimated mass corresponding to the material is calculated, and the control motor drive current of the parallel robot is optimized based on the estimated mass; The material mass calculation formula is:

[0027] In the formula, is the volume of the material, is the estimated mass of the material, is the material density corresponding to the material, is the number of sampling points corresponding to calculating the material volume, is the coordinate of the i-th sampling point, is the coordinate of the origin, is the normal vector at the i-th sampling point, is the infinitesimal area corresponding to the i-th sampling point; After determining the estimated mass of the material, when the clamping device contacts the material, the six-axis force sensor on the clamping device real-time feedbacks the contact force corresponding to the contact with the material, and adjusts the grasping posture of the clamping device based on the preset impedance control model.

[0028] Beneficial effects: Through the preset material quality calculation formula, the quality can be accurately estimated based on the material volume and the corresponding material density. Based on this estimated quality, the control motor drive current of the parallel robot is optimized, enabling the motor output power to precisely match the material weight. This avoids grasping failures caused by insufficient power or energy waste and equipment wear caused by excessive power, improving the stability and reliability of the grasping operation. This strategy organically combines quality estimation, motor control, and force feedback control, realizing the automation and intelligence of the material grasping process. Without frequent manual intervention and adjustment, it can automatically optimize the operation parameters according to the actual situation of the material, improving the efficiency and quality of the entire grasping operation and enhancing the autonomy and adaptability of the system.

[0029] The present invention also provides an automatic unloading device for the lower layer of a belt conveyor, using the above-mentioned intelligent control method for automatic unloading of the lower layer of a belt conveyor, including a server end and a unloading end; The unloading end includes a support base, the support base includes two H-shaped support parts, a parallel robot installation platform is arranged between the two H-shaped support parts, a parallel robot is arranged on the lower end surface of the parallel robot installation platform, and a clamping device for clamping the material is arranged on the parallel robot; It also includes an industrial camera and a lidar for real-time monitoring of the materials on the conveyor; The server end includes: A data acquisition module, which is used for real-time acquisition through a multi-modal sensor array arranged on the belt conveyor to form corresponding sensor data. The sensor data includes the material RGB image data collected by the industrial camera and the point cloud data collected by the lidar; A material recognition module, which is used for recognizing the material corresponding to the material RGB image based on the preset improved YOLOv7-tiny model according to the real-time collected material RGB image, and outputting the material recognition information corresponding to the material. The material recognition information includes a material bounding box, a material type, and a number of contour key points; A preprocessing module, which is used for spatially aligning each contour key point with the point cloud data according to the collected point cloud data and a number of contour key points in the material recognition information; A volume calculation module, which is used for extracting the point cloud corresponding to the material from the aligned point cloud data according to the material bounding box, constructing a corresponding three-dimensional model of the material, and calculating the volume of the material corresponding to the three-dimensional model of the material; A construction module, which is used for constructing a corresponding spatio-temporal graph according to the material recognition information and the material volume , the node features of the spatio-temporal graph include position and speed , the volume and material type of the material; the calculation formula for the edge weight of the spatio-temporal graph is:

[0030] In the formula, is the weight coefficient, which can be adjusted through experiments to balance the influence of different factors on the edge weight; is the Euclidean norm of the difference between the feature vectors of nodes i and j, measuring the degree of difference between the features of the two nodes, is the material compatibility coefficient, is the cosine similarity of the feature vectors of nodes i and j; Queue priority module, used to generate a grasping priority queue based on the node features of the spatio-temporal graph and a preset grasping priority strategy; Coordinate determination module, used to generate the coordinate data of the grasping path points corresponding to a certain material based on the material information corresponding to the material in the generated grasping priority queue and a preset grasping path prediction strategy; Grasping control module, used to control the clamping device of the parallel robot to grasp the material on the conveyor according to the coordinate data of the grasping path points corresponding to a certain material and a preset material grasping strategy.

[0031] The technical principle and effect of this solution: The industrial camera continuously captures the RGB images of the materials on the conveyor, and uses image recognition technology to obtain information such as the appearance features, positions, and shapes of the materials. The lidar emits laser beams and receives the reflected signals to generate point cloud data, which is used to accurately determine the three-dimensional spatial positions, volumes, etc. of the materials. The combination of the two can comprehensively obtain the material state data. After receiving the sensor data transmitted by the industrial camera and the lidar, the server analyzes and processes the data. Through specific algorithms (such as using object detection algorithms to process image data and point cloud processing algorithms to process lidar data), it identifies information such as the material type, position, and posture, and then generates control commands according to the preset grasping strategies (such as planning the grasping path according to the material position and determining the clamping force according to the material quality and characteristics). The parallel robot is installed on the lower end face of the parallel robot installation platform, and it has multiple degrees of freedom and can move flexibly. When receiving the control commands sent by the server, the parallel robot controls the movement of its respective joints according to the commands, drives the clamping device to reach the specified position, and clamps the material according to the set posture and force, thus realizing the unloading function. The H-shaped support part and the cross bar of the support base provide stable support for the parallel robot and related equipment.

[0032] Industrial cameras and lidar continuously collect material information in real time. Once a material reaches the discharge position, it can quickly capture and transmit data to the server. The server rapidly analyzes and processes the data, generates control instructions within a short time based on efficient algorithms, and sends them to the parallel robot. Compared with manual observation and operation, it greatly shortens the time interval from material discovery to the start of discharging, enables the discharging operation to be carried out more promptly, reduces the unnecessary residence time of materials on the conveyor belt, and speeds up the overall conveying process.

[0033] The parallel robot has multiple degrees of freedom, is flexible and fast in movement, and can quickly move to the material according to the preset path for grasping and discharging. Its automated operation process has strong coherence and can continuously process materials, avoiding factors such as fatigue and pauses that may affect efficiency in manual operation, thereby greatly increasing the discharging volume per unit time and improving the working efficiency of the belt conveyor. That is, it can achieve efficient, safe and accurate discharging on the belt conveyor, greatly improving the intelligence, automation and reliability of discharging.

[0034] Traditional discharging methods often result in grasping errors due to judgment deviations in the position and shape of materials. This control method constructs a three-dimensional information network of materials by using RGB images and point cloud data collected by a multi-modal sensor array. The improved YOLOv7-tiny model accurately identifies the material bounding box, material and contour key points, and then through the spatial alignment of the point cloud data, perfectly integrates the two-dimensional image information and three-dimensional space information of the material. This collaborative processing of multi-source data enables the parallel robot to accurately locate the material, and its grasping error can be controlled within a very small range. Compared with the traditional method, the discharging accuracy is greatly improved, effectively avoiding problems such as material spillage and equipment damage caused by grasping errors, and ensuring the stable operation of the discharging process.

[0035] In the scenario of continuous operation of the belt conveyor, discharging efficiency is crucial. This method can dynamically plan the optimal grasping order based on multi-dimensional features such as the position, speed, volume, and material of the material by constructing a spatio-temporal graph and combining a grasping priority strategy. Materials that are close to the discharge port, have a small volume or are easy to roll are processed first, avoiding time waste and congestion caused by improper grasping order. Brief Description of the Drawings

[0036] Figure 1 It is a schematic diagram of the oblique view structure of the automatic discharging device for the lower layer of the belt conveyor in the first embodiment of the present invention; Figure 2 It is a schematic diagram of the bottom view structure of the automatic discharging device for the lower layer of the belt conveyor in the first embodiment of the present invention; Figure 3 It is a schematic diagram of the front view structure of the automatic discharging device for the lower layer of the belt conveyor in the first embodiment of the present invention; Figure 4It is the logic block diagram of the automatic unloading device for the lower layer of the belt conveyor in Embodiment 1 of the present invention; Figure 5 It is the flowchart of the intelligent control method for the automatic unloading of the lower layer of the belt conveyor in Embodiment 1 of the present invention. Specific embodiments

[0037] The following is a more detailed description through specific embodiments: The marks in the attached drawings of the specification include: parallel robot installation platform 1, channel steel 2, support base 3, pin 4, split pin 5, parallel robot platform cross beam 6, upper connecting rod 7, first ball hinge 8, lower connecting rod 9, second ball hinge 10, clamping device 11, servo motor 12, coupling 13, rotating shaft bearing seat 14, guide rail 15, rotating shaft 16.

[0038] Embodiment 1 The automatic unloading device for the lower layer of the belt conveyor is basically as Figure 1 、 Figure 2 、 Figure 3 shown, including a server end and a discharging end; The discharging end includes a support base 3, and the support base 3 includes two H-shaped support parts; in this embodiment, the two H-shaped support parts are respectively arranged on both sides of the conveyor. Two channel steels 2 are placed in parallel under the belt conveyor, and the lower end surfaces of the H-shaped support parts are welded to the two channel steels 2. An opening pin 5 is arranged in the middle of the H-shaped support part, and the H-shaped support part is fixed on the longitudinal beam of the conveyor through a pin 4. The parallel robot installation platform 1 is fixed between the two H-shaped support parts through the parallel robot platform cross beam 6.

[0039] A parallel robot installation platform 1 is arranged on the transverse rod. A parallel robot is arranged on the lower end surface of the parallel robot installation platform 1, and a clamping device 11 for clamping materials is arranged on the parallel robot; in this embodiment, four groups of symmetrically arranged servo motors 12 are installed on the parallel robot. The rotating shaft 16 on each group of motors is connected to the carbon fiber upper connecting rod 7 through an elastic coupling 13, and the end of the upper connecting rod 7 is embedded in the magnetorheological damper array. The rotating shaft 16 rotates under the support of the rotating shaft bearing seat 14, and transmits power to the corresponding components to realize motion control. The rotating shaft bearing seat 14 is arranged on the guide rail 15.

[0040] The driving mechanism is connected to the modular clamping mechanism through the ball hinge of the first ball hinge 8 to ensure the flexibility of six-degree-of-freedom movement. The corresponding modular clamping mechanism includes a lower connecting rod 9 connected to the first ball hinge 8, and the lower connecting rod 9 is connected to the clamping device 11 through a second ball hinge 10.

[0041] It also includes an industrial camera and a lidar for real-time monitoring of the materials on the conveyor; in this embodiment, the resolution of the industrial camera is 1920×1080, the frame rate is 60fps, it is installed 1.5m directly above the conveyor belt, and the downward viewing angle is 45°; the lidar has a scanning frequency of 20Hz, the accuracy is between -2mm and 2mm, and it is integrated with the multispectral sensor covering the 400 - 2500nm band on the same bracket, 0.8m away from the conveyor belt surface. Through a checkerboard calibration board with 10×10 squares and a single square side length of 50mm, multi-sensor spatial calibration is carried out to align the RGB image of the material and the point cloud data to ensure that the spatial alignment accuracy ≤ 3mm.

[0042] The server is used to receive the sensor data collected by the industrial camera and the lidar, and send control instructions to the parallel robot to control the clamping device 11 of the parallel robot to clamp the material.

[0043] As Figure 4 shown, the server includes: A data acquisition module, which is used to perform real-time acquisition through the multi-modal sensor array set on the belt conveyor to form corresponding sensor data. The sensor data includes the RGB image data of the material collected by the industrial camera and the point cloud data collected by the lidar; A material recognition module, which is used to identify the material corresponding to the RGB image of the material based on the preset improved YOLOv7-tiny model according to the real-time collected RGB image of the material, and output the material recognition information corresponding to the material. The material recognition information includes the material bounding box, the material type, and several contour key points; A preprocessing module, which is used to spatially align each contour key point with the point cloud data according to the collected point cloud data and several contour key points in the material recognition information; A volume calculation module, which is used to extract the point cloud corresponding to the material from the aligned point cloud data according to the material bounding box, construct a corresponding three-dimensional model of the material, and calculate the volume of the material corresponding to the three-dimensional model of the material based on the constructed three-dimensional model of the material; A construction module, which is used to construct a corresponding spatio-temporal graph according to the material recognition information and the material volume , the node features of the spatio-temporal graph include position , speed , material volume, material type; the calculation formula for the edge weight of the spatio-temporal graph is:

[0044] In the formula, is the weight coefficient, which can be adjusted through experiments to balance the influence of different factors on the edge weight; is the Euclidean norm of the difference between the feature vectors of node i and node j, measuring the degree of difference between the features of two nodes, is the material compatibility coefficient, is the cosine similarity of the feature vectors of node i and node j; Queue priority module, used to generate a grasping priority queue based on the node features of the spatio-temporal graph according to a preset grasping priority strategy; Coordinate determination module, used to generate the coordinate data of the grasping path points corresponding to a certain material based on the material information corresponding to the material in the generated grasping priority queue according to a preset grasping path prediction strategy; Grasping control module, used to control the clamping device of the parallel robot to grasp the material on the conveyor according to the coordinate data of the grasping path points corresponding to a certain material based on a preset material grasping strategy.

[0045] As Figure 5 shown, this embodiment also discloses an intelligent control method for automatic unloading of the lower layer of a belt conveyor, including the following steps: S1. Real-time collection is carried out through a multi-modal sensor array arranged on the belt conveyor to form corresponding sensor data, and the sensor data includes the material RGB image data collected by an industrial camera and the point cloud data collected by a lidar; S2. Based on the real-time collected material RGB image, in a preset improved YOLOv7-tiny model, the material corresponding to the material RGB image is identified, and the material identification information corresponding to the material is output. The material identification information includes a material bounding box, a material type, and several contour key points; during the training process of the model, the training data set contains 100,000 underground scene images, and data augmentation adopts dust atomization simulation and random brightness jitter. The confidence level of the output material ≥0.8, the material bounding box, the material classification is metal, wood, plastic, and 5 contour key points.

[0046] The S2 includes: S20. Input the real-time collected material RGB image into the backbone network of the improved YOLOv7-tiny model, and under the action of the convolutional layer of the backbone network, gradually generate a feature map ; S21. According to the generated feature map , based on a preset sparse factor calculation formula, calculate the sparse factor of the feature map ; the sparse factor calculation formula is:

[0047] In the formula, is the activation value of the k-th channel at the position (i, j) in the input feature map. H represents the height of the feature map F, W represents the width of the feature map F, and C represents the number of channels of the feature map F. is the maximum value of all elements in the input feature map F. is the activation threshold. is the indicator function to judge whether the condition holds. If , it returns 1, otherwise it returns 0. S22. Calculate the sparsity factor of the feature map . Based on the preset convolutional kernel structure switching strategy, adaptively switch the convolutional kernel structure in the improved YOLOv7-tiny model, and switch the convolutional kernel structure of the backbone network of the improved YOLOv7-tiny model in S20 to the convolutional kernel structure corresponding to this adaptive switch. The convolutional kernel structure switching strategy is as follows: When , use a complete 3×3 convolutional kernel to retain detailed features. When , decompose the 3×3 convolutional kernel into an asymmetric cascade structure.

[0048] S3. According to the collected point cloud data and several contour key points in the material recognition information, spatially align each contour key point with the point cloud data. S4. According to the material bounding box, extract the point cloud corresponding to the material from the aligned point cloud data, construct the corresponding three-dimensional model of the material, and calculate the volume of the material corresponding to the three-dimensional model of the material based on the constructed three-dimensional model of the material. S5. According to the material recognition information and the material volume, construct the corresponding spatio-temporal graph . The node features of the spatio-temporal graph include the position , the velocity , the material volume, and the material type. The calculation formula for the edge weight of the spatio-temporal graph is:

[0049] In the formula, is the weight coefficient, which can be adjusted through experiments to balance the influence of different factors on the edge weight. is the Euclidean norm of the difference between the feature vectors of nodes i and j, measuring the degree of difference between the features of two nodes. is the material compatibility coefficient. is the cosine similarity of the feature vectors of nodes i and j. S6. Generate a grasping priority queue based on the node characteristics of the spatio-temporal graph and a pre-set grasping priority strategy; use a Graph Attention Network (GAT) to iteratively update the node embeddings to generate the grasping priority queue Q. The input layer dimension of the Graph Attention Network is 12, including position, velocity, volume, and material. The activation function of the hidden layer is LeakyReLU, and the output is normalized by Softmax to generate the grasping priority weights.

[0050] S7. Based on the material information corresponding to a certain material in the generated grasping priority queue, generate the coordinate data of the grasping path points corresponding to the material based on a pre-set grasping path prediction strategy. The pre-set grasping priority strategy is as follows: Step 1. Initialize the key parameters corresponding to the parallel robot. The key parameters include the starting point coordinates , the target point coordinates , the sampling space range, the maximum number of iterations, the step size, and the neighborhood radius, and create an empty node set and an edge set, and add the starting point coordinates as the root node to the node set. Step 2. Generate a random point within the sampling space range through a random number generator , and the corresponding coordinates are: ; Step 3. In the node set, calculate the node distances between each node in the node set and the random point based on the node distance calculation formula, and select the node with the closest node distance ; The node distance calculation formula is:

[0051] In the formula, is the node distance between node and the random point ; Step 4. According to the selected node with the closest node distance , expand a new node from node in the direction of node , and calculate the estimated path cost corresponding to the new node ; Step 5. According to the expanded new node and node , judge whether the connection line between the new node and node collides with an obstacle. If not, add the expanded new node to the node set and record the corresponding path cost Otherwise, re-execute step 2; Step 6: Expand the new node Centered on the preset neighborhood radius, search for a set of neighborhood nodes within the neighborhood radius , for each node in the neighborhood node set , calculate the new node from the expansion To Node The path cost , and the new node To Node The path cost With Node The parent node to The path cost For comparison, if , then update the node The parent node is , while updating the node The path cost is , and update the corresponding edge in the edge set. Otherwise, the node The parent node and path cost of remain unchanged; Step 7: When the target point is found, start from the target point and trace back to the starting point through the parent node to generate the coordinate data of the grabbing path point In this embodiment, after obtaining the coordinate data of the grabbing path point, the LSTM network is used to predict the future The material displacement within the path can dynamically correct the coordinates of the path points.

[0052] S8. According to the coordinate data of the grabbing path points corresponding to a certain material and based on the preset material grabbing strategy, the clamping device 11 of the parallel robot is controlled to grab the material on the conveyor.

[0053] The preset material grabbing strategy is: When determining the coordinate data of the grasping path point corresponding to a certain material, the estimated mass corresponding to the material is calculated based on the preset material mass calculation formula, and the driving current of the servo motor 12 of the parallel robot is optimized based on the estimated mass; The material mass calculation formula is:

[0054] In the formula, is the volume of the material, is the estimated mass of the material, is the material density corresponding to the material, To calculate the number of sampling points corresponding to the material volume, is the coordinate of the i-th sampling point, is the coordinate of the origin, is the normal vector at the i-th sampling point, is the infinitesimal area corresponding to the i-th sampling point; in this embodiment, the material density is determined through a preset density database, such as the density of coal mine filling materials , the density of mechanical equipment materials , the density of transportation equipment materials , the density of lubricating oil emulsion for query. Of course, when the corresponding material density cannot be queried, the actual density is calculated by the moment balance equation.

[0055] After determining the estimated mass of the material, when the clamping device 11 contacts the material, the six-axis force sensor on the clamping device 11 real-time feeds back the contact force corresponding to the contact with the material , and based on the preset impedance control model, adjusts the grasping posture of the clamping device 11.

[0056] The above are only the embodiments of the present invention. The specific structures and characteristics and other common knowledge in the solution are described in too much detail here. Those of ordinary skill in the art know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can know all the existing technologies in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become an obstacle for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can still be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be based on the content of its claims, and the specific implementation manners and other records in the specification can be used to explain the content of the claims.

Claims

1. The intelligent control method for automatic discharging of the lower layer of a belt conveyor is characterized in that: It includes the following steps: S1. Perform real-time acquisition through a multi-modal sensor array arranged on a belt conveyor to form corresponding sensor data. The sensor data includes the material RGB image data collected by an industrial camera and the point cloud data collected by a lidar; S2. Based on the real-time acquired material RGB image, in a preset improved YOLOv7-tiny model, identify the material corresponding to the material RGB image and output the material identification information corresponding to the material. The material identification information includes a material bounding box, a material type, and several contour key points; S3. According to the collected point cloud data and several contour key points in the material identification information, spatially align each contour key point with the point cloud data; S4. According to the material bounding box, extract the point cloud corresponding to the material from the aligned point cloud data, construct a corresponding three-dimensional model of the material, and calculate the volume of the material corresponding to the three-dimensional model of the material based on the constructed three-dimensional model of the material; S5. Construct a corresponding spatio-temporal graph according to the material identification information and the material volume. , the node features of the spatio-temporal graph include the position , the speed , the material volume, and the material type; the calculation formula for the edge weight of the spatio-temporal graph is: In the formula, is the weight coefficient, which can be adjusted through experiments to balance the influence of different factors on the edge weight; is the Euclidean norm of the difference between the feature vectors of node i and node j, measuring the degree of difference between the features of two nodes, is the material compatibility coefficient, is the cosine similarity of the feature vectors of node i and node j; S6. Based on the node features of the spatio-temporal graph, generate a grasping priority queue according to a preset grasping priority strategy; S7. According to the material information corresponding to a certain material in the generated grasping priority queue, generate the coordinate data of the grasping path points corresponding to the material based on a preset grasping path prediction strategy; S8. According to the coordinate data of the grasping path points corresponding to a certain material, control the clamping device of the parallel robot to grasp the material on the conveyor based on a preset material grasping strategy.

2. The intelligent control method for automatic discharging of the lower layer of the belt conveyor according to claim 1, wherein: The S2 includes: S20. Input the real-time collected RGB image of the material into the backbone network of the improved YOLOv7-tiny model. Under the action of the convolutional layer of the backbone network, a feature map is gradually generated. ; S21. According to the generated feature map , based on a preset sparse factor calculation formula, calculate the sparse factor of the feature map ; the sparse factor calculation formula is as follows: ​ In the formula, is the activation value of the k-th channel at the position (i, j) in the input feature map. H represents the height of the feature map F, W represents the width of the feature map F, and C represents the number of channels of the feature map F. is the maximum value of all elements in the input feature map F. is the activation threshold. is the indicator function that determines whether the condition holds. If , it returns 1; otherwise, it returns 0. S22. According to the calculated feature map of the sparsity factor , based on the preset convolutional kernel structure switching strategy, adaptively switch the convolutional kernel structure in the improved YOLOv7-tiny model, and switch the convolutional kernel structure of the backbone network of the improved YOLOv7-tiny model in S20 to the convolutional kernel structure corresponding to this adaptive switch; The convolutional kernel structure switching strategy is: When , use a complete 3×3 convolutional kernel to retain detailed features; When , the 3×3 convolution kernel is decomposed into an asymmetric cascade structure.

3. The intelligent control method for automatic discharging of the lower layer of the belt conveyor according to claim 2, characterized in that: The preset grasping priority strategy is: Step 1, initialize the key parameters corresponding to the parallel robot, and the key parameters include the starting point coordinates , the target point coordinates , the sampling space range, the maximum number of iterations, the step size, and the neighborhood radius, and create an empty node set and an edge set, and add the starting point coordinates as the root node to the node set; Step 2, within the sampling space range, generate random points through a random number generator , with corresponding coordinates being: ; Step 3: In the node set, based on the node distance calculation formula, calculate the node distances between each node in the node set and the random point, and select the node with the closest node distance ; The node distance calculation formula is: In the formula, is the node and the random point the node distance between; Step 4, according to the node closest to the selected node , from node to node in the direction of the newly expanded node , and based on a preset path cost calculation formula, calculate the estimated path cost corresponding to the new node ; Step 5, according to the expanded new node and node , determine whether the connection between the new node and node collides with an obstacle. If there is no collision, add the expanded new node to the node set and record the corresponding path cost , otherwise, re - execute Step 2; Step 6: Expand the new node Centered on the preset neighborhood radius, search for a set of neighborhood nodes within the neighborhood radius , for each node in the neighborhood node set , calculate the new node from the expansion To Node The path cost , and the new node To Node The path cost With Node The parent node to The path cost For comparison, if , then update the node The parent node is , while updating the node The path cost is , and update the corresponding edge in the edge set. Otherwise, the node The parent node and path cost of remain unchanged; Step 7, when the target point is found, starting from the target point, backtrack to the starting point through the parent nodes to generate the coordinate data of the grasping path points .

4. The intelligent control method for automatic discharging of the lower layer of the belt conveyor according to claim 3, wherein: The preset material grasping strategy is: When determining the coordinate data of the grasping path points corresponding to a certain material, calculate the estimated mass corresponding to the material based on a preset material mass calculation formula, and optimize the driving current of the control motor of the parallel robot based on the estimated mass; The material mass calculation formula is: In the formula, is the volume of the material, is the estimated mass of the material, is the material density corresponding to the material, is the number of sampling points corresponding to the calculated material volume, is the coordinate of the i-th sampling point, is the coordinate of the origin, is the normal vector at the i-th sampling point, is the infinitesimal area corresponding to the i-th sampling point; After determining the estimated mass of the material, when the clamping device comes into contact with the material, the six-axis force sensor on the clamping device real-time feedbacks the contact force corresponding to the contact with the material , and based on the preset impedance control model, adjusts the grasping posture of the clamping device.

5. The lower-layer automatic discharging device of a belt conveyor, which uses the intelligent control method for the lower-layer automatic discharging of a belt conveyor according to any one of the above claims 1 to 4, is characterized in that: It includes a server end and a discharging end; The discharging end includes a support base. The support base includes two H-shaped support parts. A parallel robot installation platform is arranged between the two H-shaped support parts. A parallel robot is arranged on the lower end surface of the parallel robot installation platform. A clamping device for clamping the material is arranged on the parallel robot; It further includes an industrial camera and a lidar for real-time monitoring of the materials on the conveyor; The server end includes: A data acquisition module for performing real-time acquisition through a multi-modal sensor array arranged on a belt conveyor to form corresponding sensor data. The sensor data includes the material RGB image data collected by an industrial camera and the point cloud data collected by a lidar; A material identification module for identifying the material corresponding to the material RGB image based on a preset improved YOLOv7-tiny model according to the real-time acquired material RGB image and outputting the material identification information corresponding to the material. The material identification information includes a material bounding box, a material type, and several contour key points; A preprocessing module, which is used to spatially align each contour key point with the point cloud data according to the collected point cloud data and several contour key points in the material recognition information; A volume calculation module, which is used to extract the point cloud corresponding to the material from the aligned point cloud data according to the material bounding box, construct a corresponding three-dimensional model of the material, and calculate the volume of the material corresponding to the three-dimensional model of the material based on the constructed three-dimensional model of the material; Building block, used to construct a corresponding spatio-temporal graph according to material identification information and material volume , the node features of the spatio-temporal graph include position , speed , material volume, material type; The calculation formula for the edge weight of the spatio-temporal graph is: In the formula, is the weight coefficient, which can be adjusted through experiments to balance the influence of different factors on the edge weight; is the Euclidean norm of the difference between the feature vectors of node i and node j, measuring the degree of difference between the features of the two nodes, is the material compatibility coefficient, is the cosine similarity of the feature vectors of node i and node j; A queue priority module, which is used to generate a grasping priority queue based on the node features of the spatio-temporal graph and a preset grasping priority strategy; A coordinate determination module, which is used to generate the coordinate data of the grasping path points corresponding to a certain material based on the material information corresponding to the material in the generated grasping priority queue and a preset grasping path prediction strategy; A grasping control module, which is used to control the clamping device of the parallel robot to grasp the material on the conveyor according to the coordinate data of the grasping path points corresponding to a certain material and a preset material grasping strategy.

Images