Depth map-based automatic monitoring and correcting method for deviation of conveying belt

Abstract

The invention belongs to the technical field of belt conveyors, and particularly discloses an automatic monitoring and correcting method for deviation of a conveying belt based on a depth map, which automatically constructs a belt datum line through a visual unit, performs edge detection based on depth information, can directly output a millimeter-level deviation distance, has a definite engineering application value, and is suitable for popularization and application. Linkage with a control system is facilitated; the vision unit captures the deviation state of the conveying belt in real time and quantizes the deviation state into a digital signal; the PLC compares the detection signal with a set reference and generates a corresponding deviation correction instruction; the servo electric push rod accurately responds to the instruction to drive the aligning carrier roller to generate transverse deviation rectifying force, and the running track of the conveying belt is dynamically corrected to achieve deviation rectifying; a dynamic feedback adjusting mechanism is formed in the whole process until the conveying belt is recovered to be within the set allowable error range, and stable operation is kept.

Description

Technical Field

[0001] This invention relates to the field of belt conveyor technology, and in particular to an automatic monitoring and correction method for conveyor belt misalignment based on depth maps. Background Technology

[0002] Most existing conveyor belt correction systems use eccentric rollers to adjust the belt alignment. These rollers detect belt misalignment, and when misalignment is detected, the belt friction on the misaligned side of the roller increases, forcing the roller on the misaligned side to rotate around the roller's axis, thus moving the belt towards the center line of the conveyor and correcting the belt alignment.

[0003] Chinese patent application CN 120348636 A discloses an automatic conveyor belt alignment device, which includes a horizontally arranged frame for supporting the conveyor belt; a pressure detection unit including two rows of pressure detectors symmetrically arranged along the width direction of the conveyor belt; double-row inclined conveyor rollers symmetrically mounted on the frame via supports with rotational damping and in contact with the lower surface of the conveyor belt; an adjustable support assembly including a top block slidably disposed on the conveyor roller; and a control unit electrically connected to the pressure detection unit, including a comparison module and a signal output module; wherein, when the comparison module detects a pressure difference between the two sides of the same position on the conveyor belt, the signal output module sends a drive signal to the adjustable support assembly corresponding to the side with lower pressure, causing the top block to extend out of the surface of the conveyor roller.

[0004] However, existing conveyor belt correction systems use contact testing, which only works when the conveyor belt deviates to the sensor, making real-time correction impossible. Furthermore, multiple correction devices are required to achieve correction, resulting in low efficiency and high cost. If the conveyor belt keeps deviating to the sensor without making contact, the conveyor belt correction system cannot detect the deviation during long-term operation, causing the conveyor to operate in a deviated state for an extended period.

[0005] Therefore, how to effectively improve the correction efficiency of conveyors and ensure stable and reliable correction has become an urgent problem to be solved in this field. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an automatic monitoring and correction method for conveyor belt misalignment based on depth maps.

[0007] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0008] An automatic monitoring and correction method for conveyor belt misalignment based on depth maps includes the following steps:

[0009] S1. An industrial camera is fixed above the conveyor belt of the belt conveyor to capture images of the conveyor belt; a self-aligning idler is rotatably mounted on a self-aligning idler frame below the conveyor belt, and the bottom of the self-aligning idler frame is rotatably mounted on the frame of the belt conveyor via a rotating shaft. A servo electric push rod is fixed on one side of the belt conveyor frame, and the top end of the servo electric push rod is hinged to one side wall of the self-aligning idler frame; the servo electric push rod is electrically connected to a PLC controller, and both the PLC controller and the industrial camera are connected to a host computer.

[0010] S2. After the belt conveyor is started and the conveyor belt is in normal operation, the industrial camera continuously acquires image sequences containing the conveyor belt and transmits them to the host computer. The host computer extracts the left and right edges of the conveyor belt based on the acquired images and calculates the reference center line position of the conveyor belt.

[0011] S3. After the belt conveyor enters the normal operation stage, the industrial camera begins to continuously and in real-time acquire images of the conveyor belt and transmit them to the host computer. The host computer calculates the average value of the lateral coordinates of the left and right edge points detected in the current frame image, and takes the midpoint of the average position of the left and right edges as the current center position of the conveyor belt, thereby calculating the real-time deviation direction and offset, and transmitting it as a feedback signal to the PLC controller. The PLC controller compares the received real-time deviation with the preset correction threshold of 10 mm. If it exceeds the preset correction threshold, it calculates the precise correction control quantity, generates a control command, and sends it to the servo electric push rod. The linear displacement of the push rod of the servo electric push rod drives the correction roller frame to rotate, converting the electric push rod stroke and roller angle into the rotation angle of the self-aligning roller frame, thereby applying a precise lateral correction force to the running conveyor belt and driving its current center line back to the reference center line position.

[0012] Furthermore, in step S2, the specific process of extracting the left and right edges of the conveyor belt and calculating the reference centerline position of the conveyor belt is as follows:

[0013] 1) In the first frame of the acquired image sequence containing the conveyor belt, the region of interest containing the conveyor belt area is first selected, a corresponding depth map is generated for the region of interest, and the depth value is normalized; then, based on the depth abrupt change characteristics of the conveyor belt edge in the depth map, the image is scanned line by line from the middle position to the left and right, and the position where a significant depth gradient abrupt change first appears in each line is located as the candidate point of the conveyor belt edge. This scanning method ensures that the real boundary between the conveyor belt body and the background is detected first, and avoids false detection of background structures.

[0014] 2) When the number of rows of candidate points on the edge of the conveyor belt in the longitudinal direction reaches the preset ratio threshold (e.g., 40), the preset ratio threshold is set to 30-40, and the edge detection is considered effective. The candidate point sets of the left and right edges are modeled using the robust straight line fitting method to obtain the reference edge lines of the conveyor belt, and the reference center line is further calculated.

[0015] Furthermore, the specific process of step S3 is as follows:

[0016] 1) Repeatedly generate a depth map for the current frame image, and scan the conveyor belt edge in the current frame line by line along the horizontal direction, starting from the reference center position; during the scanning process, by comparing the depth changes of adjacent pixels, identify the pixel positions that meet the depth transition characteristics as candidate edge points, and retain only the first abrupt change point that meets the conditions for each row, and finally obtain the point set of the left and right edges of the conveyor belt in the current frame.

[0017] 2) Calculate the average value of the horizontal coordinates of the left and right edge points detected in the current frame image, and take the midpoint of the average position of the left and right edges and connect them to form a straight line, which is the center position of the current conveyor belt.

[0018] 3) The difference in the horizontal direction between the current center position of the conveyor belt and the reference center line position of the conveyor belt is taken as the pixel-level deviation amount. The deviation direction is determined according to the positive or negative value of the pixel-level deviation amount, where a positive value indicates right deviation and a negative value indicates left deviation. Based on the conversion relationship between pixels and actual physical dimensions obtained in the pre-calibration, the pixel-level deviation amount is converted into millimeter-level deviation distance and transmitted as a feedback signal to the PLC controller.

[0019] 4) The PLC controller compares the received real-time deviation with the preset correction threshold of 10 mm. If it exceeds the preset correction threshold, it calculates the precise correction control quantity according to the PID control algorithm, generates a control command, and sends it to the servo electric push rod. The linear displacement of the push rod of the servo electric push rod drives the correction roller frame to rotate, converting the stroke of the servo electric push rod and the roller angle into the rotation angle of the self-aligning roller frame, thereby applying a precise lateral correction force to the running conveyor belt and driving its current centerline back to the reference centerline position.

[0020] Furthermore, the PID control algorithm is specifically as follows:

[0021] The controller generates the control signal u(t) based on the deviation Δx (i.e., e(t) = Δx) using a proportional-integral-derivative (PID) algorithm:

[0022] (1)

[0023] In the formula: e(t) = real-time deviation - target position; K is a proportional coefficient used to adjust the response speed and can quickly offset most deviations. i The integral coefficient is used to eliminate steady-state error and correct long-term deviations of the system by accumulating historical deviations. The differential coefficient is used to suppress overshoot and reduce adjustment fluctuations by predicting the trend of deviation changes. t is the integral variable; t is time.

[0024] Furthermore, the method for converting the electric push rod stroke and idler angle into the rotation angle of the self-aligning idler frame is as follows:

[0025] 1) The conversion between the stroke of the servo electric actuator and the angle of the idler roller is shown in Figure 3.

[0026] Servo electric actuator extension s and idler roller rotation angle satisfy:

[0027] s= (2)

[0028] In the formula: The radius of the lever arm at the connection point between the push rod and the roller frame;

[0029] 2) The relationship between idler roller angle and conveyor belt displacement correction:

[0030] roller rotation angle This will cause a lateral corrective displacement Δx in the conveyor belt, calculated using the following formula:

[0031] Δx= (3)

[0032] In the formula: L is the effective working length of the idler, that is, the length of the idler that is in contact with the conveyor belt and can effectively push the conveyor belt laterally; This represents the rotation angle of the idler roller.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1) This invention automatically constructs the belt baseline through a vision unit, avoiding the manual annotation and calibration process and realizing the automation of the detection process; it performs edge detection based on depth information, resulting in high detection stability; it can directly output millimeter-level deviation distance, which has clear engineering application value and is easy to link with the control system.

[0035] 2) The vision unit captures the belt deviation status in real time and quantifies it into a digital signal; the PLC controller compares the detected signal with the set benchmark and generates corresponding correction commands; the servo electric push rod responds precisely to the commands, drives the self-aligning idler to generate lateral correction force, and dynamically corrects the running trajectory of the conveyor belt to achieve deviation correction; the whole process forms a dynamic feedback adjustment mechanism until the conveyor belt returns to the set allowable error range and maintains stable operation. Attached Figure Description

[0036] Figure 1 This invention provides the framework for an automatic monitoring and correction system for conveyor belt misalignment.

[0037] Figure 2 This is a schematic diagram showing the connection between the servo electric actuator and the self-aligning idler frame.

[0038] Figure 3 This is an image of the conveyor belt after depth extraction.

[0039] Figure 4 This is an image of the conveyor belt after depth extraction, showing the belt conveyor in its normal operating state.

[0040] Figure 5 This is a depth-extracted image of a belt conveyor in a current state where the conveyor belt is deviating to the left and exceeding the limit.

[0041] Figure 6 This is a depth-extracted image of a belt conveyor in a current state where the conveyor belt is deviating to the right beyond its operating limits.

[0042] Figure 7 This is a diagram showing the relationship between the stroke of the servo electric push rod and the angle of the self-aligning idler in the offset state. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0044] This embodiment exemplifies an automatic monitoring and correction method for conveyor belt misalignment based on depth maps. The hardware and software configuration of the algorithm's operating environment is as follows:

[0045] The system used an Intel Core i7-14700KF processor, 32 GB of RAM, and a 64-bit Windows operating system (Windows 10, version 10.0.19045). The algorithm was implemented using Python 3.9, with PyTorch 2.0.1 (CUDA 11.8) as the deep learning framework. Dependencies related to the depth estimation model and image processing included OpenCV 4.11.0, NumPy 1.26.4, SciPy 1.13.1, and scikit-learn 1.6.1. The specific process is as follows:

[0046] S1, such as Figure 1 and Figure 2 As shown, an industrial camera 3 is fixed above the conveyor belt 1 of the belt conveyor to capture images containing the conveyor belt; a self-aligning idler is rotatably mounted on a self-aligning idler frame 2 below the conveyor belt 1. The bottom of the self-aligning idler frame 2 is rotatably mounted on the frame of the belt conveyor via a rotating shaft 5. A servo electric push rod 4 is fixed on one side of the belt conveyor frame, and the top end of the servo electric push rod 4 is hinged to one side wall of the self-aligning idler frame 2; the servo electric push rod 4 is electrically connected to a PLC controller, and both the PLC controller and the industrial camera 3 are connected to a host computer; the servo electric push rod 4 and the PLC controller can be purchased commercially.

[0047] S2. After the belt conveyor starts and the conveyor belt is in normal operating condition, an industrial camera continuously acquires a sequence of images containing the conveyor belt and transmits them to the host computer. In the first frame of the acquired image sequence containing the conveyor belt, a region of interest (ROI) containing the conveyor belt area is selected. A corresponding depth map is generated for this ROI, and the depth values ​​are normalized. The conveyor belt image after depth extraction is as follows: Figure 3 As shown; then, based on the depth abrupt change characteristics of the conveyor belt edge in the depth map, the image is scanned line by line from the middle to the left and right, and the position of the first significant depth gradient abrupt change in each line is located as the candidate point of the conveyor belt edge. This scanning method ensures that the real boundary between the conveyor belt body and the background is detected first, and avoids false detection of background structures; when the number of rows covered by the detected conveyor belt edge candidate points in the vertical direction reaches a preset ratio threshold (such as 40), the edge detection is considered to be effective; the candidate point sets of the left and right edges are modeled using the robust straight line fitting method to obtain the left and right reference edge lines of the conveyor belt, and the reference center line is further calculated;

[0048] S3. After the belt conveyor enters the normal operation stage, the industrial camera begins to continuously and in real-time acquire images containing the conveyor belt and transmit them to the host computer. The host computer repeatedly generates a depth map for the current frame image and scans the edge of the conveyor belt in the current frame line by line along the horizontal direction, starting from the reference center position. During the scanning process, by comparing the depth changes of adjacent pixels, the pixel positions that meet the depth transition characteristics are identified as candidate edge points, and only the first abrupt change point that meets the conditions is retained in each row, finally obtaining the point set of the left and right edges of the conveyor belt in the current frame.

[0049] Calculate the average horizontal coordinates of the left and right edge points detected in the current frame image, and take the midpoint of the average position of the left and right edges and connect them to form a straight line, which is the center position of the current conveyor belt.

[0050] The difference in the horizontal direction between the current center position of the conveyor belt and the reference center line position is taken as the pixel-level deviation amount. The deviation direction is determined by the positive or negative value of the pixel-level deviation amount, where a positive value indicates right deviation and a negative value indicates left deviation. Based on the pre-calibrated conversion relationship between pixels and actual physical dimensions, the pixel-level deviation amount is converted into a millimeter-level deviation distance. Figures 4-6 As shown, in this embodiment, the pre-calibrated deviation distance of one pixel corresponds to a deviation distance of 2 mm; and this deviation distance is transmitted to the PLC controller as a feedback signal.

[0051] The PLC controller compares the received real-time deviation with a preset correction threshold of 10 mm. If the deviation is less than the threshold, no correction is performed. Figure 3 As shown, when the pixel deviation at the center of the conveyor belt is 3 pixels (leftward deviation), the corresponding offset is -6.00 mm, which is less than the preset correction threshold of 10 mm. In this case, no correction is performed, and it is recorded as normal. If it exceeds the preset correction threshold, such as... Figure 4 As shown, when the pixel deviation at the center position of the conveyor belt is 12 pixels, shifting to the left, the corresponding offset is -24.00 mm, which is greater than the preset correction threshold of 10 mm. This is recorded as an excessive leftward deviation, and correction is required. Figure 5 As shown, when the pixel deviation at the center position of the conveyor belt is 9 pixels, which is a rightward deviation, the corresponding offset is 18.00 mm, which is greater than the preset correction threshold of 10 mm. This is recorded as an excessive rightward deviation, and correction is required. When correction is required, the precise correction control quantity is calculated according to the PID control algorithm, and a control command is generated and sent to the servo electric actuator. The PID control algorithm is as follows:

[0052] The controller generates the control signal u(t) based on the deviation Δx (i.e., e(t) = Δx) using a proportional-integral-derivative (PID) algorithm:

[0053] (1)

[0054] In the formula: e(t) = real-time deviation - target position; K is a proportional coefficient used to adjust the response speed and can quickly offset most deviations. i The integral coefficient is used to eliminate steady-state error and correct long-term deviations of the system by accumulating historical deviations. The differential coefficient is used to suppress overshoot and reduce adjustment fluctuations by predicting the trend of deviation changes. t is the integral variable; t is time.

[0055] The linear displacement of the servo electric push rod drives the rotation of the self-aligning idler frame, converting the stroke of the servo electric push rod and the idler angle into the rotation angle of the self-aligning idler frame. Figure 7 As shown, the method for converting the electric actuator stroke and idler angle into the rotation angle of the self-aligning idler frame is as follows:

[0056] 1) The conversion between the stroke of the servo electric actuator and the angle of the idler roller is shown in Figure 3.

[0057] Servo electric actuator extension s and idler roller rotation angle satisfy:

[0058] s= (2)

[0059] In the formula: The radius of the lever arm at the connection point between the push rod and the roller frame;

[0060] 2) The relationship between idler roller angle and conveyor belt displacement correction:

[0061] roller rotation angle This will cause a lateral corrective displacement Δx in the conveyor belt, calculated using the following formula:

[0062] Δx= (3)

[0063] In the formula: L is the effective working length of the idler, that is, the length of the idler that is in contact with the conveyor belt and can effectively push the conveyor belt laterally; This represents the rotation angle of the idler roller.

[0064] This applies a precise lateral correction force to the conveyor belt in operation, driving its current centerline back to the reference centerline position or reducing the real-time deviation to within the preset correction threshold of 10 mm, thus maintaining stable operation.

[0065] Through the above relationship, the system can convert the deviation detected by vision into control commands for the servo motor, realizing a complete closed-loop control from deviation detection to mechanical correction, ensuring that the conveyor belt always stays on the correct running trajectory.

[0066] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for automatic monitoring and correction of conveyor belt misalignment based on depth maps, characterized in that, Includes the following steps: S1. An industrial camera is fixed above the conveyor belt of the belt conveyor to capture images of the conveyor belt; a self-aligning idler is rotatably mounted on a self-aligning idler frame below the conveyor belt, and the bottom of the self-aligning idler frame is rotatably mounted on the frame of the belt conveyor via a rotating shaft. A servo electric push rod is fixed on one side of the belt conveyor frame, and the top end of the servo electric push rod is hinged to one side wall of the self-aligning idler frame; the servo electric push rod is electrically connected to a PLC controller, and both the PLC controller and the industrial camera are connected to a host computer. S2. After the belt conveyor is started and the conveyor belt is in normal operation, the industrial camera continuously acquires image sequences containing the conveyor belt and transmits them to the host computer. The host computer extracts the left and right edges of the conveyor belt based on the acquired images and calculates the reference center line position of the conveyor belt. S3. After the belt conveyor enters the normal operation stage, the industrial camera begins to continuously and in real-time acquire images of the conveyor belt and transmit them to the host computer. The host computer calculates the average value of the lateral coordinates of the left and right edge points detected in the current frame image, and takes the midpoint of the average position of the left and right edges as the current center position of the conveyor belt, thereby calculating the real-time deviation direction and offset, and transmitting it as a feedback signal to the PLC controller. The PLC controller compares the received real-time deviation with the preset correction threshold of 10 mm. If it exceeds the preset correction threshold, it calculates the precise correction control quantity, generates a control command, and sends it to the servo electric push rod. The linear displacement of the push rod of the servo electric push rod drives the correction roller frame to rotate, converting the electric push rod stroke and roller angle into the rotation angle of the self-aligning roller frame, thereby applying a precise lateral correction force to the running conveyor belt and driving its current center line back to the reference center line position.

2. The method for automatic monitoring and correction of conveyor belt misalignment based on depth maps according to claim 1, characterized in that, In step S2, the specific process of extracting the left and right edges of the conveyor belt and calculating the reference centerline position of the conveyor belt is as follows: 1) In the first frame of the acquired image sequence containing the conveyor belt, the region of interest containing the conveyor belt area is first selected, a corresponding depth map is generated for the region of interest, and the depth value is normalized; then, based on the depth abrupt change characteristics of the conveyor belt edge in the depth map, the image is scanned line by line from the middle position to the left and right, and the position where a significant depth gradient abrupt change first appears in each line is located as the candidate point of the conveyor belt edge. This scanning method ensures that the real boundary between the conveyor belt body and the background is detected first, and avoids false detection of background structures. 2) When the number of rows covered by the detected candidate points of the conveyor belt edge in the longitudinal direction reaches the preset ratio threshold, the preset ratio threshold is set to 30-40, and the edge detection is considered to be effective; the candidate point sets of the left and right edges are modeled using the robust straight line fitting method to obtain the left and right reference edge lines of the conveyor belt, and the reference center line is further calculated.

3. The method for automatic monitoring and correction of conveyor belt misalignment based on depth maps according to claim 1, characterized in that, The specific process of step S3 is as follows: 1) Repeatedly generate a depth map for the current frame image, and scan the conveyor belt edge in the current frame line by line along the horizontal direction, starting from the reference center position; during the scanning process, by comparing the depth changes of adjacent pixels, identify the pixel positions that meet the depth transition characteristics as candidate edge points, and retain only the first abrupt change point that meets the conditions for each row, and finally obtain the point set of the left and right edges of the conveyor belt in the current frame. 2) Calculate the average value of the horizontal coordinates of the left and right edge points detected in the current frame image, and take the midpoint of the average position of the left and right edges and connect them to form a straight line, which is the center position of the current conveyor belt. 3) The difference in the horizontal direction between the current center position of the conveyor belt and the reference center line position of the conveyor belt is taken as the pixel-level deviation amount. The deviation direction is determined according to the positive or negative value of the pixel-level deviation amount, where a positive value indicates right deviation and a negative value indicates left deviation. The pixel-level deviation amount is converted into millimeter-level deviation distance according to the pre-calibrated conversion relationship between pixels and actual physical dimensions. And transmit it as a feedback signal to the PLC controller; 4) The PLC controller compares the received real-time deviation with the preset correction threshold of 10 mm. If it exceeds the preset correction threshold, it calculates the precise correction control quantity according to the PID control algorithm, generates a control command, and sends it to the servo electric push rod. The linear displacement of the push rod of the servo electric push rod drives the correction roller frame to rotate, converting the stroke of the servo electric push rod and the roller angle into the rotation angle of the self-aligning roller frame, thereby applying a precise lateral correction force to the running conveyor belt and driving its current centerline back to the reference centerline position.

4. The method for automatic monitoring and correction of conveyor belt misalignment based on depth maps according to claim 3, characterized in that, The PID control algorithm is specifically as follows: The controller generates the control signal u(t) based on the deviation Δx (i.e., e(t) = Δx) using a proportional-integral-derivative (PID) algorithm: （1） In the formula: e(t) = real-time deviation - target position; K is a proportional coefficient used to adjust the response speed and can quickly offset most deviations. i The integral coefficient is used to eliminate steady-state error and correct long-term deviations of the system by accumulating historical deviations. The differential coefficient is used to suppress overshoot and reduce adjustment fluctuations by predicting the trend of deviation changes. For integration variables; t represents time.

5. The method for automatic monitoring and correction of conveyor belt misalignment based on depth maps according to claim 3, characterized in that, The method for converting the electric actuator stroke and idler roller angle into the rotation angle of the self-aligning idler frame is as follows: 1) The conversion between the stroke of the servo electric actuator and the angle of the idler roller is shown in Figure 3. Servo electric actuator extension s and idler roller rotation angle satisfy: s= （2） In the formula: The radius of the lever arm at the connection point between the push rod and the roller frame; 2) The relationship between idler roller angle and conveyor belt displacement correction: roller rotation angle This will cause a lateral corrective displacement Δx in the conveyor belt, calculated using the following formula: Δx= (3) In the formula: L is the effective working length of the idler, that is, the length of the idler that is in contact with the conveyor belt and can effectively push the conveyor belt laterally; This represents the rotation angle of the idler roller.

Images