Coal conveying belt switching control method

Abstract

The invention discloses a coal conveying belt switching control method, which belongs to the field of automation control, and comprises the following steps: controlling starting and stopping of a material taking machine and a conveying belt according to a task chain; collecting image data of the coal flow running on the conveyor belt; a coal flow image and a conveyor belt image are distinguished through a set threshold value, and contour recognition is carried out on image data of the coal flow; calculating the pixel displacement of the image data of the coal flow by adopting an optical flow method, and performing coordinate transformation on the pixel displacement to obtain the running displacement and running speed of the coal flow on the conveyor belt; predicting the position of the head of the coal flow on the conveyor belt according to the contour information and the running speed of the coal flow; when the next task needs to switch the tail end conveyor belt, the central control system calculates the moment t1 when the coal flow of the current task on the conveyor belt is completely conveyed according to the prediction information of the head position of the coal flow, and controls the switching of the conveyor belt; aiming at low coal conveying efficiency in the prior art, the coal conveying efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of automation control, and more specifically, to a coal conveyor belt switching control method. Background Art

[0002] The coal port is an important hub connecting railway coal transportation and sea coal transportation. It consists of multiple work areas, and the coal conveyor belt is an important equipment connecting each work area. The main work of the coal port is as follows: coal is unloaded from the train and transported to the storage area via the conveyor belt. The reclaimer then transports the coal in the storage area to the ship loader via the conveyor belt, and then loads it into the ship. The conveyor belt is an important component of almost all work tasks, and the operation of the conveyor belt directly affects the work efficiency of the coal port tasks.

[0003] The coal conveyor belt is a large-scale transportation system composed of multiple conveyor belts, which can be hundreds of meters or even more than one kilometer in length. During the work task, the multi-section conveyor belt is connected to other large equipment in different work areas to transfer coal. Since there are multiple work equipment and work tasks in each work area, the multi-section conveyor belt may switch the connection with the corresponding conveyor belt and work equipment according to the service object of the work task, so as to smoothly complete the conversion of different work tasks. For example, coal is taken out from the storage area by the reclaimer and transported to the ship loader by the conveyor belt for the task of loading the ship. After completing a task, the head or tail end of the multi-section conveyor belt may need to perform the switching task due to different requirements for coal types and different types of ships, so as to achieve connection with different equipment.

[0004] In the current production scenario, there is a lot of room for improvement in the control of conveyor belts. From the perspective of safe production, first of all, conveyor belts have a strict upstream and downstream relationship, that is, once the downstream belt stops, the upstream belt will stop immediately; the upstream belt can only be started after the downstream belt starts. Secondly, when the conveyor belts are switched, all conveyor belts need to be completely stopped, which causes many unnecessary delays and repeated starts and stops of some conveyor belts.

[0005] In terms of equipment control, conveyor belts in many production scenarios lack control linkage, and usually can only realize remote control of switches and other behaviors. Specific task deployment and equipment operation still require a lot of manpower and material resources to execute; in many existing work scheduling systems that include conveyor belts, usually only a simple "prohibit sharing of equipment for different tasks" logic is set, resulting in different tasks of terminal equipment requiring the operation of all conveyor belts to be stopped before switching. Summary of the invention

[0006] 1. Technical issues to be solved

[0007] In response to the problem of low coal transportation efficiency in the prior art, the present application provides a coal conveyor belt switching control method, which receives task requirements and generates task chains through a PLC-based central control system to control the reclaimer and conveyor belt to transport coal; collects image data of the coal flow running on the conveyor belt, and after preprocessing, binarization, contour recognition and other processing, uses the optical flow method to calculate the running displacement and speed of the coal flow on the conveyor belt; then predicts the position of the coal flow head on the conveyor belt, and the central control system reasonably arranges the time for conveyor belt switching and the next task coal supply based on the coal flow head position prediction information, thereby realizing intelligent switching control of the coal conveyor belt and improving coal transportation efficiency.

[0008] 2. Technical solution

[0009] The purpose of this application is achieved through the following technical solutions.

[0010] The present specification provides a coal conveyor belt switching control method, including: establishing a central control system based on PLC, receiving task requirements and generating a task chain; transmitting the generated task chain to the material reclaimer and the conveyor belt, and controlling the start and stop of the material reclaimer and the conveyor belt according to the task chain to control coal transportation; collecting image data of coal flow running on the conveyor belt; preprocessing the collected image data; binarizing the preprocessed image data, distinguishing the coal flow image and the conveyor belt image by setting a threshold, and performing contour recognition on the binary image data of the coal flow to extract the contour information of the coal flow; using the optical flow method to process the continuous coal flow image data, calculating the pixel displacement of the coal flow image data, and performing coordinate transformation on the pixel displacement to obtain the running displacement and running speed of the coal flow on the conveyor belt; predicting the position of the coal flow head on the conveyor belt according to the contour information and running speed of the coal flow. ; When the next task requires switching the end conveyor belt, the central control system calculates the time t1 when all the coal flows of the current task on the conveyor belt are transported completely based on the coal flow head position prediction information; at time t2 after time t1, after receiving the conveyor belt switching instruction, the central control system controls the end conveyor belt to stop running and executes the switching action, and the switching action lasts for δt0; the central control system calculates the time δt1 required for the coal flow head of the next task to reach the preset safety position of the end conveyor belt based on the coal flow running speed; the central control system calculates the time t3 to start the coal supply of the next task based on the switching action duration δt0 and the time δt1 required for the coal flow head of the next task to reach the preset safety position of the end conveyor belt, t3≤t2+δt0-δt1; at time t3, the central control system controls the reclaimer to start the coal supply of the next task and start the coal transportation of the next task.

[0011] Among them, PLC stands for Programmable Logic Controller, which is a digital electronic device specially designed for industrial control. Binarization: It is a commonly used method in image processing. The pixel values ​​in the image are divided into two categories according to the set threshold. Generally, the pixels above the threshold are set to white (255) and the pixels below the threshold are set to black (0), so as to convert the original grayscale image into a binary image containing only black and white colors. Optical flow method: It is a method used to estimate the motion of objects in image sequences. It is based on the grayscale invariance assumption, that is, the grayscale value of the same object remains unchanged between different frames. By calculating the spatiotemporal gradient of the pixel grayscale value in the image sequence, the motion displacement vector of the pixel between adjacent frames is estimated, thereby obtaining the motion information of the entire image. Coal flow head: It refers to the frontmost part of the coal flow on the conveyor belt during the transportation process. Accurately identifying and tracking the position of the coal flow head is of great significance for realizing the automatic switching control of the coal conveyor belt. By predicting the time when the coal flow head reaches the end of the conveyor belt, the time of conveyor belt switching and coal supply for the next task can be reasonably arranged to avoid problems such as coal transportation interruption or coal overflow. Pixel displacement: refers to the change in the spatial position between corresponding pixels of the same object in adjacent frames in an image sequence. By calculating the horizontal and vertical displacement of image pixels, pixel-level motion information can be obtained, and the object's motion speed and trajectory can be estimated. In the switching control of coal conveyor belts, by calculating the pixel displacement of the coal flow image and converting it into actual physical displacement, the displacement and speed of the coal flow on the conveyor belt can be obtained.

[0012] Preferably, image preprocessing includes: performing image cutting on the received image data of the coal flow to extract the image of the target area; performing grayscale processing on the cut image to convert the RGB image into a grayscale image; performing Gaussian filtering on the grayscale image to remove noise in the image; performing linear transformation on the Gaussian filtered image to adjust the contrast and brightness of the image, and using the linearly transformed image as the preprocessed image data of the coal flow.

[0013] Furthermore, the pixel displacement of the coal flow image is calculated, including: extracting the current frame image according to the preprocessed coal flow image data; for each pixel of the current frame image, selecting multiple pixel points in its neighborhood, and constructing an optical flow equation according to the pixel grayscale invariance and the local smoothness of the optical flow field; performing least squares solution on the constructed optical flow equation to obtain the displacement of the current pixel in the horizontal and vertical directions; repeating the steps of solving the displacement for all pixels of the current frame image to obtain the pixel displacement vector field of the entire image; and outputting the obtained pixel displacement vector field as the pixel displacement of the coal flow image in the current frame.

[0014] Specifically, according to the preprocessed image data of the coal flow, the current frame image I (x, y, t) and the previous frame image I (x, y, t-1) are extracted, where x, y are pixel coordinates and t is time. For each pixel (x, y) of the current frame image, multiple pixels are selected in its neighborhood. Assuming that the displacement of the pixel (x, y) in the horizontal and vertical directions is u and v respectively, the pixel point coordinates corresponding to the pixel point (x+dx, y+dy) in the neighborhood in the previous frame image are (x+dx-u, y+dy-v). Based on the grayscale invariance assumption, that is, the grayscale value of the same object remains unchanged between adjacent frames, the constraint equation can be established: I(x, y, t) = I(xu, yv, t-1), and the first-order Taylor expansion of the above equation at (x, y) is performed to obtain: I(x, y, t) = I(x, y, t-1) - Ix(x, y, t-1)u - Iy(x, y, t-1)v + It(x, y, t-1), where Ix, Iy, and It are the partial derivatives of the image in the x, y, and t directions, respectively. Combined with the assumption of local smoothness of the optical flow field, that is, adjacent pixels have similar motion trends, multiple pixels are selected in the neighborhood to construct the optical flow equations: Ix(x1, y1, t-1)u+Iy(x1, y1, t-1)v=-It(x1, y1, t-1), Ix(x2, y2, t-1)u+Iy(x2, y2, t-1)v=-It(x2, y2, t-1), ..., Ix(xn, yn, t -1)u+Iy(xn,yn,t-1)v=-It(xn,yn,t-1); perform least squares solution on the constructed optical flow equations, that is, solve the following optimization problem: min∑[Ix(xi,yi,t-1)u+Iy(xi,yi,t-1)v+It(xi,yi,t-1)]^2. Solving this optimization problem can obtain the displacement u and v of the current pixel (x, y) in the horizontal and vertical directions. Repeat steps 2-5 for all pixels of the current frame image to obtain the pixel displacement vector field (u(x, y), v(x, y)) of the entire image. The obtained pixel displacement vector field is output as the pixel displacement of the coal flow image in the current frame, expressed as: (du(x, y), dv(x, y))=(u(x, y), v(x, y)). Through the above steps, the optical flow method can be used to calculate the pixel displacement vector field of the coal flow image between consecutive frames. The displacement vector field reflects the movement of the coal flow on the image plane and provides important information for subsequent coal flow velocity estimation and position prediction.

[0015] Furthermore, the running displacement of the coal flow on the conveyor belt is obtained, including: establishing a mapping relationship between the pixel coordinate system, the image coordinate system and the camera coordinate system according to the pinhole camera model; converting the pixel displacement vector field in the pixel coordinate system to the image coordinate system through the intrinsic parameter matrix between the camera imaging plane and the image plane; converting the displacement vector field in the image coordinate system to the camera coordinate system through the extrinsic parameter matrix between the camera coordinate system and the world coordinate system; converting the displacement vector field in the camera coordinate system to the conveyor belt coordinate system according to the positional relationship between the camera coordinate system and the conveyor belt coordinate system; multiplying the displacement vector field in the conveyor belt coordinate system by the displacement time to obtain the running displacement of the coal flow on the conveyor belt.

[0016] Furthermore, the pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system by the following formula:

[0017] Among them, J x ,J y is the image coordinate system; R x ,R y is the pixel coordinate system; Rx0, Ry0 are the coordinates of the reference point in the pixel coordinate system; converted to matrix form: The relationship between the pixel coordinate system and the image coordinate system: The pixel coordinate system uses the x and y axes as coordinate axes, the Z axis as the optical axis, the origin O as the optical center of the camera, and the origin O and O j Distance OO j is the focal length f of the camera; the position of point A in the pixel coordinate system is (x, y, z), and the image A′ in the image coordinate system is (J x ,J y ).

[0018] Furthermore, the displacement vector field in the pixel coordinate system is transformed to the camera coordinate system through the extrinsic parameter matrix between the camera coordinate system and the camera coordinate system: The camera coordinate system uses the x and y axes as coordinate axes and the Z axis as the optical axis; R x0 and R y0 is the coordinate of the reference point in the pixel coordinate system; R x ,R y is the pixel coordinate system.

[0019] Furthermore, according to the positional relationship between the camera coordinate system and the conveyor belt coordinate system, the displacement vector field in the camera coordinate system is transformed into the conveyor belt coordinate system:

[0020]

[0021] The camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is z, the pixel position of the object translated on the image is (m,n), and the actual moving distance is ΔO;

[0022] Furthermore, the displacement vector field in the conveyor belt coordinate system is multiplied by the displacement time to obtain the running displacement of the coal flow on the conveyor belt: The camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is z, the pixel position of the object translated on the image is (m,n), and the actual moving distance is ΔO;

[0023] Furthermore, the coal flow image and the conveyor belt image are distinguished by setting a threshold, including: calculating the grayscale histogram of the image according to the preprocessed coal flow image data; traversing all preset thresholds according to the grayscale histogram, and dividing the image pixels into the background area and the foreground area; for each threshold, calculating the number of pixels in the background area and the foreground area, the average grayscale value, and the average grayscale value of the entire image; using the inter-class variance formula, calculating the inter-class variance between the background area and the foreground area; selecting the threshold corresponding to the maximum inter-class variance as the optimal threshold for image binarization; using the obtained optimal threshold, setting the pixels in the coal flow image data whose grayscale values ​​are greater than or equal to the optimal threshold to 255, representing the coal flow foreground image, and vice versa, to 0, representing the conveyor belt background image; outputting the coal flow foreground image data and the conveyor belt background image data after binarization.

[0024] Specifically, the grayscale histogram of the image is calculated based on the preprocessed image data of the coal flow. Assuming that the grayscale range of the image is [0, 255], the grayscale histogram can be expressed as a 256-dimensional vector H(i), where i is the grayscale value and H(i) is the number of pixels with a grayscale value of i. According to the grayscale histogram, all preset thresholds T (with a value range of [0, 255]) are traversed to divide the image pixels into background area and foreground area. For each threshold T, pixels with grayscale values ​​less than T belong to the background area, and pixels with grayscale values ​​greater than or equal to T belong to the foreground area. For each threshold T, the number of pixels in the background area and foreground area, the average grayscale value, and the average grayscale value of the entire image are calculated. Number of pixels in the background area: w0 = ∑[i = 0 to T-1] H(i) / ∑[i = 0 to 255] H(i); Number of pixels in the foreground area: w1 = ∑[i = T to 255] H(i) / ∑[i = 0 to 255] H(i); Average grayscale value of the background area: u0 = ∑[i = 0 to T-1] i* H(i) / (w0 * ∑[i = 0 to 255] H(i)); Average grayscale value of the foreground area: u1 = ∑[i = T to 255] i* H(i) / (w1 * ∑[i = 0 to 255] H(i)); Average grayscale value of the entire image: u=∑[i=0to255]i*H(i) / ∑[i=0to255]H(i); Use the inter-class variance formula to calculate the inter-class variance between the background area and the foreground area: g=w0*w1*(u0-u1)^2; Select the threshold corresponding to the maximum inter-class variance as the optimal threshold T_opt for image binarization: T_opt=argmax(g), and use the obtained optimal threshold T_opt to set the pixels with grayscale values ​​greater than or equal to T_opt in the image data of the coal flow to 255, representing the foreground image of the coal flow, otherwise set them to 0, representing the background image of the conveyor belt.

[0025] For each pixel (x, y) in the image, its binarization result is as follows: if I(x, y) >= T_opt, then I_binary(x, y) = 255; if I(x, y) < T_opt, then I_binary(x, y) = 0. Output the foreground image data I_foreground of the coal flow and the background image data I_background of the conveyor belt after binarization processing: I_foreground(x, y) = I_binary(x, y), where I_binary(x, y) = 255, and I_background(x, y) = I_binary(x, y), where I_binary(x, y) = 0. Through the above steps, the optimal threshold can be adaptively selected using the between-class variance method to achieve the binarization segmentation of the coal flow image and the conveyor belt image. The basic idea of the between-class variance method is that when selecting the threshold, the between-class variance between the background region and the foreground region is maximized, so that the separation degree of the two regions is the highest.

[0026] Furthermore, perform contour recognition on the image data of the coal flow after binarization processing to extract the contour information of the coal flow, including: according to the obtained foreground image data of the coal flow, sample the Gaussian filter to smooth the image; calculate the gradient magnitude and direction of the smoothed image to obtain the gradient magnitude image and the gradient direction image; perform non-maximum suppression processing on the gradient magnitude image; use the double-threshold method to threshold the image after non-maximum suppression, connect the edge pixels to obtain the edge image; perform contour tracing on the edge image to extract the connected edge contour pixel coordinates; output the extracted edge contour pixel coordinates as the coal flow contour information.

[0027] Specifically, according to the obtained foreground image data I_foreground of the coal flow, use the Gaussian filter to smooth the image. The kernel function of the Gaussian filter is: G(x, y) = (1 / (2piσ^2)) * exp(-(x^2 + y^2) / (2 * σ^2)), where σ is the standard deviation of the Gaussian filter, which controls the degree of smoothing. Convolve the Gaussian filter with the foreground image to obtain the smoothed image I_smooth: Calculate the gradient magnitude and direction of the smoothed image I_smooth to obtain the gradient magnitude image I_mag and the gradient direction image I_dir. Calculate the gradients of the image in the x direction and the y direction respectively: Calculate the gradient magnitude: I_mag = sqrt(I_dx^2 + I_dy^2), calculate the gradient direction: I_dir = arctan(I_dy / I_dx), and perform non-maximum suppression on the gradient magnitude image I_mag. For each pixel (x, y), check the gradient magnitude of the two adjacent pixels in the gradient direction. If the gradient magnitude of the current pixel is not the largest, set it to 0 to obtain the image I_nms after non-maximum suppression.

[0028] The double threshold method is used to perform threshold processing on the image I_nms after non-maximum suppression, and the edge pixels are connected to obtain the edge image I_edge. Set the high threshold T_high and the low threshold T_low, where T_high = k*T_low, and k is a constant factor. For each pixel (x, y) in I_nms: if its gradient amplitude is greater than the high threshold T_high, it is marked as a strong edge pixel, I_edge(x, y) = 255. If its gradient amplitude is less than the low threshold T_low, it is marked as a non-edge pixel, I_edge(x, y) = 0. If its gradient amplitude is between the high and low thresholds, it is marked as a weak edge pixel, I_edge(x, y) = 128. For all weak edge pixels, check whether there is a strong edge pixel in its 8-neighborhood. If so, it is marked as a strong edge pixel, otherwise it is marked as a non-edge pixel. Perform contour tracing on the edge image I_edge and extract the coordinates of the connected edge contour pixels. Scan the edge image from the upper left corner and find the first strong edge pixel as the starting point. Starting from the starting point, check its 8 neighboring pixels in a clockwise direction, find the next strong edge pixel, and add its coordinates to the contour point set. Repeat the previous step until you return to the starting point and get a closed contour. Continue to scan the edge image, find the next unvisited strong edge pixel, and repeat the above process until all strong edge pixels have been visited. The extracted edge contour pixel coordinates are output as the coal flow contour information, expressed as a contour point set C = {p1, p2, ..., pn}, where pi = (xi, yi) is the contour point coordinate. Through the above steps, the Canny edge detection operator can be used to identify the contour of the coal flow image after binary processing, and the contour information of the coal flow can be extracted. The Canny operator can effectively extract the real edge in the image through operations such as gradient calculation, non-maximum suppression and double threshold connection, and has good positioning accuracy and continuity.

[0029] Furthermore, the position of the coal flow head on the conveyor belt is predicted based on the contour information and running speed of the coal flow, including: marking the position coordinates of the coal flow head in the current image frame based on the obtained coal flow contour information; using the marked coal flow head position coordinates as the observation value of the Kalman filter algorithm to establish an observation equation; using the uniform motion model to establish the state equation of the coal flow head position based on the historical position coordinates and current position coordinates of the coal flow head; using the state equation to predict the position of the coal flow head to obtain the predicted position coordinates and the predicted error covariance matrix; using the observation equation to update the prediction result to obtain the updated coal flow head position coordinates and the updated error covariance matrix; using the obtained coal flow head position coordinates as the current moment coal flow head position output in the image; using the obtained coal flow head position coordinates and the error covariance matrix as the initial value of the Kalman filter algorithm for the next frame of image; repeating the above steps to predict and update the position coordinates of the coal flow head on the conveyor belt.

[0030] Specifically, according to the obtained coal flow contour information C = {p1, p2, ..., pn}, the position coordinates of the coal flow head in the current image frame are marked. Assuming that the movement direction of the coal flow is from top to bottom, the point with the smallest ordinate in the contour point set C is the position of the coal flow head. Find the point ph = (xh, yh) with the smallest ordinate in the contour point set C, and mark it as the position coordinate of the coal flow head. Use the marked coal flow head position coordinate ph as the observation value of the Kalman filter algorithm to establish the observation equation. Define the observation value matrix Z = [xh, yh]' to represent the position of the coal flow head in the image. Define the observation matrix H = [1, 0, 0, 0; 0, 1, 0, 0] to associate the state variables with the observation values. Establish the observation equation: Z = H*X+V, where X is the state variable matrix and V is the observation noise matrix. According to the historical position coordinates and current position coordinates of the coal flow head, the uniform motion model is used to establish the state equation of the coal flow head position. Define the state variable matrix X = [x, y, vx, vy]', where (x, y) is the position coordinate of the coal flow head, and (vx, vy) is the velocity component of the coal flow head. Define the state transfer matrix A = [1, 0, dt, 0; 0, 1, 0, dt; 0, 0, 1, 0; 0, 0, 0, 1], where dt is the time interval between adjacent frames. Establish the state equation: X(k+1) = A*X(k) + W(k), where k is the time step and W is the process noise matrix. Use the state equation to predict the position of the coal flow head and obtain the predicted position coordinates and the prediction error covariance matrix. According to the state estimation value X(k|k) and the state transfer matrix A at the previous moment, the state prediction value X(k+1|k) at the current moment is calculated: X(k+1|k)=A*X(k|k). According to the error covariance matrix P(k|k) and the process noise covariance matrix Q at the previous moment, the prediction error covariance matrix P(k+1|k) at the current moment is calculated: P(k+1|k)=A*P(k|k)*A'+Q. The prediction result is updated using the observation equation to obtain the updated coal flow head position coordinates and the updated error covariance matrix. The Kalman gain matrix K(k+1) is calculated: K(k+1)=P(k+1|k)H' / (HP(k+1|k)*H'+R), where R is the observation noise covariance matrix. According to the observed value Z(k+1) and the predicted value X(k+1|k), the updated state estimate X(k+1|k+1) is calculated: X(k+1|k+1)=X(k+1|k)+K(k+1)(Z(k+1)-HX(k+1|k)), and the updated error covariance matrix P(k+1|k+1) is calculated: P(k+1|k+1)=(IK(k+1)*H)*P(k+1|k), and the obtained coal flow head position coordinates (x, y) are output as the position of the coal flow head in the image at the current moment.The obtained coal flow head position coordinates (x, y) and error covariance matrix P(k+1|k+1) are used as the initial values ​​of the Kalman filter algorithm for the next frame image. Repeat the above steps to predict and update the position coordinates of the coal flow head on the conveyor belt.

[0031] 3. Beneficial effects

[0032] Compared with the prior art, the advantages of this application are:

[0033] By establishing a PLC-based central control system, receiving task requirements and generating task chains, and transmitting the task chains to the reclaimer and conveyor belt for coal transportation control, the coal transportation process is automated, reducing the intervention of manual operations and improving coal transportation efficiency.

[0034] The image data of coal flow on the conveyor belt is collected, and the image data is preprocessed, binarized, and contour recognized to extract the contour information of the coal flow. The displacement and speed of the coal flow on the conveyor belt are calculated by combining the optical flow method, providing an accurate data basis for the subsequent prediction of the head position of the coal flow.

[0035] According to the profile information and running speed of the coal flow, the Kalman filter algorithm is used to predict the position of the coal flow head on the conveyor belt. When the conveyor belt needs to be switched, the central control system can accurately calculate the time when the current task coal transportation is completed based on the predicted information of the coal flow head position, and reasonably arrange the time for conveyor belt switching and coal supply for the next task, avoiding the interruption of coal transportation or coal overflow.

[0036] During the conveyor belt switching process, the start time of the next task coal supply is reasonably controlled by calculating the duration of the conveyor belt switching action and the time required for the head of the next task coal flow to reach a safe position. This not only ensures the completion of the conveyor belt switching action, but also avoids the delay of coal supply after the switching is completed, shortens the time of coal transportation task switching, and improves coal transportation efficiency.

[0037] Image preprocessing includes steps such as image cutting, grayscale, Gaussian filtering, and linear transformation, which effectively removes image noise, enhances image contrast and brightness, provides high-quality image data for subsequent image processing, and improves the accuracy of coal flow contour information extraction and displacement calculation.

[0038] The pixel displacement of the coal flow image is calculated using the optical flow method. By constructing and solving the optical flow equation, the displacement of the image pixels in the horizontal and vertical directions is obtained, and the pixel displacement vector field of the entire image is generated. Combining the pinhole camera model and the coordinate transformation relationship, the pixel displacement is converted to the actual running displacement of the coal flow on the conveyor belt, which improves the accuracy of the coal flow velocity calculation.

[0039] By setting a threshold to binarize the image and using the inter-class variance method to adaptively select the optimal threshold, the coal flow foreground image and the conveyor belt background image can be effectively distinguished, which facilitates the subsequent coal flow contour recognition.

[0040] Edge detection and contour tracing are performed on the binary image to extract the contour pixel coordinates of the coal flow, accurately describe the contour information of the coal flow, and provide reliable data support for the prediction of the head position of the coal flow.

[0041] When predicting the position of the coal flow head, the Kalman filter algorithm is used. By establishing the state equation and observation equation of the coal flow head position, the coal flow head position is recursively predicted and updated, which makes full use of the historical position information of the coal flow head and improves the accuracy and robustness of position prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the overall process of the system provided for this application;

[0043] Figure 2 A schematic diagram of image transformation provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the relationship between the conveyor belt switching time and the task start time provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] Example 1

[0047] See also Figure 1 , Figure 1 A schematic diagram of the overall system flow provided for this application.

[0048] First, the central control system established in this application receives a series of task requirements from superiors. These task requirements are generally in a similar form, consisting of information such as the model of the ship coming to the port, the type of coal required by the ship, etc. The superior integrates this information, and generates a task chain consisting of coal types, stacks, reclaimers, conveyor belts, loaders, and ships based on the current equipment resources in the port; at the same time, the central control system opens and closes multiple conveyor belts according to the equipment required in the task chain to ensure the smooth progress of the task.

[0049] Secondly, the visual monitoring system designed in this application uses an intelligent camera to monitor the coal transportation on the conveyor belt in real time, captures the contour, speed and other information of the coal flow, and uses the optical flow method to process the image of the coal flow in real time after image preprocessing to obtain the image frame displacement of the coal flow; then, according to the internal and external parameters of the camera, the image frame displacement of the coal flow is transformed through coordinate transformation to obtain the displacement of the coal flow in the real world, thereby obtaining its real-time speed. The real-time position of the coal flow head is predicted by combining the contour, real-time speed and other information of the coal flow, and this information is visually transmitted to the central control system to achieve real-time control of the coal flow information on the conveyor belt.

[0050] Finally, this application designs a complete problem-solving implementation method for the switching scenarios of coal conveyor belts due to task requirements.

[0051] Take a typical switching scenario as an example: suppose the ship model of the next task changes and a different ship loader is required, that is, the conveyor belt connected to the ship loader needs to be switched to adapt to the change of the task. When the previous task is about to be completed, the central control system has mastered the real-time information of the coal flow through the visual monitoring system. After stopping the coal transportation at the head end of the conveyor belt, it grasps the moment when all the coal flow on the conveyor belt is transported in real time; after all the coal on the conveyor belt is transported, the central control system issues an instruction to stop the end conveyor belt and switch. The time required for the conveyor belt to switch is; at this time, the conveyor belts of other sections do not need to stop running. The central control system calculates the time required for the head of the coal flow to reach a safe position at a certain distance from the end conveyor belt based on the running speed of the coal flow when the conveyor belt was running before; further, other conveyor belts do not need to stop running, and the head end of the conveyor belt does not need to wait for the end conveyor belt to switch before starting to transport coal. Instead, it can start the transportation of coal required for the next task in advance, saving time and cost waste caused by idle conveyor belts.

[0052] Example 2 The central control system mobilizes and controls the available resources of the ports involved in real time according to the task chain of "coal type - stack - reclaimer - conveyor belt - loader - ship" generated by the superior. Assuming that the superior receives multiple consecutive tasks, the equipment resources required are different. After receiving the tasks, the central control system first starts the conveyor belt from downstream to upstream according to the upstream and downstream relationship, after detecting the status of each device; then, after a task is completed, due to the different equipment resources required for the next task, the central control system determines that the terminal conveyor belt needs to be switched. After receiving the instruction from the superior, the start and stop action of the terminal conveyor belt is controlled in real time according to the start and stop time relationship of each device in the task. Then, the switching action is completed and the next task continues to be executed.

[0054] Example 3

[0055] The visual monitoring system receives real-time images captured by several smart cameras located above the conveyor belt; then, the visual monitoring system pre-processes the images, including image cutting, grayscale processing, Gaussian filtering, linear transformation and other operations; after obtaining the processed images, the visual monitoring system binarizes the images, sets black and white thresholds to distinguish between the coal flow part on the conveyor belt and other parts, and draws the outline of the coal flow on the original image in real time after the coal flow head passes the camera; at the same time, the visual detection system uses the optical flow method to calculate the speed of the image and obtain the real-time frame displacement of the image; through the coordinate transformation of the camera's internal and external parameters, the frame displacement of the image is converted into the displacement of the coal flow in the real world, thereby obtaining its real-time speed. After obtaining the real-time speed of the coal flow, the integral calculation can be started from the moment when the coal flow head passes under the camera, thereby obtaining the real-time position prediction of the coal flow head.

[0056] Example 4

[0057] See also Figure 2 , Figure 2 A schematic diagram of image transformation provided for this application.

[0058] Camera imaging is the process of a pinhole camera mapping the three-dimensional world into a two-dimensional picture. Therefore, it is necessary to establish a mapping relationship from a two-dimensional picture to the three-dimensional world. is the origin, is the pixel coordinate system, is the origin, is the image coordinate system, Any pixel in the image.

[0059] The conversion relationship between the pixel coordinate system and the image coordinate system is as follows:

[0060] That is: The camera coordinate system uses the x and y axes as coordinate axes, the Z axis as the optical axis, the origin O as the optical center of the camera, and the origin O and O j Distance OO j is the focal length f of the camera. Assume that the position of point A in the camera coordinate system is (x, y, z), and the image A′ in the image coordinate system is (J x ,J y ), we get the following relationship between the camera coordinate system and the image coordinate system: The above formula can be used to deduce the relationship between the pixel coordinate system and the physical coordinate system: Finally, the internal parameters of the camera are obtained and R x0 ,R y0. Assume that the plane of the object's motion is parallel to the image plane. A point A0(x0, y0, z0) moves to A1(x1, y1, z1). At this time, the corresponding point A′0(R x0 , R y0 ) on the image plane moves to A′1(R x1 , R y1 ). The following relationship can be obtained: Assume that the camera is perpendicular to the conveyor belt and the distance from the conveyor belt is z. Assume that the pixel position of the object translated on the image is (m, n). Then the actual moving distance ΔO is: In this way, the displacement of the pixel points on the image can be converted into the displacement of the actual coal flow, so as to obtain the real-time speed of the coal flow.

[0061] Embodiment 5

[0062] Refer to Figure 3 , Figure 3 , which is a schematic diagram of the relationship between the conveyor belt switching time and the task start time provided by this application. Assume that within a certain time period, the system receives two tasks and generates two task chains, which are: "Task 1: Reclaimer 1 - Conveyor Belt A - Conveyor Belt B - Conveyor Belt C - Shiploader 1" and "Task 2: Reclaimer 1 - Conveyor Belt A - Conveyor Belt B - Conveyor Belt D - Shiploader 2", where Conveyor Belt C is connected to Shiploader 1 and Conveyor Belt D is connected to Shiploader 2. After the system completes the first task, due to the change of the end shiploader equipment, an instruction needs to be issued to switch the original connection between Conveyor Belt B and Conveyor Belt C to the connection between Conveyor Belt B and Conveyor Belt D. Assume that the time required to complete this action is Δt0.

[0063] When Task 1 is about to end, the system issues an instruction to stop the coal supply of Reclaimer 1. Assume that this moment is t0. Through the visual monitoring system, it is predicted that all the coal required for Task 1 will be transported at t1. After receiving the message, the system issues an instruction to start the conveyor belt switching action at t2. The central control system sets that when the switching action is carried out, the head of the coal flow shall not exceed the safe position on Conveyor Belt B. The central control system starts the transportation of the coal required for Task 2 according to the previous information of the coal flow. Assume that the system commands Reclaimer 1 to start supplying coal at t3. The central control system predicts that the time required for the head of the coal flow to reach the set safe position on Conveyor Belt B is Δt1. Then the time relationship satisfies: t3 + Δt1 ≤ t2 + Δt0. In this way, if t3 > t1, the time when there is no coal on the conveyor belt is only t = t3 - t1. If t3 < t1, there is always coal being transported on the conveyor belt. Compared with before the improvement, the time when there is no coal on the conveyor belt is t = t2 + Δt0 - t1, which greatly reduces the idle running time of the conveyor belt and improves the work efficiency.

[0064] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A coal conveyor belt switching control method, comprising: Establish a PLC-based central control system to receive task requirements and generate task chains; The generated task chain is transmitted to the reclaimer and conveyor belt, and the start and stop of the reclaimer and conveyor belt are controlled according to the task chain to control coal transportation; Collect image data of coal flow running on conveyor belt; Preprocessing the collected image data; Binarization is performed on the pre-processed image data, and the coal flow image and the conveyor belt image are distinguished by setting a threshold value. Contour recognition is performed on the binary processed coal flow image data to extract the contour information of the coal flow. The optical flow method is used to process the image data of the continuous coal flow, the pixel displacement of the image data of the coal flow is calculated, and the pixel displacement is transformed into coordinates to obtain the running displacement and running speed of the coal flow on the conveyor belt; Predict the position of the head of the coal flow on the conveyor belt based on the profile information and running speed of the coal flow; When the next task requires switching the end conveyor belt, the central control system calculates the time t1 when all the coal flows of the current task on the conveyor belt are transported, based on the coal flow head position prediction information; At time t2 after time t1, after receiving the conveyor belt switching instruction, the central control system controls the conveyor belt at the end to stop running and performs the switching action, and the switching action lasts for δt0; The central control system calculates the time δt1 required for the head of the next task's coal flow to reach the preset safe position of the end conveyor belt according to the running speed of the coal flow; The central control system calculates the time t3 for starting the coal supply for the next task based on the duration of the switching action δt0 and the time δt1 required for the head of the coal flow of the next task to reach the preset safe position of the end conveyor belt, t3≤t2+δt0-δt1; At time t3, the central control system controls the reclaimer to start supplying coal for the next task and start transporting coal for the next task.

2. The coal conveyor belt switching control method according to claim 1 is characterized in that: Calculate the pixel displacement of the coal flow image, including: Extracting the current frame image according to the preprocessed image data of the coal flow; For each pixel of the current frame image, multiple pixels are selected in its neighborhood, and the optical flow equation is constructed based on the invariance of pixel grayscale and the local smoothness of the optical flow field. Solve the constructed optical flow equation by least squares to obtain the displacement of the current pixel in the horizontal and vertical directions; Repeat the steps of solving the displacement for all pixels of the current frame image to obtain the pixel displacement vector field of the entire image; The obtained pixel displacement vector field is output as the pixel displacement of the coal flow image in the current frame.

3. The coal conveyor belt switching control method according to claim 1, characterized in that: Obtain the running displacement of coal flow on the conveyor belt, including: According to the pinhole camera model, the mapping relationship between the pixel coordinate system, the image coordinate system and the camera coordinate system is established; The pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system through the intrinsic parameter matrix between the camera imaging plane and the image plane; The displacement vector field in the image coordinate system is converted to the camera coordinate system through the extrinsic parameter matrix between the camera coordinate system and the world coordinate system; According to the positional relationship between the camera coordinate system and the conveyor belt coordinate system, the displacement vector field in the camera coordinate system is converted to the conveyor belt coordinate system; The displacement vector field in the conveyor belt coordinate system is multiplied by the displacement time to obtain the running displacement of the coal flow on the conveyor belt.

4. The coal conveyor belt switching control method according to claim 3 is characterized in that: The pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system using the following formula: Among them, J x and J y are the image coordinate system; R x and R y is the pixel coordinate system; R x0 and R y0 is the coordinate of the reference point in the pixel coordinate system; Converted to matrix form: The relationship between the pixel coordinate system and the image coordinate system: The pixel coordinate system uses the x and y axes as coordinate axes, the z axis as the optical axis, the origin O as the optical center of the camera, and f as the focal length. The position of point A in the pixel coordinate system is (x, y, z), and the image A' in the image coordinate system is (J x ,J y ).

5. The coal conveyor belt switching control method according to claim 3 is characterized in that: The displacement vector field in the pixel coordinate system is converted to the camera coordinate system through the external parameter matrix between the camera coordinate system and the camera coordinate system: The camera coordinate system uses the x and y axes as coordinate axes and the Z axis as the optical axis; R x0 and R y0 is the coordinate of the reference point in the pixel coordinate system; R x and R y is the pixel coordinate system.

6. The coal conveyor belt switching control method according to claim 3 is characterized in that: According to the positional relationship between the camera coordinate system and the conveyor belt coordinate system, the displacement vector field in the camera coordinate system is transformed to the conveyor belt coordinate system. Among them, point A0 (x0, y0, z0) moves to A1 (x1, y1, z1), and the corresponding point A0' (R x0 ,R y0 )Move to A1'(R x1 ,R y1 ).

7. The coal conveyor belt switching control method according to claim 3 is characterized in that: Multiply the displacement vector field in the conveyor belt coordinate system by the displacement time to obtain the running displacement of the coal flow on the conveyor belt: The camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is z, the pixel position of the object translated on the image is (m,n), and the actual moving distance is ΔO; k x and k y Represents the coefficient.

8. The coal conveyor belt switching control method according to claim 1, characterized in that: The coal flow image and the conveyor belt image are distinguished by setting a threshold, including: Calculate the grayscale histogram of the image according to the preprocessed image data of the coal flow; According to the grayscale histogram, traverse all preset thresholds to divide the image pixels into background area and foreground area; For each threshold, the number of pixels, the average gray value of the background and foreground regions, and the average gray value of the entire image are calculated; Using the inter-class variance formula, calculate the inter-class variance between the background area and the foreground area; When the inter-class variance is the largest, the corresponding threshold is selected as the optimal threshold for image binarization; Using the obtained optimal threshold, the pixels in the coal flow image data whose grayscale values ​​are greater than or equal to the optimal threshold are set to 255, representing the coal flow foreground image, and vice versa, set to 0, representing the conveyor belt background image; Output the coal flow foreground image data and conveyor belt background image data after binary processing.

9. The coal conveyor belt switching control method according to claim 1, characterized in that: Perform contour recognition on the binary processed coal flow image data to extract the contour information of the coal flow, including: According to the obtained coal flow foreground image data, the image is smoothed by sampling Gaussian filter; Calculate the gradient magnitude and direction of the smoothed image to obtain a gradient magnitude image and a gradient direction image; Perform non-maximum suppression on the gradient magnitude image; The double threshold method is used to perform threshold processing on the image after non-maximum suppression, and the edge pixels are connected to obtain the edge image; Perform contour tracing on the edge image and extract the connected edge contour pixel coordinates; The extracted edge contour pixel coordinates are output as coal flow contour information.

10. The coal conveyor belt switching control method according to claim 1, characterized in that: According to the profile information and running speed of the coal flow, the position of the head of the coal flow on the conveyor belt is predicted, including: According to the obtained coal flow contour information, the position coordinates of the coal flow head in the current image frame are marked; The marked coal flow head position coordinates are used as the observation values ​​of the Kalman filter algorithm to establish the observation equation; According to the historical position coordinates and current position coordinates of the coal flow head, the state equation of the coal flow head position is established by using the uniform motion model; The coal flow head position is predicted using the state equation to obtain the predicted position coordinates and the prediction error covariance matrix; The prediction results are updated using the observation equation to obtain the updated coal flow head position coordinates and the updated error covariance matrix; The obtained position coordinates of the coal flow head are output as the position of the coal flow head in the image at the current moment; The obtained coal flow head position coordinates and error covariance matrix are used as the initial values ​​of the Kalman filter algorithm for the next frame of image; Repeat the above steps to predict and update the position coordinates of the coal flow head on the conveyor belt.

Images