Coal conveying belt switching control method

Abstract

The application discloses a coal conveying belt switching control method, and belongs to the field of automatic control, which comprises the following steps: controlling the start and stop of a reclaimer and a conveying belt according to a task chain; collecting image data of coal flow running on the conveying belt; distinguishing the coal flow image and the conveying belt image by setting a threshold value, and performing contour recognition on the image data of the coal flow; calculating the pixel displacement of the image data of the coal flow by using an optical flow method, performing coordinate conversion on the pixel displacement, and obtaining the running displacement and running speed of the coal flow on the conveying belt; predicting the position of the head of the coal flow on the conveying belt according to the contour information and the running speed of the coal flow; when the next task needs to perform a switching action on the terminal conveying belt, a central control system calculates the time t1 when the coal flow of the current task on the conveying belt is completely conveyed according to the position prediction information of the head of the coal flow, and controls the switching of the conveying belt; and the coal conveying efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation control, more particularly, to a coal conveying belt switching control method. BACKGROUND

[0002] The coal port is an important hub connecting railway coal conveying and marine coal conveying, which is composed of multiple work areas, and the coal conveying belt is an important equipment connecting various work areas. The main work of the coal port is roughly as follows: coal is unloaded from the train, transported to the stacking area through the conveying belt, and then transported to the ship loader through the conveying belt by the reclaimer, and then loaded into the ship. The conveying belt is an important component of almost all work tasks, and the operation of the conveying belt directly affects the work efficiency of the coal port task.

[0003] The coal conveying belt is a large-scale transportation system composed of multiple conveying belts, with a length of hundreds of meters or even more than a kilometer. In the work task, multiple conveying belts are connected with other large equipment in different work areas to transfer coal. Due to multiple work equipment and work tasks in each work area, the multiple conveying belts may switch the connection with the corresponding conveying belt and work equipment according to the service object of the work task, thereby smoothly completing the conversion of different work tasks. For example, coal is taken out from the stacking area by the reclaimer, transported to the ship loader by the conveying belt, and loaded into the ship for the task of ship loading. After completing a task, the head or tail part of the multiple conveying belts may need to perform a switching task to connect with different equipment due to different coal types required by the ship, different types of ships, etc.

[0004] In the current production scene, there is much room for improvement in the control of the conveying belt. From the perspective of safety production, first, the conveying belt has a strict upstream and downstream relationship, i.e., once the downstream belt stops, the upstream belt will immediately stop; only after the downstream belt starts can the upstream belt start. Second, when the conveying belt performs a switching action, it needs to completely stop the operation of all conveying belts, thereby causing unnecessary delays and repeated start-stop of some conveying belts.

[0005] From the perspective of equipment control, many production scenes lack control linkage for the conveying belt, and usually only remote control switching and other behaviors can be achieved, and specific task deployment and equipment operation still need a lot of manpower and resources to execute; in many existing work scheduling systems containing conveying belts, only a simple logic of "prohibiting different task work equipment from sharing" is usually set, resulting in the need to stop the operation of all conveying belts to switch different work of terminal equipment. SUMMARY

[0006] 1. Technical problem to be solved

[0007] In view of the low coal conveying efficiency in the prior art, the application provides a coal conveying belt switching control method, a central control system based on PLC is used to receive task requirements and generate a task chain, a reclaimer and a conveying belt are controlled to convey coal; image data of coal flow running on the conveying belt is collected, and after preprocessing, binarization, contour recognition and other treatments, the optical flow method is used to calculate the running displacement and speed of the coal flow on the conveying belt; the position of the head of the coal flow on the conveying belt is further predicted, and the central control system reasonably arranges the time of conveying belt switching and next task coal supply according to the position prediction information of the head of the coal flow, so that intelligent switching control of the coal conveying belt is realized, and the coal conveying efficiency is improved.

[0008] 2. Technical solution

[0009] The purpose of the application is achieved by the following technical solutions.

[0010] The present application provides a coal conveying belt switching control method, which comprises: establishing a central control system based on PLC, receiving task requirements and generating a task chain; transmitting the generated task chain to a reclaimer and a conveying belt, and controlling the start and stop of the reclaimer and the conveying belt according to the task chain to control coal conveying; collecting image data of coal flow running on the conveying belt; preprocessing the collected image data; binarizing the preprocessed image data, distinguishing the coal flow image and the conveying belt image by setting a threshold, and performing contour recognition on the binarized image data of the coal flow to extract the contour information of the coal flow; processing the image data of the continuous coal flow by using the optical flow method, calculating the pixel displacement of the image data of the coal flow, performing coordinate conversion on the pixel displacement, obtaining the running displacement and speed of the coal flow on the conveying belt; predicting the position of the head of the coal flow on the conveying belt according to the contour information and running speed of the coal flow; when the next task needs to perform a switching action on the terminal conveying belt, the central control system calculates the time t1 when the coal flow of the current task on the conveying belt is completely conveyed according to the position prediction information of the head of the coal flow; at time t2 after time t1, after the central control system receives the conveying belt switching instruction, the terminal conveying belt is controlled to stop running, and the switching action is performed, and the duration of the switching action is δt0; the central control system calculates the time δt1 required for the head of the coal flow of the next task to reach the preset safe position of the terminal conveying belt according to the running speed of the coal flow; the central control system calculates the time t3 of starting the next task coal supply according to the duration δt0 of the switching action and the time δt1 required for the head of the coal flow of the next task to reach the preset safe position of the terminal conveying belt, t3≤t2+δt0-δt1; at time t3, the central control system controls the reclaimer to start the coal supply of the next task, and starts the coal conveying of the next task.

[0011] PLC stands for Programmable Logic Controller, which is a digital electronic device designed specifically for industrial control. Binary processing: a common method in image processing, which divides the pixel values in the image into two categories according to the set threshold. Generally, pixels above the threshold are set to white (255), and pixels below the threshold are set to black (0), thus converting the original grayscale image into a binary image containing only black and white colors. Optical flow method: a method for estimating the motion of objects in image sequences. It is based on the assumption of gray-scale invariance, that is, the gray-scale values of the same object remain unchanged between different frames. By calculating the spatial and temporal gradients of pixel gray-scale values in the image sequence, the motion displacement vector of the pixel between adjacent frames is estimated, and the motion information of the entire image is obtained. Coal flow head: refers to the frontmost part of the coal flow on the conveyor belt during the conveying process. Accurate identification and tracking of the position of the coal flow head is of great significance for realizing the automatic switching control of the coal conveying belt. By predicting the time when the coal flow head reaches the end of the conveyor belt, the time for switching the conveyor belt and supplying the next task coal can be reasonably arranged, avoiding problems such as interruption of coal conveying or coal overflow. Pixel displacement: refers to the spatial position change between corresponding pixel points of the same object in adjacent frames of an image sequence. By calculating the displacement of image pixels in the horizontal and vertical directions, pixel-level motion information can be obtained, and the motion speed and trajectory of the object can be estimated. In the control of the switching of the coal conveying belt, by calculating the pixel displacement of the coal flow image and converting it into actual physical displacement, the running displacement and speed of the coal flow on the conveyor belt can be obtained.

[0012] Preferably, the image preprocessing includes: image cutting on the received image data of the coal flow to extract the image of the target area; gray-scale processing on the cut image to convert the RGB image into a grayscale image; Gaussian filtering on the gray-scale processed image to remove noise in the image; linear transformation on the Gaussian filtered image to adjust the contrast and brightness of the image, and the linear transformed image is taken as the preprocessed image data of the coal flow.

[0013] Further, the pixel displacement of the coal flow image is calculated, including: extracting the current frame image according to the preprocessed image data of the coal flow; selecting multiple pixel points in the neighborhood of each pixel of the current frame image, constructing an optical flow equation based on the invariance of pixel gray-scale and the local smoothness of the optical flow field; solving the constructed optical flow equation by least squares to obtain the displacement of the current pixel in the horizontal and vertical directions; repeating the step of solving the displacement for all pixels of the current frame image to obtain the pixel displacement vector field of the entire image; and outputting the obtained pixel displacement vector field as the pixel displacement of the coal flow image in the current frame.

[0014] Specifically, according to the image data of the pre-processed coal flow, the current frame image I(x, y, t) and the previous frame image I(x, y, t-1) are extracted, where x and y are pixel coordinates, and t is time. For each pixel (x, y) of the current frame image, a plurality of pixel points are selected in its neighborhood. Assuming that the displacement amounts of the pixel (x, y) in the horizontal and vertical directions are u and v respectively, the pixel point coordinates corresponding to the pixel point (x+dx, y+dy) in the previous frame image are (x+dx-u, y+dy-v). According to the gray invariance assumption, that is, the gray value of the same object remains unchanged between adjacent frames, a constraint equation can be established: I(x, y, t) = I(x-u, y-v, t-1). A first-order Taylor expansion is performed on the above equation at (x, y) to obtain: I(x, y, t) = I(x, y, t-1)-Ix(x, y, t-1)u-Iy(x, y, t-1)v+It(x, y, t-1), where Ix, Iy and It are the partial derivatives of the image in the x, y and t directions respectively. In combination with the local smoothness assumption of the optical flow field, that is, adjacent pixels have similar motion trends, a plurality of pixel points are selected in the neighborhood to construct an optical flow equation group: Ix(x1, y1, t-1)u+Iy(x1, y1, t-1)v = -It(x1, y1, t-1), Ix(x2, y2, t-1)u+Iy(x2, y2, t-1)v = -It(x2, y2, t-1), …, Ix(xn, yn, t-1)u+Iy(xn, yn, t-1)v = -It(xn, yn, t-1). The least squares solution of the constructed optical flow equation group is obtained, that is, the following optimization problem is solved: min∑[Ix(xi, yi, t-1)u+Iy(xi, yi, t-1)v+It(xi, yi, t-1)]^2. The optimization problem can be solved to obtain the displacement amounts u and v of the current pixel (x, y) in the horizontal and vertical directions. Steps 2-5 are repeated for all pixels of the current frame image to obtain the pixel displacement vector field (u(x, y), v(x, y)) of the entire image. The obtained pixel displacement vector field is taken as the pixel displacement output of the coal flow image in the current frame, which is represented as: (du(x, y), dv(x, y)) = (u(x, y), v(x, y)). Through the above steps, the pixel displacement vector field of the coal flow image between consecutive frames can be calculated by using the optical flow method. The displacement vector field reflects the motion of the coal flow on the image plane, and provides important information for subsequent coal flow velocity estimation and position prediction.

[0015] Further, the running displacement of the coal flow on the conveying belt is obtained by: establishing a mapping relationship among a pixel coordinate system, an image coordinate system and a camera coordinate system according to a pinhole camera model; converting a pixel displacement vector field in the pixel coordinate system to the image coordinate system through an internal parameter matrix between an imaging plane of the camera and an image plane; converting the displacement vector field in the image coordinate system to the camera coordinate system through an external parameter matrix between a camera coordinate system and a world coordinate system; converting the displacement vector field in the camera coordinate system to a conveying belt coordinate system according to a positional relationship between the camera coordinate system and the conveying belt coordinate system; and multiplying the displacement vector field in the conveying belt coordinate system by a displacement time to obtain the running displacement of the coal flow on the conveying belt.

[0016] Further, the pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system by the following formula:

[0017] wherein J x ,J y is the image coordinate system; R x ,R y is the pixel coordinate system; Rx0, Ry0 is the coordinate of the reference point in the pixel coordinate system; and the conversion to a matrix form is as follows: Relationship between the pixel coordinate system and the image coordinate system: wherein the pixel coordinate system takes x, y axes as coordinate axes, Z axis as an optical axis, and the origin O as the optical center of the camera; the distance OO j between the origin O and O j is the focal length f of the camera; and the position of point A in the pixel coordinate system is (x, y, z), and the imaging A' in the image coordinate system is (J x ,J y ).

[0018] Further, the displacement vector field in the pixel coordinate system is converted to the camera coordinate system through an external parameter matrix between the camera coordinate system and the camera coordinate system: wherein the camera coordinate system takes x, y axes as coordinate axes, and Z axis as an optical axis; R x0 and R y0 are the coordinates of the reference point in the pixel coordinate system; and R x ,R y is the pixel coordinate system.

[0019] Further, the displacement vector field in the camera coordinate system is converted to the conveying belt coordinate system according to the positional relationship between the camera coordinate system and the conveying belt coordinate system:

[0020]

[0021] Wherein, the camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is z, the pixel position of the object on the image is (m, n), and the actual moving distance is ΔO;

[0022] Further, the displacement vector field under the conveyor belt coordinate system is multiplied by the displacement time to obtain the running displacement of the coal flow on the conveyor belt: Wherein, the camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is z, the pixel position of the object on the image is (m, n), and the actual moving distance is ΔO;

[0023] Further, the threshold is set to distinguish the coal flow image and the conveyor belt image, including: calculating the gray histogram of the image according to the preprocessed image data of the coal flow; according to the gray histogram, all preset thresholds are traversed, and the image pixels are divided into background area and foreground area; for each threshold, the pixel number, the average gray value of the background area and the foreground area, and the average gray value of the whole image are calculated; the inter-class variance between the background area and the foreground area is calculated by using the inter-class variance formula; the threshold corresponding to the maximum inter-class variance is selected as the optimal threshold for image binarization; the optimal threshold is used to set the pixels with a gray value greater than or equal to the optimal threshold in the image data of the coal flow to 255, representing the coal flow foreground image, and vice versa, representing the conveyor belt background image; output the coal flow foreground image data and the conveyor belt background image data after the binarization processing.

[0024] Specifically, according to the image data of the pretreated coal flow, a gray histogram of the image is calculated. Assuming that the gray range of the image is [0, 255], the gray histogram can be represented as a 256-dimensional vector H(i), where i is the gray value, and H(i) is the number of pixels with the gray value i. According to the gray histogram, all preset thresholds T (with a value range of [0, 255]) are traversed, and the image pixels are divided into a background region and a foreground region. For each threshold T, the pixels with a gray value less than T belong to the background region, and the pixels with a gray value greater than or equal to T belong to the foreground region. For each threshold T, the number of pixels, the average gray value of the background region and the foreground region, and the average gray value of the entire image are calculated. The number of background region pixels: w0 = ∑[i=0 to T-1]H(i) / ∑[i=0 to 255]H(i); the number of foreground region pixels: w1 = ∑[i=T to 255]H(i) / ∑[i=0 to 255]H(i); the average gray value of the background region: u0 = ∑[i=0 to T-1]i*H(i) / (w0*∑[i=0 to 255]H(i)); the average gray value of the foreground region: u1 = ∑[i=T to 255]i*H(i) / (w1*∑[i=0 to 255]H(i)); the average gray value of the entire image: u = ∑[i=0 to 255]i*H(i) / ∑[i=0 to 255]H(i); the inter-class variance between the background region and the foreground region is calculated by using an inter-class variance formula: g = w0*w1*(u0-u1)^2; the threshold corresponding to the maximum inter-class variance is selected as the optimal threshold T_opt for image binarization: T_opt = argmax(g); the optimal threshold T_opt is used to set the pixels with a gray value greater than or equal to T_opt in the image data of the coal flow to 255, representing the foreground image of the coal flow, and set the pixels to 0, representing the background image of the conveyor belt.

[0025] For each pixel (x, y) in the image, the binarization result is: if I(x, y) >= T_opt, then I_binary(x, y) = 255, if I(x, y) < T_opt, then I_binary(x, y) = 0, output the foreground image data I_foreground and the background image data I_background of the coal flow after binarization processing: I_foreground(x, y) = I_binary(x, y), where I_binary(x, y) = 255, I_background(x, y) = I_binary(x, y), where I_binary(x, y) = 0, through the above steps, the optimal threshold can be adaptively selected by using the inter-class variance method, and the binarization segmentation of the coal flow image and the conveyor belt image is realized. The basic idea of the inter-class variance method is to maximize the inter-class variance between the background region and the foreground region when selecting the threshold, so that the separation degree of the two regions is the highest.

[0026] Further, the image data of the coal flow after binarization processing is subjected to contour recognition, and the contour information of the coal flow is extracted, including: according to the obtained foreground image data of the coal flow, a Gaussian filter is used to smooth the image; the gradient amplitude and direction of the smoothed image are calculated to obtain a gradient amplitude image and a gradient direction image; the non-maximum suppression processing is performed on the gradient amplitude image; the double-threshold method is used to perform thresholding processing on the non-maximum suppressed image, and the edge pixels are connected to obtain an edge image; the edge image is subjected to contour tracking, and the connected edge contour pixel coordinates are extracted; and the extracted edge contour pixel coordinates are output as the coal flow contour information.

[0027] Specifically, according to the obtained foreground image data I_foreground of the coal flow, a Gaussian filter is used to smooth the image. The kernel function of the Gaussian filter is: G(x, y) = (1 / (2piσ^2))*exp(-(x^2+y^2) / (2*σ^2)), wherein σ is the standard deviation of the Gaussian filter, which controls the smoothing degree. The Gaussian filter is convolved with the foreground image to obtain the smoothed image I_smooth: The gradient amplitude and direction of the smoothed image I_smooth are calculated to obtain a gradient amplitude image I_mag and a gradient direction image I_dir. The gradients of the image in the x direction and the y direction are calculated respectively: Calculate gradient magnitude: I mag = sqrt(I dx^2 + I dy^2), and gradient direction: I dir = arctan(I dy / I dx). Perform non-maximum suppression on the gradient magnitude image I mag. For each pixel (x, y), check the gradient magnitude of its two neighboring pixels in the gradient direction. If the gradient magnitude of the current pixel is not the maximum, set it to 0 to obtain the non-maximum suppression image I nms.

[0028] Perform thresholding on the non-maximum suppression image I nms using a double thresholding method, and connect the edge pixels to obtain the edge image I edge. Set a high threshold T high and a low threshold T low, where T high = k*T low, and k is a constant factor. For each pixel (x, y) in I nms: if its gradient magnitude is greater than the high threshold T high, mark it as a strong edge pixel, and I edge(x, y) = 255. If its gradient magnitude is less than the low threshold T low, mark it as a non-edge pixel, and I edge(x, y) = 0. If its gradient magnitude is between the high and low thresholds, mark it as a weak edge pixel, and I edge(x, y) = 128. For all weak edge pixels, check whether there is a strong edge pixel in their 8-neighborhood. If there is, mark it as a strong edge pixel, otherwise mark it as a non-edge pixel. Perform contour tracing on the edge image I edge to extract connected edge contour pixel coordinates. Scan the edge image from the top-left corner to find the first strong edge pixel as the starting point. From the starting point, check its 8-neighborhood pixels in a clockwise direction to find the next strong edge pixel, and add its coordinates to the contour point set. Repeat the above step until returning to the starting point to obtain a closed contour. Continue scanning the edge image to find the next unvisited strong edge pixel, and repeat the above process until all strong edge pixels are visited. Output the extracted edge contour pixel coordinates as coal flow contour information as a contour point set C = {p1, p2,..., pn}, where pi = (xi, yi) is the contour point coordinate. Through the above steps, the Canny edge detection operator can be used to perform contour recognition on the binarized coal flow image to extract the contour information of the coal flow. The Canny operator can effectively extract the true edges in the image through gradient calculation, non-maximum suppression, and double thresholding connection, and has good positioning accuracy and continuity.

[0029] Further, according to the profile information and the running speed of the coal flow, the position of the coal flow head on the conveyor belt is predicted, including: according to the obtained profile information of the coal flow, marking the position coordinates of the coal flow head in the current image frame; taking the marked position coordinates of the coal flow head as the observation value of the Kalman filtering algorithm, and establishing an observation equation; according to the historical position coordinates and the current position coordinates of the coal flow head, adopting a uniform motion model to establish a state equation of the position of the coal flow head; using the state equation to predict the position of the coal flow head, obtaining the predicted position coordinates and a predicted error covariance matrix; using the observation equation to update the prediction result, obtaining the updated position coordinates of the coal flow head and an updated error covariance matrix; taking the obtained position coordinates of the coal flow head as the position of the coal flow head in the image at the current time, and outputting; taking the obtained position coordinates of the coal flow head and the error covariance matrix as the initial value of the Kalman filtering algorithm of the next frame of image; repeating the above steps to predict and update the position coordinates of the coal flow head on the conveyor belt.

[0030] Specifically, according to the obtained coal flow contour information C={p1, p2,..., pn}, the position coordinates of the coal flow head in the current image frame are marked. Assuming that the movement direction of the coal flow is from top to bottom, the point with the smallest vertical coordinate in the contour point set C is the position of the coal flow head. Find the point ph=(xh, yh) with the smallest vertical coordinate in the contour point set C, and mark it as the position coordinates of the coal flow head. Mark the position coordinates of the coal flow head ph as the observation value of the Kalman filtering algorithm, and establish the observation equation. Define the observation value matrix Z=[xh, yh]', which represents the position of the coal flow head in the image. Define the observation matrix H=[1, 0, 0, 0; 0, 1, 0, 0], which associates the state variable with the observation value. The observation equation is established: Z=H*X+V, where X is the state variable matrix, and V is the observation noise matrix. According to the historical position coordinates and the current position coordinates of the coal flow head, a uniform motion model is used to establish the state equation of the coal flow head position. Define the state variable matrix X=[x, y, vx, vy]', where (x, y) is the position coordinates of the coal flow head, and (vx, vy) is the velocity component of the coal flow head. Define the state transition matrix A=[1, 0, dt, 0; 0, 1, 0, dt; 0, 0, 1, 0; 0, 0, 0, 1], where dt is the time interval between adjacent frames. The state equation is established: X(k+1)=A*X(k)+W(k), where k is the time step, and W is the process noise matrix. The state equation is used to predict the position of the coal flow head, and the predicted position coordinates and the predicted error covariance matrix are obtained. According to the state estimation value X(k|k) at the last time and the state transition matrix A, the state prediction value X(k+1|k) at the current time is calculated: X(k+1|k)=A*X(k|k), and according to the error covariance matrix P(k|k) at the last time and the process noise covariance matrix Q, the prediction error covariance matrix P(k+1|k) at the current time is calculated: P(k+1|k)=A*P(k|k)*A'+Q. The prediction result is updated using the observation equation to obtain the updated coal flow head position coordinates and the updated error covariance matrix. The Kalman gain matrix K(k+1) is calculated: K(k+1)=P(k+1|k)H' / (HP(k+1|k)*H'+R), where R is the observation noise covariance matrix. According to the observation value Z(k+1) and the predicted value X(k+1|k), the updated state estimation value X(k+1|k+1) is calculated: X(k+1|k+1)=X(k+1|k)+K(k+1)(Z(k+1)-HX(k+1|k)), and the updated error covariance matrix P(k+1|k+1) is calculated: P(k+1|k+1)=(I-K(k+1)*H)*P(k+1|k). The obtained coal flow head position coordinates (x, y) are output as the position of the coal flow head in the image at the current time.The obtained coal stream head position coordinates (x, y) and error covariance matrix P(k+1|k+1) are taken as initial values of Kalman filtering algorithm of the next frame image. The above steps are repeated to predict and update the position coordinates of the coal stream head on the conveyor belt.

[0031] 3. Beneficial effects

[0032] Compared with the prior art, the application has the advantages that:

[0033] By establishing a central control system based on PLC, receiving task requirements and generating a task chain, and transmitting the task chain to the reclaimer and the conveyor belt for coal transportation control, the automation control of the coal conveying process is realized, the intervention of manual operation is reduced, and the coal conveying efficiency is improved.

[0034] Image data of the coal stream running on the conveyor belt is collected, and the image data is preprocessed, binarized, and contour recognized to extract the contour information of the coal stream. The running displacement and speed of the coal stream on the conveyor belt are calculated by combining the optical flow method, providing an accurate data basis for subsequent prediction of the position of the coal stream head.

[0035] According to the contour information and running speed of the coal stream, the position of the coal stream head on the conveyor belt is predicted by using the Kalman filtering algorithm. When the conveyor belt needs to be switched, the central control system can accurately calculate the time when the current task coal conveying is completed according to the prediction information of the position of the coal stream head, reasonably arrange the time of conveyor belt switching and the next task coal supply, and avoid the problems of interruption of coal conveying or coal overflow.

[0036] During the conveyor belt switching process, the starting time of the next task coal supply is reasonably controlled by calculating the duration of the conveyor belt switching action and the time required for the head of the next task coal stream to reach a safe position. Both the completion of the conveyor belt switching action and the delay of the coal supply after the switching are avoided, the time of coal conveying task switching is shortened, and the coal conveying efficiency is improved.

[0037] Image preprocessing includes image cutting, grayscale, Gaussian filtering, linear transformation and other steps, effectively removing image noise, enhancing image contrast and brightness, providing high-quality image data for subsequent image processing, and improving the accuracy of coal stream contour information extraction and displacement calculation.

[0038] The optical flow method is used to calculate the pixel displacement of the coal stream image. By constructing the optical flow equation and solving it, the displacement of the image pixels in the horizontal and vertical directions is obtained, and the pixel displacement vector field of the entire image is generated. Combined with the pinhole camera model and the coordinate conversion relationship, the pixel displacement is converted to the actual running displacement of the coal stream on the conveyor belt, improving the accuracy of the coal stream speed calculation.

[0039] The image is binarized by setting a threshold, and the optimal threshold is adaptively selected by using the inter-class variance method, so that the foreground image of the coal flow and the background image of the conveying belt can be effectively distinguished, and convenience is provided for subsequent coal flow contour recognition.

[0040] The edge detection and contour tracking are performed on the binarized image, and the contour pixel coordinates of the coal flow are extracted, so that the contour information of the coal flow is accurately described, and reliable data support is provided for the prediction of the head position of the coal flow.

[0041] In the prediction of the head position of the coal flow, the Kalman filtering algorithm is adopted, the state equation and the observation equation of the head position of the coal flow are established, the head position of the coal flow is recursively predicted and updated, the historical position information of the head of the coal flow is fully utilized, and the prediction accuracy and robustness of the position are improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The schematic diagram of the overall process of the system provided by the present application is provided.

[0043] Figure 2 The schematic diagram of the image transformation provided by the embodiment of the present application is provided.

[0044] Figure 3 The schematic diagram of the relationship between the switching time of the conveying belt and the start time of the task provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0045] The present application will be described in detail below in combination with the drawings and specific examples in the specification.

[0046] Example 1

[0047] Reference Figure 1 , Figure 1 The schematic diagram of the overall process of the system provided by the present application is provided.

[0048] Firstly, the central control system of the present application receives a series of task requirements from the superior, and these task requirements generally have similar forms, such as the type of the ship from the port, the type of coal required by the ship, etc. The superior integrates these information, generates a task chain composed of coal type, stacking position, reclaimer, conveying belt, ship loader, ship, according to the current equipment resource situation of the port; at the same time, the central control system performs opening, closing and other operations on the multiple conveying belts according to the required equipment in the task chain, to ensure the smooth progress of the task.

[0049] Secondly, the visual monitoring system designed by the application uses an intelligent camera to monitor the coal transportation on the conveyor belt in real time, capture the profile, speed and other information of the coal flow, after image preprocessing, use the optical flow method to process the image of the coal flow in real time to obtain the image frame displacement of the coal flow; then, according to the internal and external parameters of the camera, the image frame displacement of the coal flow is obtained through coordinate transformation to obtain the displacement of the real world coal flow, so as to obtain the real-time speed. Combining the profile, real-time speed and other information of the coal flow, the real-time position of the head of the coal flow is predicted, and these information is visually transmitted to the central control system to realize real-time control of the coal flow information on the conveyor belt.

[0050] Finally, the application designs a complete problem solving implementation method for the switching scene of the coal conveyor belt due to the task needs.

[0051] Taking a typical switching scene as an example: assuming that the ship model of the next task changes, a different ship loader needs to be used, that is, the conveyor belt connected with the ship loader needs to be switched to adapt to the change of the task. When the previous task is about to be completed, the central control system has mastered the real-time information of the coal flow through the visual monitoring system, and the time when the coal flow on the conveyor belt is completely transported is mastered in real time after the coal transportation at the head end of the conveyor belt is stopped; after the coal on the conveyor belt is completely transported, the central control system issues an instruction to stop the operation of the tail end conveyor belt and perform switching action; the time required for the switching action of the conveyor belt is; at this time, the other sections of the conveyor belt do not need to stop running, and the central control system calculates the time required for the head of the coal flow to reach a safe position away from the tail end conveyor belt according to the running speed of the coal flow during the running of the conveyor belt; further, the other conveyors do not need to stop running, and the head end of the conveyor belt does not need to wait for the completion of the switching action of the tail end conveyor belt before starting to transport coal, but can start the transportation of the coal required for the next task in advance, saving the time and cost waste caused by the idle of the conveyor belt.

[0052] Embodiment 2

[0053] The central control system mobilizes and controls the available resources of the port in real time according to the task chain generated by the upper level, which has "coal type - stack position - reclaimer - conveyor belt - ship loader - ship". Assuming that the upper level receives multiple consecutive tasks, in which the required equipment resources are different, after receiving the task, the central control system first starts the conveyor belt from downstream to upstream according to the upstream and downstream relationship after detecting the conditions of each device; then, after the completion of a task, since the equipment resources required for the next task are different, the central control system judges that the tail end conveyor belt needs to perform switching action, and after receiving the instruction from the upper level, controls the start and stop actions of the tail end conveyor belt in real time according to the start and stop time relationship of each device in the task. Then, the switching action is completed, and the next task continues to be executed.

[0054] Example 3

[0055] The visual monitoring system receives real-time images captured by several smart cameras located above the conveyor belt. The system then preprocesses the images, including image segmentation, grayscale processing, Gaussian filtering, and linear transformation. After obtaining the processed images, the system binarizes them, setting black-and-white thresholds to distinguish between parts of the coal flow on the conveyor belt and other parts, and draws the outline of the coal flow in real-time on the original image after the head of the coal flow passes the camera. Simultaneously, the system calculates the velocity of the image using optical flow, obtaining the real-time frame displacement. Through coordinate transformation of the camera's intrinsic and extrinsic parameters, the frame displacement of the image is converted into the displacement of the coal flow in the real world, thus obtaining its real-time velocity. After obtaining the real-time velocity of the coal flow, integration calculations can be performed starting from the moment the head of the coal flow passes below the camera, thereby predicting the real-time position of the head of the coal flow.

[0056] Example 4

[0057] See Figure 2 , Figure 2 A schematic diagram of the image transformation provided in this application.

[0058] Camera imaging is the process by which a pinhole camera maps the three-dimensional world onto a two-dimensional image. Therefore, it is necessary to establish a mapping relationship from a two-dimensional image to the three-dimensional world. With the origin as the point, Using pixel coordinates, With the origin as the point, For the image coordinate system, Let be any pixel in the image.

[0059] The transformation relationship between pixel coordinate system and image coordinate system is as follows.

[0060] That is: The camera coordinate system uses the x and y axes as coordinate axes, the Z axis as the optical axis, and the origin O as the optical center of the camera. The origin O and O j Distance OO j This is the camera's focal length f. Assuming point A's position in the camera coordinate system is (x, y, z), its image A' in the image coordinate system is (J...). x J y The following relationship between the camera coordinate system and the image coordinate system is obtained: From the above formula, we can deduce the relationship between the pixel coordinate system and the physical coordinate system: Finally, the camera's internal parameters are obtained. and R x0 ,R y0Suppose the plane of the object movement is parallel to the image plane, a point A0(x0, y0, z0) moves to A1(x1, y1, z1), at this time the corresponding point A'0(R x0 ,R y0 ) of the image plane moves to A'1(R x1 ,R y1 ). The relationship can be obtained: Suppose the camera is perpendicular to the conveyor belt, and the distance from the camera to the conveyor belt is z, and the pixel position of the object translation on the image is (m, n), then the actual moving distance ΔO is: Thus the displacement of the pixel point on the image can be converted into the displacement of the actual coal flow, so as to obtain the real-time speed of the coal flow.

[0061] Embodiment 5

[0062] Referring to Figure 3 , Figure 3 the schematic diagram of the relationship between the switching time of the conveyor belt and the task start time provided by the present application. Suppose that within a certain time period, the system receives two tasks, generating two task chains: "Task 1: reclaimer 1-conveyor belt A-conveyor belt B-conveyor belt C-ship loader 1" and "Task 2: reclaimer 1-conveyor belt A-conveyor belt B-conveyor belt D-ship loader 2", wherein conveyor belt C is connected to ship loader 1, and conveyor belt D is connected to ship loader 2. After completing the first task, the system needs to issue an instruction to switch the original connection between conveyor belt B and conveyor belt C to the connection between conveyor belt B and conveyor belt D due to the change of the end ship loader device. It is assumed that the time required for this action is Δt0.

[0063] Task 1 is about to end, and the system issues an instruction for the reclaimer 1 to stop supplying coal, assuming that the time is t0 at this time. The vision monitoring system predicts that the coal required for task 1 will be completely transported at time t1. After receiving the message, the system issues an instruction to start the switching action of the conveyor belt at time t2, and the central control system sets that the head of the coal flow should not exceed the safe position on conveyor belt B during the switching action. The central control system starts the transportation of the coal required for task 2 according to the information of the coal flow before. It is assumed that the system instructs the reclaimer 1 to start supplying coal at time t3, and the central control system predicts that the time required for the head of the coal flow to reach the set safe position on conveyor belt B is Δt1. Therefore, the time relationship satisfies: t3+Δt1≤t2+Δt0, so if t3>t1, the time during which the conveyor belt is completely without coal is only t=t3-t1, and if t3

[0064] The above description of the application and its embodiments is illustrative, and not restrictive. Many variations of the application will become apparent to those of ordinary skill in the art upon review of the foregoing description. The only true limitation being set forth by the claims, the application can be practiced in other specific ways which are not expressly disclosed herein. Any and all such variations are intended to be included within the scope of the application. The use of the terms "include," "includes," "including," "have," "has," or "having" has the generic meaning as "comprise," "comprises," or "comprising," and does not exclude other elements or steps. The use of the terms "a" or "an" does not exclude a plurality, and "one," "single," or "the only" means exactly one. The use of the term "one" or "only one" does not exclude other elements or steps. The use of the terms "first," "second," "third," or "fourth" does not limit the number of elements or steps to these three or four, but rather these terms are used to distinguish these elements or steps from one another. The use of the term "about" means that the value includes the stated value plus or minus ten percent.

Claims

1. A coal conveying belt switching control method, comprising: establishing a PLC-based central control system to receive task requirements and generate a task chain; transmitting the generated task chain to a reclaimer and a conveying belt, and controlling the start and stop of the reclaimer and the conveying belt according to the task chain to perform coal conveying control; collecting image data of the coal flow running on the conveying belt; preprocessing the collected image data; performing binaryzation processing on the preprocessed image data, distinguishing the coal flow image and the conveying belt image by setting a threshold, and performing contour recognition on the binaryzation-processed image data of the coal flow to extract contour information of the coal flow; processing the image data of the continuous coal flow by using an optical flow method to calculate pixel displacement of the image data of the coal flow, performing coordinate conversion on the pixel displacement to obtain running displacement and running speed of the coal flow on the conveying belt; predicting the position of the head of the coal flow on the conveying belt according to the contour information and the running speed of the coal flow; When the next task needs to switch the end conveyor, the central control system calculates the time when the coal flow of the current task is completely transported according to the position prediction information of the coal flow head ; At any moment The following moments After receiving the conveyor belt switching command, the central control system stops the end conveyor belt and executes the switching action. The duration of the switching action is... ; The central control system calculates the time required for the head of the coal flow of the next task to reach the preset safety position of the terminal conveyor according to the running speed of the coal flow ; The central control system calculates the instant at which to start the coal feed for the next task, as a function of the switching action duration and the time required for the head of the coal flow of the next task to reach the end of the conveyor belt preset safety position , , ; At time , the central control system controls the reclaimer to start the coal supply of the next task and starts the coal transportation of the next task; performing contour recognition on the binaryzation-processed image data of the coal flow to extract contour information of the coal flow, comprising: sampling a Gaussian filter to perform smoothing processing on the image according to the obtained foreground image data of the coal flow; calculating the gradient amplitude and direction of the smoothed image to obtain a gradient amplitude image and a gradient direction image; performing non-maximum suppression processing on the gradient amplitude image; performing thresholding processing on the non-maximum suppression-processed image by using a double-threshold method to connect edge pixels to obtain an edge image; performing contour tracking on the edge image to extract connected edge contour pixel coordinates; outputting the extracted edge contour pixel coordinates as the contour information of the coal flow; wherein the double-threshold method is used to perform thresholding processing on the non-maximum suppression-processed image I_nms to connect edge pixels to obtain an edge image I_edge; a high threshold T_high and a low threshold T_low are set, wherein T_high=k*T_low, k is a constant factor; for each pixel (x, y) in I_nms: if the gradient amplitude thereof is greater than the high threshold T_high, it is marked as a strong edge pixel, I_edge(x, y)=255; if the gradient amplitude thereof is less than the low threshold T_low, it is marked as a non-edge pixel, I_edge(x, y)=0; if the gradient amplitude thereof is between the high and low thresholds, it is marked as a weak edge pixel, I_edge(x, y)=128; for all weak edge pixels, it is checked whether there is a strong edge pixel in its 8-neighborhood, if there is, it is marked as a strong edge pixel, otherwise, it is marked as a non-edge pixel; contour tracking is performed on the edge image I_edge to extract connected edge contour pixel coordinates. 2.The coal conveying belt switching control method according to claim 1, wherein: calculating the pixel displacement of the coal flow image comprises: extracting a current frame image according to the preprocessed image data of the coal flow; for each pixel of the current frame image, selecting a plurality of pixel points in its neighborhood, and constructing an optical flow equation according to the invariance of pixel grayscale and the local smoothness of an optical flow field; Solving the constructed optical flow equation by least squares to obtain the displacement of the current pixel in the horizontal and vertical directions; Repeating the step of solving the displacement for all pixels of the current frame image to obtain the pixel displacement vector field of the entire image; Outputting the obtained pixel displacement vector field as the pixel displacement of the coal flow image in the current frame.

3. The coal conveying belt switching control method according to claim 1, wherein the running displacement of the coal flow on the conveying belt is obtained by: According to the pinhole camera model, the mapping relationship between the pixel coordinate system, the image coordinate system and the camera coordinate system is established; The pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system through the internal parameter matrix between the camera imaging plane and the image plane; The displacement vector field in the image coordinate system is converted to the camera coordinate system through the external parameter matrix between the camera coordinate system and the world coordinate system; According to the positional relationship between the camera coordinate system and the conveying belt coordinate system, the displacement vector field in the camera coordinate system is converted to the conveying belt coordinate system; The displacement vector field in the conveying belt coordinate system is multiplied by the displacement time to obtain the running displacement of the coal flow on the conveying belt.

4. The coal conveying belt switching control method according to claim 3, wherein the pixel displacement vector field in the pixel coordinate system is converted to the image coordinate system by: The conversion to the matrix form is: The relationship between the pixel coordinate system and the image coordinate system is: ； wherein, and are image coordinate systems; and are pixel coordinate systems; and are coordinates of reference points in the pixel coordinate systems; 5. The coal conveying belt switching control method according to claim 3, wherein the displacement vector field in the pixel coordinate system is converted to the camera coordinate system through the external parameter matrix between the camera coordinate system and the camera coordinate system: ； 6. The coal conveying belt switching control method according to claim 3, wherein the running displacement of the coal flow on the conveying belt is obtained by: ； wherein the pixel coordinate system has the x-axis as coordinate axis, the z-axis as optical axis, and the origin as optical center of the camera, denotes the focal length; the position of point A in the pixel coordinate system is , which is imaged in the image coordinate system as .

7. The coal conveying belt switching control method according to claim 3, wherein the running displacement of the coal flow on the conveying belt is obtained by:

8. The coal conveying belt switching control method according to claim 1, wherein the coal flow image and the conveying belt image are distinguished by setting a threshold, comprising: wherein the camera coordinate system has the z-axis as a coordinate axis, the z-axis as an optical axis; and is the coordinate of the reference point in the pixel coordinate system; and is the pixel coordinate system. According to the pre-processed image data of the coal flow, the gray histogram of the image is calculated; According to the positional relationship between the camera coordinate system and the conveyor belt coordinate system, the displacement vector field in the camera coordinate system is converted to the conveyor belt coordinate system, and the displacement vector field in the conveyor belt coordinate system is obtained. : wherein moves to at this time the corresponding point of the image plane moves to . According to the gray histogram, all pre-set thresholds are traversed to divide the image pixels into background regions and foreground regions; For each threshold, the number of pixels, the average gray value of the background region and the foreground region, and the average gray value of the entire image are calculated; Wherein the camera is perpendicular to the conveyor belt, the distance between the camera and the conveyor belt is , the pixel position of the object translation on the image is , and the actual moving distance is ; , and represent the coefficients. The inter-class variance between the background region and the foreground region is calculated using the inter-class variance formula; The threshold corresponding to the maximum inter-class variance is selected as the optimal threshold for image binarization; Using the obtained optimal threshold, the pixels with a gray value greater than or equal to the optimal threshold in the image data of the coal flow are set to 255, representing the coal flow foreground image, and vice versa, representing the conveying belt background image; Output the coal flow foreground image data and the conveying belt background image data after binarization processing.

9. The coal conveying belt switching control method according to claim 1, wherein the running displacement of the coal flow on the conveying belt is obtained by: ​ ​ ​ ​ ​ According to the profile information and the running speed of the coal flow, the position of the coal flow head on the conveyor belt is predicted, comprising: According to the obtained profile information of the coal flow, the position coordinates of the coal flow head in the current image frame are marked out; The marked position coordinates of the coal flow head are taken as the observation value of the Kalman filtering algorithm, and an observation equation is established; According to the historical position coordinates and the current position coordinates of the coal flow head, a uniform motion model is adopted to establish a state equation of the position of the coal flow head; The position of the coal flow head is predicted by using the state equation, and the predicted position coordinates and the predicted error covariance matrix are obtained; The predicted results are updated by using the observation equation, and the updated position coordinates of the coal flow head and the updated error covariance matrix are obtained; The obtained position coordinates of the coal flow head are taken as the position of the coal flow head in the image at the current time and outputted; The obtained position coordinates of the coal flow head and the error covariance matrix are taken as the initial value of the Kalman filtering algorithm of the next frame of image; The above steps are repeated to predict and update the position coordinates of the coal flow head on the conveyor belt.

Images