A coal loading control method based on computer vision

Abstract

The present invention relates to the fields of visual analysis, measurement, and loading, and specifically to a computer vision-based coal loading control method. The coal loading control system includes a vehicle identification camera and a vehicle identification processing module arranged in a waiting area, a loading monitoring camera and a computer vision processing module arranged in a loading area, and an integrated control and processing center. The method includes: obtaining truck information; performing a first loading operation and determining the angle of the coal pile based on the direction of dust movement, thereby calculating the loaded coal mass; performing a second loading operation and determining the angle of the coal pile based on the direction of dust movement, thereby calculating the loaded coal mass; repeating the calculation until the last loading operation and determining the angle of the coal pile based on the direction of dust movement, thereby calculating the total loaded coal mass; and stopping the coal loading when the rated loading mass is reached. The present invention can ensure loading accuracy and uniformity, improve loading efficiency, and avoid the high investment in traditional weighing equipment, thereby having high economic and practical value.

Description

Technical Field

[0001] The present invention relates to the fields of visual analysis (G06T), measurement (G01N) and loading (B65G), and in particular to a coal loading control method based on computer vision. Background Art

[0002] When coal mines use open-pit mining technology, the mined coal is often loaded into trucks using coal feeders and then transported out by trucks. During the loading process using coal feeders, a fixed amount of coal should be loaded into the truck to facilitate measurement and statistics of the coal quantity. In the existing technology, the operator can control the working speed and start and stop of the quantitative coal feeder to release the coal in a fixed amount. However, manual loading and control of the coal loading amount are directly affected by the operator's experience, the loading accuracy is poor, and the operator's repetitive labor intensity is high, which easily leads to fatigue. Using a scale to measure the loading amount is also a conventional method in this field. However, due to the limited space in the loading area, it is inconvenient to install a scale in the loading area, and the scale is expensive. At the same time, during the loading process, the coal will cause impact on the truck, and then on the scale, which will result in poor measurement stability and accuracy of the scale, and easy damage. Therefore, if it is possible to load the truck quickly and accurately without using a scale, it will be of great significance to improve the loading efficiency of open-pit mines, save the construction cost of the loading area, and improve the accuracy of the loading amount. Summary of the Invention

[0003] In response to the above technical problems, the present invention provides a coal loading and unloading control system based on computer vision, comprising a vehicle recognition camera; the vehicle recognition camera is arranged in a waiting area, and is used to obtain the license plate information of the truck waiting at the front of the queue in the waiting area, and the vehicle recognition camera is connected to a vehicle recognition processing module; the vehicle recognition processing module internally stores the license plate information of all trucks, as well as the truck loading compartment size information and rated load capacity information corresponding to the license plate, the truck loading compartment size information including the length, width and height of the truck loading compartment, and also including the number of loading times; the vehicle recognition processing module receives the license plate information measured by the vehicle recognition camera, and extracts the loading compartment size information and rated load capacity information of the truck corresponding to the license plate, the vehicle recognition processing module is connected to an integrated control and processing center, and transmits the extracted loading compartment size information and rated load capacity information of the truck corresponding to the license plate to the integrated control and processing center;

[0004] The loading monitoring camera is arranged on the front side of the loading area, the loading camera is connected to the computer vision processing module, and the computer vision processing module is connected to the integrated control and processing center;

[0005] A coal feeder is provided above the truck travel route in the loading area. The coal feeder is connected to an integrated control and processing center, which controls the opening and closing of the coal feeder and its operating speed. The loading compartments of all trucks have the same width and height. The width of the coal feeder's coal discharge belt is the same as the width of the truck's loading compartment. The coal conveying direction of the coal feeder's coal discharge belt is the same as the truck's travel direction. The coal feeder's coal discharge belt can discharge a first length of coal at a time in the length direction of the loading compartment. The loading number N indicates how many times a truck is loaded. The length of the truck is an integer multiple of the first length.

[0006] The laser rangefinder is arranged at the front side of the truck's route to measure the position of the truck in the direction of travel of the loading area. The signal indicator board is arranged at the front side of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder is displayed on the signal indicator board. The signal indicator board and the laser rangefinder are both connected to the integrated control and processing center. The measurement data of the laser rangefinder is transmitted to the integrated control and processing center. The integrated control and processing center processes the data and displays it through the signal indicator board.

[0007] Preferably, as a parallel technical solution, several speed bumps are set at intervals on the truck's route, and the interval between the speed bumps is a first length. Every time the truck moves forward to a speed bump position, it stops and releases coal once; a signal sign is set on the front side of the truck's route to provide forward information to the truck driver, and the signal sign is connected to the integrated control and processing center.

[0008] The present invention also proposes a coal loading control method based on computer vision, comprising the following steps:

[0009] S1: A truck drives to the waiting area and waits. When it reaches the front of the queue, the vehicle recognition camera reads the truck's license plate information and transmits it to the vehicle recognition processing module. The vehicle recognition processing module obtains the truck's loading compartment dimensions and rated load information based on pre-stored information within the module. The loading compartment dimensions include length, width, and height, and transmits this information to the integrated control and processing center. All trucks have the same width and height.

[0010] S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. One truck enters the loading area at a time, and only one truck is loaded at a time. The integrated control and processing center controls the loading based on the truck's loading compartment size and rated load information.

[0011] After the truck enters the loading area, it first stops at the first position according to the instructions. When using a laser rangefinder, it moves forward to the first position according to the instructions of the signal sign. At the first position, the first length of the rightmost part of the truck loading compartment corresponds to the bottom of the loader. When using a speed bump, the first position is set with the first speed bump in the direction of the truck's travel.

[0012] The computer vision processing module obtains the four corner point information of the side of the loading compartment based on the loading camera, which are named as point A(x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, retrieve the loading compartment size information of the truck, and use formula (1) to calculate the ratio t between the image obtained by the loading monitoring camera and the actual size

[0013]

[0014] Set a uniform maximum loading height h for all trucks max ;

[0015] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, coal loading is stopped. During the last loading, coal loading is stopped when the rated load is reached.

[0016] Before stopping the coal loading, the loading monitoring camera acquires the image in real time, and the computer vision processing module passes through the highest point E of the outer contour. j The horizontal line is the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. This includes predicting the angle of the coal pile on the left side of the area blocked by the loaded carriage based on the dust movement direction of the area on the left side of the outer contour.

[0017]

[0018] According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted

[0019]

[0020] The highest point of the outer contour of the first loading coal pile is E1. The right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the car top is V 1r, calculate its intersection point with the right side of the car T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , intersection point V with the left side of the car top surface 1l ; Calculate the mass of the right half of the coal pile for the first loading;

[0021] S3: The truck moves forward the first distance according to the instructions and stops at the second position. When a laser rangefinder is used, the truck moves forward to the second position according to the instructions of the signal sign. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel.

[0022] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera and stops loading when the maximum height of the outer contour approaches the maximum loading height.

[0023] Before stopping coal loading, the loading monitoring camera acquires real-time images. The computer vision processing module uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading compartment is predicted based on the dust movement direction of the area on the left side of the outer contour using the same method as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ;

[0024] The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the top surface of the carriage is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ for the first loading 1l , calculate the intersection point T 2r , that is, T 1l The intersection of the second loading coal pile and the left side of the carriage top is V 2l ; Calculate the mass of the right half of the coal pile for the second loading; Calculate the mass of the left half of the coal pile for the first loading;

[0025] S4: The truck moves forward once every first length according to the instruction and calculates the mass of the coal pile according to the method of step S3 until it reaches the last, i.e., Nth, coal loading position and stops at the Nth position; when a laser rangefinder is used, the truck moves forward to the Nth position according to the instruction information of the signal sign; when a speed bump is used, the Nth speed bump is set at the Nth position in the direction of the truck's travel;

[0026] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the outer contour appears during the last loading, that is, the Nth loading, the computer vision processing module obtains the image in real time through the loading monitoring camera. The computer vision processing module passes through the highest point E of the outer contour of the Nth coal pile. n The horizontal line is used as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the angle θ of the coal pile on the left side of the area blocked by the loading carriage based on the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ;

[0027] The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the car top is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , calculate the intersection point T nr , that is, T n-1l The intersection point of the left side of the coal pile and the top of the carriage for the Nth loading is V nl , the intersection point with the left side of the carriage is T nl ; Calculate the mass of the right half of the coal pile for the Nth loading; Calculate the mass of the left half of the coal pile for the Nth and N-1th loading;

[0028] S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

[0029] Preferably, in step S2, the first method for obtaining the mass of the right half of the first loading coal pile is to approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r Angle θ 1r According to the density of coal, width of loading carriage, and the ratio t between image and actual size, the mass of the right half of the coal pile of the first loading can be obtained.

[0030] Preferably, in step S2, the second method for obtaining the mass of the right half of the first loading coal pile is: the right half of the first loading coal pile is divided into two parts, the first part is E1V 1r The corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1rAccording to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a ; For the first part, E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a By adding them together, we can calculate the mass of the right half of the coal pile for the first loading.

[0031] Preferably, in step S3, the mass of the right half of the coal pile of the second loading is obtained in the same manner as in step S2.

[0032] Preferably, in step S3, the first method for obtaining the mass of the left half of the first loading coal pile is to approximate the left half of the first loading coal pile to a trapezoid, where E1T 1l Angle θ 1l According to the density of coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass of the left half of the coal pile of the first loading can be calculated.

[0033] Preferably, in step S3, the second method for obtaining the mass of the left half of the first loading coal pile is: the left half of the first loading coal pile is divided into two parts, the first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l Angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a By adding them together, we can calculate the mass of the left half of the coal pile for the first loading.

[0034] Preferably, in step S4, the mass of the right half of the coal pile loaded for the Nth time is obtained in the same manner as in step S2.

[0035] Preferably, in step S4, the mass of the left half of the coal pile of the Nth and N-1th loadings is obtained in the same manner as in step S3.

[0036] The key means and beneficial effects of the present invention are:

[0037] 1. Based on computer vision and positioning technology, the system recognizes truck license plates and vehicle dimensions in real time, uses a laser rangefinder to precisely locate trucks in the loading area, and uses signal signs to guide vehicles to dock. This ensures loading accuracy and uniformity, improves loading efficiency, and reduces human error.

[0038] 2. The optical flow method calculates the coal loading amount in real time, ensuring uniform coal loading and accurate quantitative measurement. At the same time, it avoids the high investment of traditional floor scale equipment and has high economic and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the waiting area in the computer vision coal loading system of the present invention;

[0040] Figure 2 This is a schematic diagram of the loading area in the computer vision coal loading system of the present invention;

[0041] Figure 3 This is a side view of the loading area in the computer vision coal loading system of the present invention;

[0042] Figure 4 Schematic diagram of the truck loading process in the computer vision coal loading system of the present invention;

[0043] In the figure, there are vehicle identification camera 1, truck 2, vehicle identification processing module 3, computer vision processing module 4, integrated control and processing center 5, loading monitoring camera 6, signal sign 7, laser rangefinder 8, and coal feeder 9. DETAILED DESCRIPTION

[0044] The technical solution of the present invention will be described in more detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] like Figure 1-3As shown, the coal loading control system based on computer vision of the present invention comprises a vehicle recognition camera 1, a vehicle recognition processing module 3, a computer vision processing module 4, an integrated control and processing center 5, a loading monitoring camera 6, a signal indicator 7 and a coal feeder 9; the vehicle recognition camera 1 is arranged in the waiting area, and is used to obtain the license plate information of the truck 2 waiting in the waiting area. The truck 2 in the front row refers to the truck 2 in the front row in the waiting area that is about to enter the loading area. The vehicle recognition camera 1 is connected to the vehicle recognition processing module 3, and the vehicle recognition processing module 3 internally stores the license plate information of all trucks 2, as well as the truck loading compartment size information and the rated load corresponding to the license plate. Loading capacity information, the truck loading compartment size information includes the length, width and height of the truck loading compartment, and also includes the number of loading times information; the vehicle identification processing module 3 receives the license plate information measured by the vehicle identification camera 1, and extracts the loading compartment size information and rated loading capacity information of the truck corresponding to the license plate. The vehicle identification processing module 3 is connected to the integrated control and processing center 5, and transmits the extracted loading compartment size information and rated loading capacity information of the truck corresponding to the license plate to the integrated control and processing center 5; the vehicle identification camera 1 is preferably arranged in the waiting area near the entrance of the loading area, located in front of the truck's travel route, the loading area is located on the right side of the waiting area, and the truck 2 travels in the left-right direction and drives to the right;

[0046] The loading monitoring camera 6 is arranged on the front side of the loading area. The direction of travel of the truck 2 is from left to right. The loading camera 6 can capture the image of the entire loading area. The loading camera 6 is connected to the computer vision processing module 4, and the computer vision processing module 4 is connected to the integrated control and processing center 5.

[0047] A coal feeder 9 is provided above the truck travel route in the loading area. The coal feeder 9 is connected to the integrated control and processing center 5, which controls the opening and closing of the coal feeder 9 and the working speed. The loading compartments of all trucks 2 have substantially the same width and the same or similar heights. The width of the coal belt of the coal feeder 9 is the same as that of the truck's loading compartment, or slightly smaller than that of the truck's loading compartment. The coal conveying direction of the coal belt of the coal feeder 9 is the same as that of the truck's travel direction, that is, the length direction of the coal belt of the coal feeder 9 is the same as that of the truck's travel direction. The length direction is the same as that of the truck's loading compartment; the coal feeding machine 9 discharges coal in the length direction of the loading compartment once, which corresponds to a loading compartment length of 2m. The loading times information is how many times a truck is loaded; if the length of the loading compartment is 8m, the coal is discharged in four times along the loading compartment, and the loading times N is four times. If the loading compartment is 10m, the coal is discharged in five times, and the loading times are five times. The length of the truck is 2m, which is an integer multiple of the length of the loading compartment corresponding to one coal discharge. The loading times N information is how many times a truck is loaded;

[0048] The laser rangefinder 8 is arranged at the front side of the truck's route to measure the position of the truck in the direction of travel of the loading area. The signal indicator board 7 is arranged at the front side of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder 8 is displayed on the signal indicator board 7. The signal indicator board 7 and the laser rangefinder 8 are both connected to the integrated control and processing center 5. The measurement data of the laser rangefinder 8 is transmitted to the integrated control and processing center 5. The integrated control and processing center 5 processes the data and displays it through the signal indicator board 7. The parallel technical solution is: a number of speed bumps are arranged at intervals on the truck's route. The interval between the speed bumps is 2m, which is the length of the loading compartment corresponding to one coal discharge. The truck stops at a speed bump position every time it discharges coal. The signal indicator board 7 is arranged at the front side of the truck's route to provide forward information to the truck driver. The signal indicator board 7 is connected to the integrated control and processing center 5.

[0049] Based on the above-mentioned coal placing and loading control system based on computer vision, the present invention further proposes a coal placing and loading control method based on computer vision, which includes the following steps:

[0050] S1: Truck 2 drives to the waiting area and waits. When it is at the front of the queue, the vehicle recognition camera 1 reads the license plate information of truck 2 and transmits the license plate information to the vehicle recognition processing module 3. The vehicle recognition processing module 3 obtains the loading compartment size information and rated load capacity information of the truck based on the pre-stored information within the module. The loading compartment size information includes length l, width and height h, and transmits this information to the integrated control and processing center 5. All trucks have the same width and height, which are both fixed values.

[0051] S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. Only one truck enters the loading area at a time and is loaded at a time. The integrated control and processing center 5 controls the loading based on the obtained information about the loading compartment size and rated load capacity of the truck.

[0052] After entering the loading area, truck 2 first stops at the first position according to the instructions. When using a laser rangefinder, it moves forward to the first position according to the instructions of the signal sign 7. The first position is 2 meters from the rightmost side (near the front side) of the truck loading compartment, which corresponds to directly below the loader 9. When a speed bump is used, the first position is set to the first speed bump in the direction of the truck's travel.

[0053] The computer vision processing module 4 obtains the four corner point information of the side of the loading compartment based on the loading camera 6, which are named as point A (x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, since the position of the loading monitoring camera 6 remains unchanged, the coordinate origin of the images obtained before and after remains unchanged, and retrieving the loading compartment size information of the truck, using formula (1) to calculate the ratio t between the image obtained by the loading monitoring camera 6 and the actual size

[0054]

[0055] Set the maximum loading height. In order to prevent the loading height from exceeding the height of the loading compartment and avoid coal spillage during transportation, and to adapt to the height of the loading area, a unified maximum loading height h is set for all trucks 2. max ;

[0056] The integrated control and processing center 5 controls the coal feeder to discharge coal. The computer vision processing module 4 obtains the outer contour of the loaded coal body (along the length direction of the truck, that is, approximately triangular in the left and right directions) through the loading monitoring camera 6. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. For example, when the height reaches less than h max When the load is about 20cm, the information is fed back to the integrated control and processing center 5 and the coal feeder is controlled to stop loading coal; during the last loading, the coal loading is stopped when the rated load is reached;

[0057] Before stopping the coal loading, the loading monitoring camera 6 is used to obtain the real-time image. The computer vision processing module 4 passes through the highest point E of the outer contour. jThe horizontal line is the dividing line (1≤j≤n), and the optical flow method is used to calculate the dust movement direction at the outer contours on both sides of the area below the dividing line. The optical flow method is calculated as follows:

[0058] I i (x i ,y i ,t)=I i (x i +dx i ,y i +dy i ,t+dt) (2)

[0059] Get the dust in the horizontal direction u i Velocity component and vertical velocity component v i

[0060]

[0061] Then the dust movement direction at the outer contour is θ i

[0062]

[0063] According to the dust movement direction in the left area of ​​the outer contour, the angle of the coal pile on the left side of the area blocked by the loading carriage is predicted

[0064]

[0065] According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted

[0066]

[0067] The highest point of the outer contour of the first loading coal pile is E1. The right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the car top is V 1r , calculate its intersection point with the right side of the car T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , intersection point V with the left side of the car top surface 1l ; To obtain the mass of the right half of the first loading coal pile, the first method is to approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r Angle θ 1r According to the density of coal, width of loading compartment, and ratio t of image to actual size, the mass of the right half of the first loading coal pile can be obtained. The second method is to divide the right half of the first loading coal pile into two parts. The first part is E1V 1rThe corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1r According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a For the first part, E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a Add them together to find the mass of the right half of the coal pile for the first loading;

[0068] S3: Truck 2 moves forward 2 m according to the instructions and stops at the second position. When a laser rangefinder is used, it moves forward to the second position according to the instructions of the signal sign 7. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel;

[0069] The integrated control and processing center 5 controls the coal feeder to discharge coal. The computer vision processing module 4 obtains the outer contour of the loaded coal body (along the length direction of the truck, that is, approximately triangular in the left and right directions) through the loading monitoring camera 6. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. For example, when the height reaches less than h max When the depth is about 20 cm, the information is fed back to the integrated control and processing center 5 and the coal feeder is controlled to stop loading coal;

[0070] Before stopping coal loading, the loading monitoring camera 6 acquires images in real time. The computer vision processing module 4 uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading compartment is predicted based on the dust movement direction of the area on the left side of the outer contour using the same method as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ;

[0071] The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the top surface of the carriage is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ for the first loading 1l , we can calculate the intersection point T of the two2r (Also T 1l ), the intersection of the second loading coal pile and the left side of the carriage top is V 2l ; Obtain the mass of the right half of the second loading coal pile, the method is the same as step S2; Obtain the mass of the left half of the first loading coal pile, the method is similar to step S2, the first method is: the left half of the first loading coal pile is approximated as a trapezoid, where E1T 1l Angle θ 1l , the mass of the left half of the first loading coal pile can be obtained according to the density of coal, the width of the loading car, and the ratio t of the image to the actual size; the second method is to divide the left half of the first loading coal pile into two parts, the first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l Angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a Add them together to find the mass of the left half of the coal pile for the first loading;

[0072] S4: Truck 2 moves forward every 2 meters according to the instructions and calculates the mass of the coal pile according to the method in step S3 until it reaches the last, i.e., the Nth, coal loading position, and stops at the Nth position. When a laser rangefinder is used, the truck moves forward to the Nth position according to the instructions of the signal sign 7. When a speed bump is used, the Nth speed bump is set at the Nth position in the direction of the truck's travel.

[0073] The integrated control and processing center 5 controls the coal feeder to discharge coal. The computer vision processing module 4 obtains the outer contour of the loaded coal through the loading monitoring camera 6 (along the length direction of the truck, that is, the left and right directions are approximately triangular). When the last time the coal is loaded, that is, the Nth time, the outer contour appears, the image is obtained in real time through the loading monitoring camera 6. The computer vision processing module 4 passes through the highest point E of the outer contour of the Nth coal pile. nThe horizontal line is used as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the angle θ of the coal pile on the left side of the area blocked by the loading carriage based on the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ;

[0074] The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the car top is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , we can calculate the intersection point T of the two nr (Also T n-1l ), the intersection point between the left side of the coal pile and the top surface of the carriage for the Nth loading is V nl , the intersection point with the left side of the carriage is T nl ; Obtain the mass of the right half of the coal pile for the Nth loading, using the same method as step S2; Obtain the mass of the left half of the coal pile for the Nth and N-1th loading, using the same method as step S3;

[0075] S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

[0076] Those skilled in the art can make various corresponding changes and modifications based on the above technical solutions and concepts, and all of these changes and modifications should be included in the scope of protection of the claims of the present invention.

Claims

1. A coal loading and unloading control system based on computer vision, including a vehicle recognition camera; characterized in that: The vehicle recognition camera is arranged in the waiting area and is used to obtain the license plate information of the truck waiting at the front of the queue in the waiting area. The vehicle recognition camera is connected to the vehicle recognition processing module; the vehicle recognition processing module internally stores the license plate information of all trucks, as well as the loading compartment size information and rated load capacity information of the truck corresponding to the license plate, the loading compartment size information of the truck including the length, width and height of the truck loading compartment, as well as the number of loading times; the vehicle recognition processing module receives the license plate information measured by the vehicle recognition camera and extracts the loading compartment size information and rated load capacity information of the truck corresponding to the license plate. The vehicle recognition processing module is connected to the integrated control and processing center and transmits the extracted loading compartment size information and rated load capacity information of the truck corresponding to the license plate to the integrated control and processing center; The loading monitoring camera is arranged on the front side of the loading area, and the loading monitoring camera is connected to the computer vision processing module, and the computer vision processing module is connected to the integrated control and processing center; A coal feeder is provided above the truck travel route in the loading area. The coal feeder is connected to an integrated control and processing center, which controls the opening and closing of the coal feeder and its operating speed. The loading compartments of all trucks have the same width and height. The width of the coal feeder's coal discharge belt is the same as the width of the truck's loading compartment. The coal conveying direction of the coal feeder's coal discharge belt is the same as the truck's travel direction. The coal feeder's coal discharge belt can discharge a first length of coal at a time in the length direction of the loading compartment. The loading number N indicates how many times a truck is loaded. The length of the truck is an integer multiple of the first length. A laser rangefinder is arranged in front of the truck's route to measure the truck's position in the direction of travel of the loading area. A signal indicator is arranged in front of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder is displayed on the signal indicator. Both the signal indicator and the laser rangefinder are connected to the integrated control and processing center. The measurement data of the laser rangefinder is transmitted to the integrated control and processing center, and the integrated control and processing center processes the data and displays it through the signal indicator.

2. The coal loading control system according to claim 1, characterized in that: As a parallel technical solution, several speed bumps are set at intervals on the truck's route, and the interval between the speed bumps is a first length. Every time the truck moves forward to a speed bump position, it stops and unloads coal; a signal sign is set at the front side of the truck's route to provide forward information to the truck driver, and the signal sign is connected to the integrated control and processing center.

3. A coal loading control method based on computer vision, characterized in that: The steps include: S1: A truck drives to the waiting area and waits. When it reaches the front of the queue, the vehicle recognition camera reads the truck's license plate information and transmits it to the vehicle recognition processing module. The vehicle recognition processing module obtains the truck's loading compartment size information and rated load information based on pre-stored information within it. The truck's loading compartment size information includes length, width, and height, and transmits this information to the integrated control and processing center. All trucks have the same width and height. S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. One truck enters the loading area at a time, and only one truck is loaded at a time. The integrated control and processing center controls the loading based on the truck's loading compartment size and rated load information. After the truck enters the loading area, it first stops at the first position according to the instructions. When using a laser rangefinder, it moves forward to the first position according to the instructions of the signal sign. At the first position, the first length of the rightmost part of the truck loading compartment corresponds to the bottom of the loader. When using a speed bump, the first position is set with the first speed bump in the direction of the truck's travel. The computer vision processing module obtains the four corner point information of the side of the loading compartment based on the loading camera, which are named as point A(x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, retrieve the truck's loading compartment size information, which includes length l, width, and height h. Formula (1) is used to calculate the ratio t between the image obtained by the loading monitoring camera and the actual size. Set a uniform maximum loading height h for all trucks max ; The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, coal loading is stopped. During the last loading, coal loading is stopped when the rated load is reached. Before stopping the coal loading, the loading monitoring camera acquires the image in real time, and the computer vision processing module passes through the highest point E of the outer contour. j The horizontal line is the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. This includes predicting the angle of the coal pile on the left side of the area blocked by the loaded carriage based on the dust movement direction of the area on the left side of the outer contour. According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted The highest point of the outer contour of the first loading coal pile is E1. The right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the car top is V 1r , calculate its intersection point with the right side of the car T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , intersection point V with the left side of the car top surface 1l ; Calculate the mass of the right half of the coal pile for the first loading; S3: The truck moves forward the first distance according to the instructions and stops at the second position. When a laser rangefinder is used, the truck moves forward to the second position according to the instructions of the signal sign. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel. The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera and stops loading when the maximum height of the outer contour approaches the maximum loading height. Before stopping coal loading, the loading monitoring camera acquires real-time images. The computer vision processing module uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading compartment is predicted based on the dust movement direction of the area on the left side of the outer contour using the same method as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ; The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the car top is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ for the first loading 1l , calculate the intersection point T 2r , that is, T 1l The intersection of the second loading coal pile and the left side of the carriage top is V 2l ; Calculate the mass of the right half of the coal pile for the second loading; Calculate the mass of the left half of the coal pile for the first loading; S4: The truck moves forward once every first length according to the instruction and calculates the mass of the coal pile according to the method of step S3 until it reaches the last, i.e., Nth, coal loading position and stops at the Nth position; when a laser rangefinder is used, the truck moves forward to the Nth position according to the instruction information of the signal sign; when a speed bump is used, the Nth speed bump is set at the Nth position in the direction of the truck's travel; The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the outer contour appears during the last loading, that is, the Nth loading, the computer vision processing module obtains the image in real time through the loading monitoring camera. The computer vision processing module passes through the highest point E of the outer contour of the Nth coal pile. n The horizontal line is used as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the angle θ of the coal pile on the left side of the area blocked by the loading carriage based on the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ; The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the car top is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , calculate the intersection point T nr , that is, T n-1l The intersection point of the left side of the coal pile and the top of the carriage for the Nth loading is V nl , the intersection point with the left side of the carriage is T nl ; Calculate the mass of the right half of the coal pile for the Nth loading; Calculate the mass of the left half of the coal pile for the Nth and N-1th loading; S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

4. The coal loading control method according to claim 3, characterized in that: In step S2, the first method for obtaining the mass of the right half of the first loading coal pile is to approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r Angle θ 1r According to the density of coal, width of loading carriage, and the ratio t between image and actual size, the mass of the right half of the coal pile of the first loading can be obtained.

5. The coal loading control method according to claim 3, characterized in that: In step S2, the second method for obtaining the mass of the right half of the first loading coal pile is: the right half of the first loading coal pile is divided into two parts, the first part is E1V 1r The corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1r According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a ; For the first part, E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a By adding them together, we can calculate the mass of the right half of the coal pile for the first loading.

6. The coal loading control method according to claim 4 or 5, characterized in that: In step S3, the mass of the right half of the coal pile of the second loading is obtained in the same manner as in step S2.

7. The coal loading control method according to claim 3, characterized in that: In step S3, the first method for obtaining the mass of the left half of the first loading coal pile is to approximate the left half of the first loading coal pile to a trapezoid, where E1T 1l Angle θ 1l According to the density of coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass of the left half of the coal pile of the first loading can be calculated.

8. The coal loading control method according to claim 3, characterized in that: In step S3, the mass of the left half of the first loading coal pile is obtained. The second method is to divide the left half of the first loading coal pile into two parts. The first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l Angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be calculated. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. Based on the density of the coal, the width of the loading compartment, and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be calculated. b , the first part of the coal pile mass M b and the second part of the coal pile mass M a By adding them together, we can calculate the mass of the left half of the coal pile for the first loading.

9. The coal loading control method according to claim 4 or 5, characterized in that: In step S4, the mass of the right half of the coal pile loaded for the Nth time is obtained in the same manner as in step S2.

10. The coal loading control method according to claim 7 or 8, characterized in that: In step S4, the mass of the left half of the coal pile for the Nth and N-1th loadings is obtained in the same manner as in step S3.

Images