Coal caving and loading control method based on computer vision

Abstract

The invention relates to the field of visual analysis, measurement and loading, in particular to a coal and loading control method based on computer vision, and an adopted coal and loading control system comprises a vehicle identification camera and a vehicle identification processing module which are arranged in a waiting area, and a loading monitoring camera and a computer visual processing module which are arranged in a loading area. The system further comprises a comprehensive control and processing center. The method comprises the following steps: acquiring truck information; carrying out first loading, determining a coal pile angle based on the dust movement direction, and further calculating the coal loading mass; loading for the second time, determining the coal pile angle based on the dust movement direction, and further calculating the coal loading mass; repeatedly calculating until the last loading is carried out, and determining the coal pile angle based on the dust movement direction so as to calculate the final assembly coal mass; and stopping coal charging when the rated loading mass is reached. According to the invention, the loading precision and uniformity can be ensured, and the loading efficiency is improved; meanwhile, high investment of traditional wagon balance equipment is avoided, and high economic and practical values are achieved.

Description

Technical Field

[0001] The invention relates to the fields of visual analysis (G06T), measurement (G01N) and loading (B65G), and in particular to a coal loading control method based on computer vision. Background Art

[0002] When coal mines are mined using open-pit mining technology, the mined coal is often loaded into trucks using coal feeders and then transported out by trucks. In the process of loading with coal feeders, a certain amount of coal should be loaded into the truck to facilitate measurement and statistics of the amount of coal. In the prior art, the operator can control the working speed and start and stop of the quantitative coal feeder to quantitatively release coal, but the manual loading and control of the amount of coal loaded is directly affected by the operator's experience, the accuracy of coal loading is poor, and the operator's repetitive labor intensity is high, which is easy to cause fatigue. Using a scale to measure the loading amount is also a conventional means in this field, but due to the limitation of the space in the loading area, it is inconvenient to install a scale in the loading area, and the scale is expensive; at the same time, during the loading process, the coal will impact the truck, and then impact the scale, which will lead to poor stability and accuracy of the scale measurement, and easy damage. Therefore, if it is possible to load quickly and accurately without using a scale, it will be of great significance to improve the efficiency of open-pit mine loading, save the construction cost of the loading area, and improve the accuracy of the loading amount. Summary of the invention

[0003] In view of the above technical problems, the present invention provides a coal loading control system based on computer vision, comprising a vehicle identification camera; the vehicle identification camera is arranged in a waiting area, and is used to obtain the license plate information of the truck waiting in the front row in the waiting area, and the vehicle identification camera is connected to a vehicle identification processing module; the vehicle identification processing module internally stores the license plate information of all trucks, as well as the truck loading compartment size information and rated load information corresponding to the license plate, the truck loading compartment size information includes the length, width and height of the truck loading compartment, and also includes the number of loading times information; the vehicle identification processing module receives the license plate information measured by the vehicle identification camera, and extracts the loading compartment size information and rated load information of the truck corresponding to the license plate, the vehicle identification processing module is connected to an integrated control and processing center, and transmits the extracted loading compartment size information and rated load information of the truck corresponding to the license plate to the integrated control and processing center;

[0004] The loading monitoring camera is arranged on the front side of the loading area, the loading camera is connected to a computer vision processing module, and the computer vision processing module is connected to a comprehensive control and processing center;

[0005] A coal feeder is arranged above the truck travel route in the loading area, and the coal feeder is connected to the integrated control and processing center, and the integrated control and processing center is connected to control the opening and closing and working speed of the coal feeder. The loading compartments of all trucks have the same width and height. The width of the coal feeder coal belt is the same as the width of the truck loading compartment. The coal conveying direction of the coal feeder coal belt is the same as the traveling direction of the truck. The coal feeder coal belt can discharge a first length of coal in the length direction of the loading compartment at one time. The loading number N information is how many times a truck is loaded; the length of the truck is an integer multiple of the first length;

[0006] The laser rangefinder is arranged at the front side of the truck's route to measure the position of the truck in the direction of travel of the loading area. The signal indicator board is arranged at the front side of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder is displayed on the signal indicator board. The signal indicator board and the laser rangefinder are both connected to the integrated control and processing center. The measurement data of the laser rangefinder is transmitted to the integrated control and processing center, and the integrated control and processing center processes the data and displays it through the signal indicator board.

[0007] Preferably, as a parallel technical solution, several speed bumps are arranged at intervals on the truck's route, and the interval between the speed bumps is a first length. Every time the truck moves forward to a speed bump, it stops and releases coal once; a signal sign is arranged on the front side of the truck's route to provide forward information to the truck driver, and the signal sign is connected to the integrated control and processing center.

[0008] The present invention also proposes a coal loading control method based on computer vision, comprising the following steps:

[0009] S1: A truck drives to the waiting area and waits. When it is at the front of the queue, the vehicle recognition camera reads the license plate information of the truck and transmits the license plate information to the vehicle recognition processing module. The vehicle recognition processing module obtains the loading compartment size information and rated load information of the truck according to the information pre-stored therein. The loading compartment size information includes length, width and height, and transmits the information to the integrated control and processing center. All trucks have the same width and height.

[0010] S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. One truck enters the loading area at a time and only one truck is loaded at a time. The integrated control and processing center controls the loading according to the obtained loading compartment size information and rated load information of the truck.

[0011] After the truck enters the loading area, it first stops at the first position according to the instructions. When a laser rangefinder is used, it moves to the first position according to the instructions of the signal sign. At the first position, the first length part of the rightmost side of the truck loading compartment corresponds to the bottom of the loader. When a speed bump is used, the first position is set with the first speed bump in the direction of the truck's travel.

[0012] The computer vision processing module obtains the four corner point information of the side of the loading compartment based on the loading camera, which are named point A(x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, retrieve the size information of the truck’s loading compartment, and use formula (1) to calculate the ratio t between the image obtained by the loading monitoring camera and the actual size

[0013]

[0014] Set a uniform maximum loading height h for all trucks max ;

[0015] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. During the last loading, the coal loading is stopped when the rated loading volume is reached.

[0016] Before stopping the coal loading, the loading monitoring camera acquires the image in real time. The computer vision processing module passes through the highest point E of the outer contour. j The horizontal line is taken as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line, including predicting the angle of the coal pile on the left side of the area blocked by the loading carriage according to the dust movement direction of the area on the left side of the outer contour.

[0017]

[0018] According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted

[0019]

[0020] The highest point of the outer contour of the first loading coal pile is E1, the right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the top surface of the carriage is V 1r, calculate its intersection point with the right side of the carriage T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , and the intersection point V with the left side of the top surface of the carriage 1l ; Calculate the mass of the right half of the coal pile for the first loading;

[0021] S3: the truck moves forward a first distance according to the instruction and stops at a second position. When a laser rangefinder is used, the truck moves forward to the second position according to the instruction information of the signal sign. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel.

[0022] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped.

[0023] Before stopping the coal loading, the loading monitoring camera acquires the image in real time. The computer vision processing module uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line, and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading carriage is predicted based on the dust movement direction of the area on the left side of the outer contour in the same way as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ;

[0024] The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the top surface of the carriage is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ of the first loading coal pile 1l , calculate the intersection point T 2r , that is, T 1l The intersection point of the second loading coal pile and the left side of the carriage top surface is V 2l ; Calculate the mass of the right half of the coal pile for the second loading; Calculate the mass of the left half of the coal pile for the first loading;

[0025] S4: the truck moves forward once every first length according to the instruction, and calculates the mass of the coal pile according to the method of step S3, until it moves forward to the last, i.e., the Nth coal loading position, and stops at the Nth position; when a laser rangefinder is used, the truck moves forward to the Nth position according to the instruction information of the signal sign, and when a speed bump is used, the Nth speed bump in the direction of the truck's travel is set at the Nth position;

[0026] The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the last time, i.e., the Nth time, coal is loaded, the loading monitoring camera obtains the image in real time when the outer contour appears. The computer vision processing module passes through the highest point E of the outer contour of the Nth coal pile. n The horizontal line is taken as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the coal pile angle θ on the left side of the area blocked by the loading carriage according to the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ;

[0027] The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the top surface of the carriage is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , calculate the intersection point T nr , that is, T n-1l , the intersection point between the left side of the coal pile and the top surface of the carriage for the Nth loading is V nl , and the intersection point with the left side of the carriage is T nl ; Calculate the mass of the right half of the coal pile for the Nth loading; Calculate the mass of the left half of the coal pile for the Nth and N-1th loadings;

[0028] S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

[0029] Preferably, in step S2, the first method for obtaining the mass of the right half of the first loading coal pile is: approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r The angle θ 1r According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the right half of the first loading coal pile can be calculated.

[0030] Preferably, in step S2, the second method for obtaining the mass of the right half of the coal pile of the first loading is: the right half of the coal pile of the first loading is divided into two parts, the first part is E1V 1r The corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1rAccording to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a ; For the first part, E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can calculate the mass of the right half of the coal pile for the first loading.

[0031] Preferably, in step S3, the mass of the right half of the coal pile of the second loading is obtained in the same manner as in step S2.

[0032] Preferably, in step S3, the first method for obtaining the mass of the left half of the first loading coal pile is: approximate the left half of the first loading coal pile to a trapezoid, where E1T 1l The angle θ 1l According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the left half of the first loading coal pile can be calculated.

[0033] Preferably, in step S3, the second method for obtaining the mass of the left half of the first loading coal pile is: the left half of the first loading coal pile is divided into two parts, the first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l The angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can calculate the mass of the left half of the coal pile for the first loading.

[0034] Preferably, in step S4, the mass of the right half of the coal pile loaded for the Nth time is obtained in the same manner as in step S2.

[0035] Preferably, in step S4, the mass of the left half of the coal pile for the Nth and N-1th loadings is obtained in the same manner as in step S3.

[0036] The key means and beneficial effects of the present invention are:

[0037] 1. Based on computer vision and positioning technology, the truck license plate and compartment size information are recognized in real time, and the position of the truck in the loading area is accurately located by combining the laser rangefinder, and the vehicle is guided to stop by signal signs. This ensures loading accuracy and uniformity, improves loading efficiency, and reduces manual errors.

[0038] 2. The optical flow method calculates the coal loading amount in real time, ensuring uniform loading and accurate quantitative measurement of coal. It also avoids the high investment in traditional weighing equipment and has high economic and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the waiting area in the computer vision coal loading system of the present invention;

[0040] Figure 2 It is a schematic diagram of the loading area in the computer vision coal loading system of the present invention;

[0041] Figure 3 It is a side view of the loading area in the computer vision coal loading system of the present invention;

[0042] Figure 4 It is a schematic diagram of the truck loading process in the computer vision coal loading system of the present invention;

[0043] In the figure, there are vehicle identification camera 1, truck 2, vehicle identification processing module 3, computer vision processing module 4, integrated control and processing center 5, loading monitoring camera 6, signal sign 7, laser rangefinder 8, and coal feeder 9. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is described in more detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0045] like Figure 1-3As shown, the coal loading control system based on computer vision of the present invention comprises a vehicle recognition camera 1, a vehicle recognition processing module 3, a computer vision processing module 4, an integrated control and processing center 5, a loading monitoring camera 6, a signal indicator board 7 and a coal feeder 9; the vehicle recognition camera 1 is arranged in the waiting area, and is used to obtain the license plate information of the front-ranked truck 2 waiting in the waiting area, and the front-ranked truck 2 refers to the front-ranked truck 2 that is about to enter the loading area in the waiting area, and the vehicle recognition camera 1 is connected to the vehicle recognition processing module 3, and the vehicle recognition processing module 3 internally stores the license plate information of all trucks 2, as well as the truck loading compartment size information and the rated load corresponding to the license plate. Loading capacity information, the truck loading compartment size information includes the length, width and height of the truck loading compartment, and also includes the number of loading times information; the vehicle identification processing module 3 receives the license plate information measured by the vehicle identification camera 1, and extracts the loading compartment size information and rated loading capacity information of the truck corresponding to the license plate, the vehicle identification processing module 3 is connected to the integrated control and processing center 5, and transmits the extracted loading compartment size information and rated loading capacity information of the truck corresponding to the license plate to the integrated control and processing center 5; the vehicle identification camera 1 is preferably arranged in the waiting area near the entrance of the loading area, located in front of the truck's travel route, the loading area is located on the right side of the waiting area, and the truck 2 travels in the left and right direction and drives to the right;

[0046] The loading monitoring camera 6 is arranged on the front side of the loading area. The moving direction of the truck 2 is from left to right. The loading camera 6 can obtain the picture of the entire loading area. The loading camera 6 is connected to the computer vision processing module 4, and the computer vision processing module 4 is connected to the integrated control and processing center 5;

[0047] A coal feeder 9 is arranged above the truck travel route in the loading area. The coal feeder 9 is connected to the integrated control and processing center 5. The integrated control and processing center 5 is connected to control the opening and closing of the coal feeder 9 and the working speed. The loading compartments of all trucks 2 have substantially the same width and the same or similar height. The width of the coal conveying belt of the coal feeder 9 is the same as the width of the truck's loading compartment, or slightly smaller than the width of the truck's loading compartment. The coal conveying direction of the coal conveying belt of the coal feeder 9 is the same as the traveling direction of the truck, that is, the length direction of the coal conveying belt of the coal feeder 9 is the same as the traveling direction of the truck. The same as the length direction of the loading compartment of the truck; the coal feeding machine 9 coal discharging belt discharges coal once in the length direction of the loading compartment, corresponding to the length of the loading compartment of 2m, and the loading number information is how many times a truck is loaded; if the length of the loading compartment is 8m, the coal is discharged along the loading compartment in four times, and the loading number N is four times; if the loading compartment is 10m, the coal is discharged in five times, and the loading number is five times. The length of the truck is 2m, that is, an integer multiple of the length of the loading compartment corresponding to one coal discharge, and the loading number N information is how many times a mining truck is loaded;

[0048] The laser rangefinder 8 is arranged at the front side of the truck's route to measure the position of the truck in the direction of travel of the loading area. The signal indicator board 7 is arranged at the front side of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder 8 is displayed on the signal indicator board 7. The signal indicator board 7 and the laser rangefinder 8 are both connected to the integrated control and processing center 5. The measurement data of the laser rangefinder 8 is transmitted to the integrated control and processing center 5, and the integrated control and processing center 5 processes the data and displays it through the signal indicator board 7; the parallel technical solution is: a number of speed bumps are arranged at intervals on the truck's route, and the interval between the speed bumps is 2m, that is, the length of the loading compartment corresponding to one coal discharge, and the truck stops at a speed bump position to discharge coal once; the signal indicator board 7 is arranged at the front side of the truck's route to provide forward information to the truck driver, and the signal indicator board 7 is connected to the integrated control and processing center 5.

[0049] Based on the above-mentioned coal placing and loading control system based on computer vision, the present invention also proposes a coal placing and loading control method based on computer vision, which comprises the following steps:

[0050] S1: Truck 2 drives to the waiting area and waits. When it is at the front of the queue, the vehicle recognition camera 1 reads the license plate information of the truck 2 and transmits the license plate information to the vehicle recognition processing module 3. The vehicle recognition processing module 3 obtains the loading compartment size information and rated load information of the truck according to the information pre-stored therein. The loading compartment size information includes length l, width and height h, and transmits the information to the integrated control and processing center 5. All trucks have the same width and height, which are all fixed values.

[0051] S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. One truck enters the loading area at a time and only one truck is loaded at a time. The integrated control and processing center 5 controls the loading according to the acquired loading compartment size information and rated load information of the truck.

[0052] After the truck 2 enters the loading area, it first stops at the first position according to the instructions. When the laser rangefinder is used, it moves to the first position according to the instructions of the signal sign 7. The first position is 2m from the rightmost side (close to the front side) of the truck loading compartment, which corresponds to the bottom of the loader 9. When a speed bump is used, the first position is set with the first speed bump in the direction of the truck's travel.

[0053] The computer vision processing module 4 obtains the information of four corner points on the side of the loading compartment based on the loading camera 6, which are named as point A(x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, since the position of the loading monitoring camera 6 remains unchanged, the coordinate origin of the images obtained before and after remains unchanged, and retrieving the loading compartment size information of the truck, and using formula (1) to calculate the ratio t between the image obtained by the loading monitoring camera 6 and the actual size

[0054]

[0055] Set the maximum loading height. In order to prevent the loading height from exceeding the height of the loading compartment and avoid coal spillage during transportation, and to adapt to the height of the loading area, a unified maximum loading height h is set for all trucks 2. max ;

[0056] The integrated control and processing center 5 controls the coal feeder to release coal. The computer vision processing module 4 obtains the outer contour of the loaded coal body (along the length direction of the truck, that is, approximately triangular in the left and right directions) through the loading monitoring camera 6. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. For example, when it reaches less than h max When the load is about 20 cm, the information is fed back to the integrated control and processing center 5 and the coal feeder is controlled to stop loading coal; when the last coal loading is completed, the coal loading is stopped when the rated load is reached;

[0057] Before stopping the coal loading, the loading monitoring camera 6 acquires the image in real time, and the computer vision processing module 4 passes through the highest point E of the outer contour. jThe horizontal line is the dividing line (1≤j≤n), and the optical flow method is used to calculate the dust movement direction at the outer contours on both sides of the area below the dividing line. The optical flow method is calculated as follows:

[0058] I i (x i ,y i ,t)=I i (x i +dx i ,y i +dy i ,t+dt) (2)

[0059] Get the dust in the horizontal direction u i The velocity component and the vertical velocity component v i

[0060]

[0061] Then the dust movement direction at the outer contour is θ i

[0062]

[0063] According to the dust movement direction in the left area of ​​the outer contour, the angle of the coal pile on the left side of the area blocked by the loading carriage is predicted

[0064]

[0065] According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted

[0066]

[0067] The highest point of the outer contour of the first loading coal pile is E1, the right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the top surface of the carriage is V 1r , calculate its intersection point with the right side of the carriage T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , and the intersection point V with the left side of the top surface of the carriage 1l ; To obtain the mass of the right half of the first loading coal pile, the first method is to approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r The angle θ 1r According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the right half of the first loading coal pile can be obtained; the second method is: the right half of the first loading coal pile is divided into two parts, the first part is E1V 1rThe corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1r According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a For the first part, E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can find the mass of the right half of the coal pile for the first loading;

[0068] S3: Truck 2 moves forward 2 m according to the instructions and stops at the second position. When a laser rangefinder is used, it moves forward to the second position according to the instructions of the signal sign 7. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel.

[0069] The integrated control and processing center 5 controls the coal feeder to release coal. The computer vision processing module 4 obtains the outer contour of the loaded coal body (along the length direction of the truck, that is, approximately triangular in the left and right directions) through the loading monitoring camera 6. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. For example, when it reaches less than h max When the depth is about 20 cm, the information is fed back to the integrated control and processing center 5 and the coal feeder is controlled to stop loading coal;

[0070] Before stopping the coal loading, the loading monitoring camera 6 acquires images in real time. The computer vision processing module 4 uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line, and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading compartment is predicted based on the dust movement direction of the area on the left side of the outer contour in the same way as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ;

[0071] The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the top surface of the carriage is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ of the first loading coal pile 1l , we can calculate the intersection point T2r (Also T 1l ), the intersection point of the second loading coal pile and the left side of the carriage top surface is V 2l ; Obtain the mass of the right half of the second loading coal pile, the method is the same as step S2; Obtain the mass of the left half of the first loading coal pile, the method is similar to step S2, the first method is: approximate the left half of the first loading coal pile to a trapezoid, where E1T 1l The angle θ 1l According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the left half of the first loading coal pile can be obtained; the second method is: the left half of the first loading coal pile is divided into two parts, the first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l The angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can find the mass of the left half of the coal pile for the first loading;

[0072] S4: Truck 2 moves forward every 2 m according to the instructions, and calculates the mass of the coal pile according to the method of step S3, until it moves forward to the last, i.e., the Nth coal loading position, and stops at the Nth position; when a laser rangefinder is used, it moves forward to the Nth position according to the indication information of the signal sign 7, and when a speed bump is used, the Nth position is set with the Nth speed bump in the direction of the truck's travel;

[0073] The integrated control and processing center 5 controls the coal feeder to discharge coal. The computer vision processing module 4 obtains the outer contour of the loaded coal through the loading monitoring camera 6 (along the length direction of the truck, that is, approximately triangular in the left and right directions). When the last time, i.e., the Nth time of coal loading, the loading monitoring camera 6 obtains the image in real time when the outer contour appears. The computer vision processing module 4 passes through the highest point E of the outer contour of the Nth coal pile. nThe horizontal line is taken as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the coal pile angle θ on the left side of the area blocked by the loading carriage according to the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ;

[0074] The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the top surface of the carriage is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , we can calculate the intersection point T nr (Also T n-1l ), the intersection point between the left side of the coal pile and the top surface of the carriage for the Nth loading is V nl , and the intersection point with the left side of the carriage is T nl ; Obtain the mass of the right half of the coal pile for the Nth loading, using the same method as step S2; Obtain the mass of the left half of the coal pile for the Nth and N-1th loading, using the same method as step S3;

[0075] S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

[0076] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all of these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims

1. A coal loading control system based on computer vision, including a vehicle recognition camera; characterized in that: The vehicle recognition camera is arranged in the waiting area, and is used to obtain the license plate information of the front-ranked truck waiting in the waiting area. The vehicle recognition camera is connected to the vehicle recognition processing module; the vehicle recognition processing module internally stores the license plate information of all trucks, as well as the truck loading compartment size information and rated load capacity information corresponding to the license plate, the truck loading compartment size information includes the length, width and height of the truck loading compartment, and also includes the number of loading times information; the vehicle recognition processing module receives the license plate information measured by the vehicle recognition camera, and extracts the loading compartment size information and rated load capacity information of the truck corresponding to the license plate, the vehicle recognition processing module is connected to the integrated control and processing center, and transmits the extracted loading compartment size information and rated load capacity information of the truck corresponding to the license plate to the integrated control and processing center; The loading monitoring camera is arranged on the front side of the loading area, the loading camera is connected to a computer vision processing module, and the computer vision processing module is connected to a comprehensive control and processing center; A coal feeder is arranged above the truck travel route in the loading area, and the coal feeder is connected to the integrated control and processing center, and the integrated control and processing center is connected to control the opening and closing and working speed of the coal feeder. The loading compartments of all trucks have the same width and height. The width of the coal feeder coal belt is the same as the width of the truck loading compartment. The coal conveying direction of the coal feeder coal belt is the same as the traveling direction of the truck. The coal feeder coal belt can discharge a first length of coal in the length direction of the loading compartment at one time. The loading number N information is how many times a truck is loaded; the length of the truck is an integer multiple of the first length; The laser rangefinder is arranged at the front side of the truck's route to measure the position of the truck in the direction of travel of the loading area. The signal indicator board is arranged at the front side of the truck's route to provide forward and stop information to the truck driver, and the position information measured by the laser rangefinder is displayed on the signal indicator board. The signal indicator board and the laser rangefinder are both connected to the integrated control and processing center. The measurement data of the laser rangefinder is transmitted to the integrated control and processing center, and the integrated control and processing center processes the data and displays it through the signal indicator board.

2. The coal loading control system according to claim 1 is characterized in that: As a parallel technical solution, several speed bumps are arranged at intervals on the truck's route, and the interval between the speed bumps is a first length. Every time the truck moves forward to a speed bump, it stops and releases coal once; a signal sign is arranged on the front side of the truck's route to provide forward information to the truck driver, and the signal sign is connected to the integrated control and processing center.

3. A coal loading control method based on computer vision, characterized in that: The steps include: S1: A truck drives to the waiting area and waits. When it is at the front of the queue, the vehicle recognition camera reads the license plate information of the truck and transmits the license plate information to the vehicle recognition processing module. The vehicle recognition processing module obtains the loading compartment size information and rated load information of the truck according to the information pre-stored therein. The loading compartment size information includes length, width and height, and transmits the information to the integrated control and processing center. All trucks have the same width and height. S2: After the previous truck has finished loading, the truck at the front of the waiting area enters the loading area. One truck enters the loading area at a time and only one truck is loaded at a time. The integrated control and processing center controls the loading according to the obtained loading compartment size information and rated load information of the truck. After the truck enters the loading area, it first stops at the first position according to the instructions. When a laser rangefinder is used, it moves to the first position according to the instructions of the signal sign. At the first position, the first length part of the rightmost side of the truck loading compartment corresponds to the bottom of the loader. When a speed bump is used, the first position is set with the first speed bump in the direction of the truck's travel. The computer vision processing module obtains the four corner point information of the side of the loading compartment based on the loading camera, which are named point A(x A ,y A ), point B(x B ,y B ), point C(x C ,y C ), point D(x D ,y D ), taking the lower left corner of the image as the coordinate origin O, retrieve the size information of the truck’s loading compartment, and use formula (1) to calculate the ratio t between the image obtained by the loading monitoring camera and the actual size Set a uniform maximum loading height h for all trucks max ; The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. During the last loading, the coal loading is stopped when the rated loading volume is reached. Before stopping the coal loading, the loading monitoring camera acquires the image in real time. The computer vision processing module passes through the highest point E of the outer contour. j The horizontal line is taken as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line, including predicting the angle of the coal pile on the left side of the area blocked by the loading carriage according to the dust movement direction of the area on the left side of the outer contour. According to the dust movement direction in the right area of ​​the outer contour, the angle of the coal pile on the right side of the area blocked by the loading carriage is predicted The highest point of the outer contour of the first loading coal pile is E1, the right side is the front direction of the car, and the left side is the rear direction of the car. The right half of the coal pile angle θ of the first loading coal pile is obtained. 1r , and the intersection with the right side of the top surface of the carriage is V 1r , calculate its intersection point with the right side of the carriage T 1r ; Get the left half coal pile angle θ of the first loading coal pile 1l , and the intersection point V with the left side of the top surface of the carriage 1l ; Calculate the mass of the right half of the coal pile for the first loading; S3: the truck moves forward a first distance according to the instruction and stops at a second position. When a laser rangefinder is used, the truck moves forward to the second position according to the instruction information of the signal sign. When a speed bump is used, the second position is set with a second speed bump in the direction of the truck's travel. The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the maximum height of the outer contour approaches the maximum loading height, the coal loading is stopped. Before stopping the coal loading, the loading monitoring camera acquires the image in real time. The computer vision processing module uses the horizontal line passing through the highest point E2 of the outer contour of the second coal pile as the dividing line, and uses the optical flow method to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The angle θ of the coal pile on the left side of the area blocked by the loading carriage is predicted based on the dust movement direction of the area on the left side of the outer contour in the same way as step S2. 2l , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage 2r ; The highest point of the outer contour of the second loading coal pile is E2, and the intersection with the right side of the top surface of the carriage is V 2r , the right side is the first loading coal pile, according to the right half of the second loading coal pile angle θ 2r and the left half of the coal pile angle θ of the first loading coal pile 1l , calculate the intersection point T 2r , that is, T 1l The intersection point of the second loading coal pile and the left side of the carriage top surface is V 2l ; Calculate the mass of the right half of the coal pile for the second loading; Calculate the mass of the left half of the coal pile for the first loading; S4: the truck moves forward once every first length according to the instruction, and calculates the mass of the coal pile according to the method of step S3, until it moves forward to the last, i.e., the Nth coal loading position, and stops at the Nth position; when a laser rangefinder is used, the truck moves forward to the Nth position according to the instruction information of the signal sign, and when a speed bump is used, the Nth speed bump in the direction of the truck's travel is set at the Nth position; The integrated control and processing center controls the coal feeder to release coal. The computer vision processing module obtains the outer contour of the loaded coal through the loading monitoring camera. When the last time, i.e., the Nth time, coal is loaded, the loading monitoring camera obtains the image in real time when the outer contour appears. The computer vision processing module passes through the highest point E of the outer contour of the Nth coal pile. n The horizontal line is taken as the dividing line, and the optical flow method is used to calculate the dust movement direction of the outer contour on both sides of the area below the dividing line. The same method as step S2 is used to predict the coal pile angle θ on the left side of the area blocked by the loading carriage according to the dust movement direction of the area on the left side of the outer contour. nl , according to the dust movement direction in the right area of ​​the outer contour, predict the coal pile angle θ on the right side of the area blocked by the loading carriage nr ; The highest point of the outer contour of the coal pile for the Nth loading is E n , and the intersection with the right side of the top surface of the carriage is V nr , the right side is the N-1th loading coal pile, according to the angle θ of the right side coal pile of the Nth loading nr and the left side coal pile angle θ of the first loading nl , calculate the intersection point T nr , that is, T n-1l , the intersection point between the left side of the coal pile and the top surface of the carriage for the Nth loading is V nl , and the intersection point with the left side of the carriage is T nl ; Calculate the mass of the right half of the coal pile for the Nth loading; Calculate the mass of the left half of the coal pile for the Nth and N-1th loadings; S5: Accumulate the total coal loading mass calculated from these N times of coal loading, and stop coal loading when the rated loading mass is reached.

4. The coal loading control method according to claim 3 is characterized in that: In step S2, the first method for obtaining the mass of the right half of the first loading coal pile is to approximate the right half of the first loading coal pile to a trapezoid, where E1T 1r The angle θ 1r According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the right half of the first loading coal pile can be calculated.

5. The coal loading control method according to claim 3, characterized in that: In step S2, the second method for obtaining the mass of the right half of the first loading coal pile is: the right half of the first loading coal pile is divided into two parts, the first part is E1V 1r The corresponding coal pile directly below, the second part is V 1r T 1r The corresponding coal pile directly below, where V 1r T 1r The angle is θ 1r According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a ; For the first part, replace E1V 1r Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can calculate the mass of the right half of the coal pile for the first loading.

6. The coal loading control method according to claim 4 or 5, characterized in that: In step S3, the mass of the right half of the coal pile for the second loading is obtained in the same manner as in step S2.

7. The coal loading control method according to claim 3, characterized in that: In step S3, the first method for obtaining the mass of the left half of the first loading coal pile is to approximate the left half of the first loading coal pile to a trapezoid, where E1T 1l The angle θ 1l According to the density of coal, the width of the loading compartment, and the ratio t of the image to the actual size, the mass of the left half of the first loading coal pile can be calculated.

8. The coal loading control method according to claim 3, characterized in that: In step S3, the mass of the left half of the first loading coal pile is obtained. The second method is: the left half of the first loading coal pile is divided into two parts, the first part is E1V 1l The corresponding coal pile directly below, the second part is V 1l T 1l The corresponding coal pile directly below, where V 1l T 1l The angle θ 1l According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile can be obtained. a For the first part, E1V 1l Divide it into several sections along the left and right directions. Each section is approximately rectangular. Calculate the area and add them up to get the area of ​​the first section. According to the density of coal, the width of the loading compartment and the ratio t between the image and the actual size, the mass M of the coal pile in this section can be obtained. b , the first part of the coal pile mass M b and the second part coal pile mass M a By adding them together, we can calculate the mass of the left half of the coal pile for the first loading.

9. The coal loading control method according to claim 4 or 5, characterized in that: In step S4, the mass of the right half of the coal pile loaded for the Nth time is calculated in the same manner as in step S2.

10. The coal loading control method according to claim 7 or 8, characterized in that: In step S4, the mass of the left half of the coal pile after the Nth and N-1th loading is obtained in the same manner as in step S3.

Images