A device and method for measuring the volume of manufactured sand and gravel stockpiles

Abstract

This invention discloses a device and method for measuring the volume of manufactured sand and gravel stockpiles. The measuring device includes binocular visible light cameras, a dust removal mechanism, stockpile markers, a PoE switch, an optical module, and a server processing unit. Two sets of binocular visible light cameras are respectively installed on the enclosures on both sides of the stockpile, and each set of binocular visible light cameras is equipped with a dust removal mechanism. Two stockpile markers are located on both sides of the stockpile, and the line connecting the two stockpile markers intersects with the line connecting the two sets of binocular visible light cameras. The binocular visible light cameras send the captured image data to the drive circuit of the dust removal mechanism, which is connected to the PoE switch. The optical module converts the electrical signal from the PoE switch into an optical signal and transmits it to the server processing unit. The server processing unit calculates the volume of the stockpile using the triangular facets of the three-dimensional mesh surface of the stockpile captured by the two sets of binocular visible light cameras. This invention is low in cost, can solve the problem of dust accumulation in cameras within a closed silo, and improves the accuracy of stockpile volume measurement.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a device and method for measuring the volume of manufactured sand and gravel stockpiles. Background Technology

[0002] Sand and gravel are essential raw materials for the construction industry, and my country is the world's largest producer and consumer of them. With increasingly tight constraints on natural sand and gravel resources and growing environmental protection efforts, manufactured sand and gravel are gradually becoming the main source of construction sand and gravel in my country. Currently, my country's manufactured sand and gravel production has gradually transformed from simple, scattered manual or semi-mechanized workshops to standardized, automated, and large-scale factories. In automated manufactured sand and gravel production lines, the management of finished product silos is a crucial link, and the measurement of sand and gravel stockpile volume is a primary task in this management.

[0003] A binocular vision-based material pile volume measurement device generally includes binocular vision perception, data transmission, and server processing. Chinese invention patent 200910054129.X discloses a visual measurement system for large material piles in a stockyard. First, this solution can provide a rough calculation of the volume of large open-air material piles, but manufactured sand piles are located in enclosed silos with high dust levels, making the binocular vision system prone to dust accumulation and poor imaging. Second, this solution uses two sets of binocular vision systems, one close to the material pile and the other located within the discharge port track, making installation and subsequent maintenance complex. Third, the image data acquisition and transmission module of this solution includes a CCD camera, photoelectric conversion, image processing module, data bus, and industrial Ethernet, resulting in a complex structure that is not conducive to maintenance and expansion. Fourth, the server processing module of this solution lacks an imaging time synchronization module, making it difficult to coordinate the imaging time of the two binocular vision systems. Furthermore, the lack of a filtering module may introduce noise data during 3D reconstruction, affecting the final measurement accuracy.

[0004] Binocular vision-based methods for measuring stockpile volume generally include camera calibration, image acquisition, image correction, stereo matching, 3D reconstruction, surface interpolation, and volume measurement. Chinese invention patent 200910054130.2 discloses a visual measurement method for large stockpile yards. First, this method lacks a filtering module, potentially introducing noisy data during 3D reconstruction, which, without filtering, will affect the final measurement accuracy. Second, this method uses the product of the projected area and height of a pixel in the pixel plane in the world coordinate system as the smallest volume unit, summing these units to obtain the stockpile volume. This method relies on the accuracy of pixels in the depth map; the more noisy pixels there are, the more divergent the measurement results become.

[0005] Other publicly available binocular vision-based material pile volume measurement devices and methods typically rely on lidar, depth sensors, etc., to obtain high-density point cloud data and depth images, achieve three-dimensional reconstruction, and then estimate the material pile volume. This approach has high instrument costs, high requirements for environmental parameters, and is not suitable for enclosed spaces with high dust levels. Summary of the Invention

[0006] In view of this, the present invention provides a device and method for measuring the volume of manufactured sand and gravel stockpiles, which is low in cost, can solve the problem of dust accumulation in cameras inside enclosed silos, and improve the accuracy of stockpile volume measurement.

[0007] The technical solution adopted in this invention is as follows:

[0008] A device for measuring the volume of manufactured sand and gravel stockpiles includes two sets of binocular visible light cameras, two sets of dust removal mechanisms, two stockpile markers, a POE switch, an optical module, and a server processing unit.

[0009] Two sets of binocular visible light cameras are respectively installed on the fences on both sides of the sand and gravel pile. Each set of binocular visible light cameras is equipped with a dust removal mechanism. Two pile markers are located on both sides of the sand and gravel pile, and the line connecting the two pile markers intersects with the line connecting the two sets of binocular visible light cameras.

[0010] The binocular visible light camera sends the captured image data to the drive circuit of the dust removal mechanism. The drive circuit of the dust removal mechanism is connected to the POE switch. The optical module converts the electrical signal of the POE switch into an optical signal and transmits it to the server processing unit. The server processing unit is used to calculate the volume of the material pile using the triangular facets of the three-dimensional mesh surface of the material pile from the two sets of binocular visible light cameras.

[0011] Furthermore, the server processing unit includes an instruction synchronization module, a dust removal module, a camera focusing module, a camera calibration module, an image recognition module, a stereo matching module, a filtering module, a three-dimensional reconstruction module, and a volume calculation module;

[0012] The instruction synchronization module is used to send photo-taking instructions and focus instructions to the two sets of binocular visible light cameras, and to send dust-cleaning instructions to the two sets of dust-cleaning mechanisms.

[0013] The dust removal module includes manual dust removal and automatic dust removal. Manual dust removal is performed by the user issuing a dust removal command, which is then executed by the dust removal mechanism. Automatic dust removal is performed by the user issuing a dust removal command based on the gradient value calculated by the image recognition module, which is then executed by the dust removal mechanism.

[0014] The camera focusing module is used for focusing the binocular visible light camera;

[0015] The camera calibration module is used to obtain the intrinsic and extrinsic parameters of two sets of binocular visible light cameras;

[0016] The image recognition module is used to calculate the gradient values ​​of the image;

[0017] The stereo matching module is used to perform epipolar correction on the stockpile image based on the intrinsic and extrinsic parameters to obtain the original disparity map.

[0018] The filtering module is used to filter the original disparity map;

[0019] The 3D reconstruction module is used to generate a 3D mesh surface using the filtered disparity map;

[0020] The volume calculation module is used to calculate the total volume of the material pile using the triangular facets of the three-dimensional mesh surface of two sets of binocular visible light cameras.

[0021] Furthermore, the dust removal mechanism also includes a dust cover, a protective shell, a cleaning brush head, a connecting rod, a drive motor, and a base;

[0022] The dust cover is fixed to the top two sides of the binocular visible light camera, the protective shell is fixed to the base, the drive motor is located inside the protective shell, and the drive motor is rotatably connected to the cleaning brush head through the connecting rod. The cleaning brush head is used to remove dust from the surface of the binocular visible light camera. The drive circuit is located inside the protective shell, receives signals from the server processing unit, and controls the drive motor to drive the connecting rod to reciprocate.

[0023] Furthermore, the drive circuit is connected to the PoE switch via a copper network cable, and the PoE switch is connected to the server processing unit via an optical fiber.

[0024] The present invention also provides a method for measuring the volume of manufactured sand and gravel stockpiles, using the aforementioned manufactured sand and gravel stockpile volume measuring device, and the measurement method steps are as follows:

[0025] Step 1: The instruction synchronization module sends a cleaning instruction to the two sets of cleaning mechanisms, and the two sets of cleaning mechanisms execute the cleaning action.

[0026] Step 2: The command synchronization module sends a photo-taking command to the two sets of binocular visible light cameras. After taking photos, the two sets of binocular visible light cameras upload them to the image recognition module. The image recognition module calculates the gradient value and determines whether to request a focus adjustment command. If a focus adjustment command is requested, Step 2 is repeated; otherwise, the focus adjustment is completed.

[0027] Step 3: Use the camera calibration module to obtain the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras;

[0028] Step 4: The stereo matching module performs epipolar correction on the stockpile image based on the intrinsic and extrinsic parameters to obtain the original disparity map;

[0029] Step 5: The filtering module filters the original disparity map;

[0030] Step 6: Calculate the 3D point cloud from the disparity map generated by the filtering module, and generate a 3D mesh surface;

[0031] Step 7: The volume calculation module uses the triangular facets of the three-dimensional mesh surface from two sets of binocular visible light cameras to calculate the total volume of the material pile.

[0032] Furthermore, if step 1 involves automatic dust removal, then the automatic dust removal method is as follows:

[0033] Step 101: Request the command synchronization module to send a photo-taking command;

[0034] Step 102: The instruction synchronization module sends a photo-taking instruction to the two sets of binocular visible light cameras;

[0035] Step 103: After taking pictures with the two sets of binocular visible light cameras, the images are uploaded to the image recognition module;

[0036] Step 104: The image recognition module calculates the gradient value of the image;

[0037] Step 105: Determine whether to request a dust removal command to the command synchronization module. If yes, proceed to step 106; otherwise, repeat steps 101 to 105.

[0038] Step 106: The instruction synchronization module sends cleaning instructions to the two sets of cleaning mechanisms;

[0039] Step 107: The two dust removal mechanisms perform the dust removal action;

[0040] Step 108: Feedback on whether to execute the action is sent to the instruction synchronization module.

[0041] Furthermore, in step 3, the camera calibration module uses Zhang's calibration method to calculate the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras.

[0042] Furthermore, the material pile image correction described in step 4 requires noise reduction, which is performed using the following method:

[0043] Step 401: Calculate the pixel coordinate values ​​of the two material pile markings using the image recognition module;

[0044] Step 402: Based on the pixel coordinate values ​​of the material pile marking, crop the images of the material pile captured by the two sets of binocular visible light cameras so that the images exactly include the material pile marking;

[0045] Step 403: Use Gaussian filtering to denoise the cropped image.

[0046] Furthermore, the filtering method in step 5 is as follows:

[0047] Step 501: Perform left-right consistency detection on the original disparity map and remove outliers;

[0048] Step 502: Fill the voids in the result of step 501;

[0049] Step 503: Perform weighted least squares filtering on the result after filling in step 502;

[0050] Step 504: Save the filtered disparity map.

[0051] Furthermore, the calculation method in step 7 is as follows:

[0052] Step 701: Select the first set of three-dimensional mesh surfaces generated by the binocular visible light camera, and convert the pixel coordinates of the two sets of material pile markings obtained by the icon recognition module to the world coordinate system;

[0053] Step 702: Construct a zero plane from the coordinates of the two sets of material piles in the world coordinate system;

[0054] Step 703: For the three-dimensional mesh surface generated by the three-dimensional reconstruction module, select each triangular facet in turn, project it onto the zero plane, and calculate the volume of the right triangular prism formed by the projected triangle and the original triangular facet.

[0055] Step 704: Traverse all triangles and repeat steps 702 to 703 to calculate the volume of all triangular prisms;

[0056] Step 705: Sum the volumes of all triangular prisms to obtain the volume V1 of the stockpile as calculated by the first set of binocular visible light cameras;

[0057] Step 706: Perform the operations on the three-dimensional mesh surface generated by the first set of binocular visible light cameras in steps 701 to 705 on the three-dimensional mesh surface generated by the second set of binocular visible light cameras to obtain the volume V2 of the material pile as captured and calculated by the second set of binocular visible light cameras.

[0058] Step 707: Calculate the total volume of the stockpile, V = V1 + V2.

[0059] Beneficial effects:

[0060] 1. This invention utilizes two sets of conventional binocular visible light vision cameras, networked and transmitted via a PoE switch, reducing the cost of measuring the volume of manufactured sand stockpiles. Furthermore, the two sets of binocular visible light cameras are directly installed on the enclosed enclosure of the sand and gravel stockpile, simplifying installation. Secondly, by adding a dust removal mechanism to the cameras, the problem of dust accumulation inside the enclosed enclosure is solved. Thirdly, high-density triangular facets are used to calculate the volume, improving the accuracy of stockpile volume measurement.

[0061] 2. This invention uses a filtering module algorithm to improve the accuracy of three-dimensional reconstruction of the material pile and further improve the accuracy of material pile volume measurement.

[0062] 3. The dust removal module of this invention includes manual dust removal and automatic dust removal. Under normal operating conditions, the automatic dust removal mode can automatically remove dust according to the preset program, realizing unattended operation; the manual dust removal mode provides the system with important operational flexibility and emergency protection. When encountering sudden high dust load or automatic control failure, the operator can immediately use the manual mode to carry out targeted cleaning, which can significantly improve the adaptability, reliability and maintenance efficiency of the equipment. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall installation of the present invention.

[0064] Figure 2 This is a schematic diagram of the installation structure of a binocular visible light camera and a dust removal mechanism.

[0065] Figure 3 This is a schematic diagram of the material pile marking structure.

[0066] Figure 4 This describes the installation and implementation process of the present invention.

[0067] Figure 5 This is a diagram showing the modular composition of the server processing unit.

[0068] Figure 6 This describes the execution flow of the instruction synchronization module.

[0069] Figure 7 This describes the execution flow of the dust removal module.

[0070] Figure 8 This describes the execution flow of the focusing module.

[0071] Figure 9 The execution flow of the camera calibration module.

[0072] Figure 10 This refers to the algorithm components of the image recognition module.

[0073] Figure 11 This describes the execution flow of the stereo matching module.

[0074] Figure 12 This describes the execution flow of the filtering module.

[0075] Figure 13 This describes the execution flow of the 3D reconstruction module.

[0076] Figure 14 This describes the execution flow of the volume calculation module.

[0077] Among them, 1-binocular vision unit, 2-data acquisition and transmission unit, 3-server processing unit, 1-1-1-dust cover, 1-1-2-fixing screw, 1-1-3-binocular camera, 1-1-4-protective shell, 1-1-5-base, 1-1-6-expansion screw hole, 1-1-7-cleaning brush head, 1-1-8-connecting rod, 1-1-9-drive motor, 1-1-10-drive circuit, 1-2-1-marking body, 1-2-2-base, 1-2-3-fixing hole. Detailed Implementation

[0078] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0079] The present invention provides a device for measuring the volume of manufactured sand and gravel stockpiles, comprising: a binocular vision unit 1, a data acquisition and transmission unit 2, and a server processing unit 3. The binocular vision unit 1 is used to acquire visible light images of the stockpiles, and the data acquisition and transmission unit 2 is used to transmit the visible light images to the server processing unit 3. The server processing unit 3 consists of server hardware devices and software modules, and is used for three-dimensional reconstruction and measurement.

[0080] Specifically, in this embodiment, the binocular vision unit 1 includes: two sets of binocular visible light cameras, two sets of dust removal mechanisms, and two material pile markers; the data acquisition and transmission unit 2 includes: a POE switch, a copper network cable, an optical cable, and an optical module; and the server processing unit 3 includes: a switch and a server.

[0081] like Figure 1 As shown, two sets of binocular visible light cameras are respectively installed on the fences on both sides of the sand and gravel pile. Each set of binocular visible light cameras is equipped with a dust removal mechanism. The binocular visible light cameras are fixed to the protective shell 1-1-4 of the dust removal mechanism by screws and connected to the drive circuit 1-1-10 of the dust removal mechanism by copper cable network cable.

[0082] like Figure 2As shown, the dust removal mechanism includes a dust cover 1-1-1, a protective shell 1-1-4, a cleaning brush head 1-1-7, a connecting rod 1-1-8, a drive motor 1-1-9, a drive circuit 1-1-10, and a base 1-1-5. The dust cover 1-1-1 is fixed to the top sides of the binocular camera 1-1-3 by two fixing screws 1-1-2. The protective shell 1-1-4 is welded to the base 1-1-5. The base 1-1-5 contains four expansion screw holes 1-1-6 for fixing the entire dust removal mechanism. The drive motor 1-1-9 is located inside the protective shell 1-1-4. The signal and power lines of the drive motor 1-1-9 are connected to the drive circuit 1-1-8. 10 is connected (drive motor 1-1-9 is a common stepper motor, containing four control signal lines and two power lines, all connected to drive circuit 1-1-10); drive motor 1-1-9 is rotatably connected to cleaning brush head 1-1-7 via connecting rod 1-1-8. Cleaning brush head 1-1-7 is used to remove dust from the surface of the binocular visible light camera; drive circuit 1-1-10 is located inside protective shell 1-1-4 and is connected to the POE switch of data acquisition unit via copper cable network cable. It receives signals from server processing unit 3 and controls drive motor 1-1-9 to drive connecting rod 1-1-8 to reciprocate, thereby achieving the purpose of cleaning binocular camera 1-1-3.

[0083] like Figure 3 As shown, the material pile marker is a vertical pole, consisting of a 10cm wide steel bar, more than 400cm long, and 10cm thick, which is the main body of the marker 1-2-1. The surface is coated with 5cm wide alternating black and white stripes, used by the image recognition module in server processing unit 3. The Canny edge detection algorithm is used to quickly locate the position of the material pile marker in the image coordinate system. The bottom of the main body 1-2-1 is a base 1-2-2, on which are machined fixing holes 1-2-3, which are screw holes.

[0084] In this embodiment, the PoE (Power over Ethernet Switch) switch is a standard 8-port PoE switch. One port connects to an optical module, which is connected to the server processing unit 3 via an optical cable; the other two ports are connected to two sets of dust removal mechanisms via copper network cables. Of course, the PoE switch can also be a 4-port or 16-port type.

[0085] The copper network cable is a standard Category 7A copper network cable, used to connect the dust removal mechanism and the binocular visible light camera, as well as to connect the dust removal mechanism and the PoE switch.

[0086] The optical cable is a standard 12-core optical cable, used for connecting the PoE switch and server processing unit 3.

[0087] The optical module is a standard 10 Gigabit optical-to-electrical converter, used to convert the electrical signals of the PoE switch into optical signals and transmit the data to the server processing unit 3 via optical fiber.

[0088] The switch is a standard switch. One port of the switch is connected to an optical module, which is connected to the data acquisition and transmission unit 2 via an optical cable; the other port is connected to the server processing unit 3 via a standard Category 7A copper network cable.

[0089] Server processing unit 3 is a general-purpose computing server, which is connected to the switch via a standard Category 7A copper network cable.

[0090] like Figure 5 As shown, the server processing unit 3 includes an instruction synchronization module, a dust removal module, a camera focusing module, a camera calibration module, an image recognition module, a stereo matching module, a filtering module, a three-dimensional reconstruction module, and a volume calculation module.

[0091] The command synchronization module is used to send shooting commands and focus commands to the two sets of binocular visible light cameras, and to send cleaning commands to the two sets of cleaning mechanisms; for example... Figure 6 As shown, the execution process is as follows:

[0092] Send instructions to two sets of binocular visible cameras and a dust removal mechanism → The two sets of binocular visible cameras and a dust removal mechanism execute the instructions → The two sets of binocular visible cameras and a dust removal mechanism return the execution results → Save and upload the instruction execution results.

[0093] The dust removal module includes manual dust removal and automatic dust removal. Manual dust removal is performed by the user issuing a dust removal command, which is then executed by the dust removal mechanism. Automatic dust removal is performed by the user issuing a dust removal command based on the gradient value calculated by the image recognition module, which is then executed by the dust removal mechanism.

[0094] The camera focusing module is used for focusing a binocular visible light camera.

[0095] The camera calibration module is used to obtain the intrinsic and extrinsic parameters of two sets of binocular visible light cameras.

[0096] The image recognition module is used to calculate the gradient values ​​of the image, and includes two functions: calculating the position of the material pile marker in the pixel coordinate system using the Canny edge detection algorithm and calculating the gradient values ​​of the image, such as... Figure 10 As shown.

[0097] The stereo matching module is used to perform epipolar correction on the stockpile image based on the intrinsic and extrinsic parameters to obtain the original disparity map.

[0098] The filtering module is used to filter the original disparity map.

[0099] The 3D reconstruction module is used to generate a 3D mesh surface using the filtered disparity map.

[0100] The volume calculation module is used to calculate the total volume of the stockpile using triangular patches of the three-dimensional mesh surface from two sets of binocular visible light cameras.

[0101] like Figure 4 As shown, during installation, firstly, the binocular vision unit 1 is installed, specifically including: two sets of binocular visible light cameras respectively installed with the dust removal mechanism, two sets of material pile markings installed, and the bases of the two sets of binocular visible light cameras fixed. Secondly, the data acquisition and transmission unit 2 is installed. Finally, the server processing unit 3 is deployed.

[0102] The present invention also provides a method for measuring the volume of manufactured sand and gravel stockpiles, using the aforementioned manufactured sand and gravel stockpile volume measuring device, and the measurement method steps are as follows:

[0103] Step 1: The instruction synchronization module sends a cleaning instruction to both sets of cleaning mechanisms, and the two sets of cleaning mechanisms execute the cleaning action; for example... Figure 7 As shown, the left column represents the execution modules, and the right column represents the execution flow.

[0104] If manual dust removal is required, the manual dust removal method is as follows:

[0105] Step 101: Request the instruction synchronization module to send a dust removal instruction;

[0106] Step 102: The instruction synchronization module sends a cleaning instruction to both sets of cleaning mechanisms;

[0107] Step 103: The two dust removal mechanisms perform the dust removal action;

[0108] Step 104: The two dust removal mechanisms will send feedback on whether to perform the action to the instruction synchronization module.

[0109] If automatic dust removal is enabled, the automatic dust removal method is as follows:

[0110] Step 101: Request the command synchronization module to send a photo-taking command;

[0111] Step 102: The instruction synchronization module sends a photo-taking instruction to the two sets of binocular visible light cameras;

[0112] Step 103: After taking pictures with the two sets of binocular visible light cameras, the images are uploaded to the image recognition module;

[0113] Step 104: The image recognition module calculates the gradient value of the image;

[0114] Step 105: Determine whether to request a dust removal command to the command synchronization module. If yes, proceed to step 106; otherwise, repeat steps 101 to 105.

[0115] Step 106: The instruction synchronization module sends cleaning instructions to the two sets of cleaning mechanisms;

[0116] Step 107: The two dust removal mechanisms perform the dust removal action;

[0117] Step 108: Feedback on whether to execute the action is sent to the instruction synchronization module.

[0118] Step 2, as follows Figure 8 As shown, a photo-taking command is requested from the command synchronization module. The command synchronization module sends the photo-taking command to two sets of binocular visible light cameras. After taking photos, the two sets of binocular visible light cameras upload them to the image recognition module. The image recognition module calculates the gradient value and determines whether to request a focus adjustment command. If a focus adjustment command is requested, step 2 is repeated; otherwise, focus adjustment is completed.

[0119] Step 3: Use the camera calibration module to obtain the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras; for example... Figure 9 As shown, the specific calibration method is as follows:

[0120] Step 301: Using Zhang's calibration method, prepare black and white checkerboard squares;

[0121] Step 302: Select a suitable location and place the chessboard grid so that both sets of binocular visible light cameras can capture the complete chessboard.

[0122] Step 303: Request the instruction synchronization module to send a photo-taking instruction;

[0123] Step 304: The instruction synchronization module sends a photo-taking instruction to the two sets of binocular visible light cameras;

[0124] Step 305: After taking pictures with the two sets of binocular visible light cameras, upload them to the camera calibration module;

[0125] Step 306: Repeat steps 302 to 305 to take multiple images;

[0126] Step 307: Using Zhang's calibration method, calculate the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras.

[0127] Step 4: The stereo matching module performs epipolar correction on the stockpile image based on intrinsic and extrinsic parameters to obtain the original disparity map; for example... Figure 11 As shown, the workflow of the stereo matching module is as follows:

[0128] Step 401: Calculate the pixel coordinate values ​​of the two material pile markings using the image recognition module;

[0129] Step 402: Based on the pixel coordinate values ​​of the material pile marking, crop the images of the material pile captured by the two sets of binocular visible light cameras so that the material pile image exactly includes the material pile marking;

[0130] Step 403: Use Gaussian filtering to denoise the cropped stockpile image;

[0131] Step 404: Using the intrinsic and extrinsic parameters obtained from the camera calibration module, perform epipolar correction on the denoised stockpile image;

[0132] Step 405: Calculate the disparity map using the SGM (Semi-Global Matching) algorithm;

[0133] Step 406: Save the original disparity map.

[0134] Step 5: The filtering module filters the original disparity map;

[0135] like Figure 12 As shown, the filtering method is as follows:

[0136] Step 501: Perform left-right consistency detection on the original disparity map and remove outliers;

[0137] Step 502: Fill the voids in the result of step 501;

[0138] Step 503: Perform weighted least squares filtering (WLS filtering) on ​​the results after filling in step 502.

[0139] Step 504: Save the filtered disparity map.

[0140] Step 6, as follows Figure 13 As shown, the disparity map generated by the filtering module is used to calculate the 3D point cloud, and the Ball Pivoting algorithm (BPA) is used to generate a 3D mesh surface, which is then saved. The Ball Pivoting algorithm is an algorithm for point cloud surface reconstruction that generates a triangular mesh model by simulating a sphere "rolling" on the point cloud surface.

[0141] Step 7: The volume calculation module uses the triangular facets of the three-dimensional mesh surface from two sets of binocular visible light cameras to calculate the total volume of the material pile.

[0142] like Figure 14 As shown, the calculation method steps are as follows:

[0143] Step 701: Select the first set of three-dimensional mesh surfaces generated by the binocular visible light camera, and convert the pixel coordinates of the two sets of material pile markings obtained by the icon recognition module to the world coordinate system;

[0144] Step 702: Construct a zero plane from the coordinates of the two sets of material piles in the world coordinate system;

[0145] Step 703: For the three-dimensional mesh surface generated by the three-dimensional reconstruction module, select each triangular facet in turn, project it onto the zero plane, and calculate the volume of the right triangular prism formed by the projected triangle and the original triangular facet.

[0146] Step 704: Traverse all triangles and repeat steps 702 to 703 to calculate the volume of all triangular prisms;

[0147] Step 705: Sum the volumes of all triangular prisms to obtain the volume V1 of the stockpile as calculated by the first set of binocular visible light cameras;

[0148] Step 706: Perform the operations on the three-dimensional mesh surface generated by the first set of binocular visible light cameras in steps 701 to 705 on the three-dimensional mesh surface generated by the second set of binocular visible light cameras to obtain the volume V2 of the material pile as captured and calculated by the second set of binocular visible light cameras.

[0149] Step 707: Calculate the total volume of the stockpile, V = V1 + V2.

[0150] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A device for measuring the volume of manufactured sand and gravel stockpiles, characterized in that, It includes two sets of binocular visible light cameras, two sets of dust removal mechanisms, two material pile markers, a PoE switch, optical modules, and a server processing unit; Two sets of binocular visible light cameras are respectively installed on the fences on both sides of the sand and gravel pile. Each set of binocular visible light cameras is equipped with a dust removal mechanism. Two pile markers are located on both sides of the sand and gravel pile, and the line connecting the two pile markers intersects with the line connecting the two sets of binocular visible light cameras. The binocular visible light camera sends the captured image data to the drive circuit of the dust removal mechanism. The drive circuit of the dust removal mechanism is connected to the POE switch. The optical module converts the electrical signal of the POE switch into an optical signal and transmits it to the server processing unit. The server processing unit is used to calculate the volume of the material pile using the triangular facets of the three-dimensional mesh surface of the material pile from the two sets of binocular visible light cameras. The server processing unit includes an instruction synchronization module, a dust removal module, a camera focusing module, a camera calibration module, an image recognition module, a stereo matching module, a filtering module, a three-dimensional reconstruction module, and a volume calculation module. The instruction synchronization module is used to send photo-taking instructions and focus instructions to the two sets of binocular visible light cameras, and to send dust-cleaning instructions to the two sets of dust-cleaning mechanisms. The dust removal module includes manual dust removal and automatic dust removal. Manual dust removal is performed by the user issuing a dust removal command, which is then executed by the dust removal mechanism. Automatic dust removal is performed by the user issuing a dust removal command based on the gradient value calculated by the image recognition module, which is then executed by the dust removal mechanism. The camera focusing module is used for focusing the binocular visible light camera; The camera calibration module is used to obtain the intrinsic and extrinsic parameters of two sets of binocular visible light cameras; The image recognition module is used to calculate the gradient values ​​of the image; The stereo matching module is used to perform epipolar correction on the stockpile image based on the intrinsic and extrinsic parameters to obtain the original disparity map. The filtering module is used to filter the original disparity map; The 3D reconstruction module is used to generate a 3D mesh surface using the filtered disparity map; The volume calculation module is used to calculate the total volume of the material pile using triangular facets of a three-dimensional mesh surface from two sets of binocular visible light cameras. The dust removal mechanism also includes a dust cover, a protective shell, a cleaning brush head, a connecting rod, a drive motor, and a base; The dust cover is fixed to the top two sides of the binocular visible light camera, the protective shell is fixed to the base, the drive motor is located inside the protective shell, and the drive motor is rotatably connected to the cleaning brush head through the connecting rod. The cleaning brush head is used to remove dust from the surface of the binocular visible light camera. The drive circuit is located inside the protective shell, receives signals from the server processing unit, and controls the drive motor to drive the connecting rod to reciprocate.

2. The device for measuring the volume of manufactured sand and gravel stockpiles as described in claim 1, characterized in that, The drive circuit is connected to the PoE switch via a copper network cable, and the PoE switch is connected to the server processing unit via an optical fiber cable.

3. A method for measuring the volume of manufactured sand and gravel stockpiles, characterized in that, The measurement method using the manufactured sand and gravel stockpile volume measuring device as described in claim 1 is as follows: Step 1: The instruction synchronization module sends a cleaning instruction to the two sets of cleaning mechanisms, and the two sets of cleaning mechanisms execute the cleaning action. Step 2: The command synchronization module sends a photo-taking command to the two sets of binocular visible light cameras. After taking photos, the two sets of binocular visible light cameras upload them to the image recognition module. The image recognition module calculates the gradient value and determines whether to request a focus adjustment command. If a focus adjustment command is requested, Step 2 is repeated; otherwise, the focus adjustment is completed. Step 3: Use the camera calibration module to obtain the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras; Step 4: The stereo matching module performs epipolar correction on the stockpile image based on the intrinsic and extrinsic parameters to obtain the original disparity map; Step 5: The filtering module filters the original disparity map; Step 6: Calculate the 3D point cloud from the disparity map generated by the filtering module, and generate a 3D mesh surface; Step 7: The volume calculation module uses the triangular facets of the three-dimensional mesh surface from two sets of binocular visible light cameras to calculate the total volume of the material pile.

4. The method for measuring the volume of manufactured sand and gravel stockpiles as described in claim 3, characterized in that, Step 1 involves automatic dust removal, and the automatic dust removal method is as follows: Step 101: Request the command synchronization module to send a photo-taking command; Step 102: The instruction synchronization module sends a photo-taking instruction to the two sets of binocular visible light cameras; Step 103: After taking pictures with the two sets of binocular visible light cameras, the images are uploaded to the image recognition module; Step 104: The image recognition module calculates the gradient value of the image; Step 105: Determine whether to request a dust removal command to the command synchronization module. If yes, proceed to step 106; otherwise, repeat steps 101 to 105. Step 106: The instruction synchronization module sends cleaning instructions to the two sets of cleaning mechanisms; Step 107: The two dust removal mechanisms perform the dust removal action; Step 108: Feedback on whether to execute the action is sent to the instruction synchronization module.

5. The method for measuring the volume of manufactured sand and gravel stockpiles as described in claim 3, characterized in that, In step 3, the camera calibration module uses Zhang's calibration method to calculate the intrinsic and extrinsic parameters of the two sets of binocular visible light cameras.

6. The method for measuring the volume of manufactured sand and gravel stockpiles as described in claim 3, characterized in that, Before the image correction of the stockpile described in step 4, noise reduction is required. The method is as follows: Step 401: Calculate the pixel coordinate values ​​of the two material pile markings using the image recognition module; Step 402: Based on the pixel coordinate values ​​of the material pile marking, crop the images of the material pile captured by the two sets of binocular visible light cameras so that the images exactly include the material pile marking; Step 403: Use Gaussian filtering to denoise the cropped image.

7. The method for measuring the volume of manufactured sand and gravel stockpiles as described in claim 3, characterized in that, The filtering method in step 5 is as follows: Step 501: Perform left-right consistency detection on the original disparity map and remove outliers; Step 502: Fill the voids in the result of step 501; Step 503: Perform weighted least squares filtering on the result after filling in step 502; Step 504: Save the filtered disparity map.

8. The method for measuring the volume of manufactured sand and gravel stockpiles as described in any one of claims 3-7, characterized in that, The calculation method in step 7 is as follows: Step 701: Select the first set of three-dimensional mesh surfaces generated by the binocular visible light camera, and convert the pixel coordinates of the two sets of material pile markings obtained by the icon recognition module to the world coordinate system; Step 702: Construct a zero plane from the coordinates of the two sets of material piles in the world coordinate system; Step 703: For the three-dimensional mesh surface generated by the three-dimensional reconstruction module, select each triangular facet in turn, project it onto the zero plane, and calculate the volume of the right triangular prism formed by the projected triangle and the original triangular facet. Step 704: Traverse all triangles and repeat steps 702 to 703 to calculate the volume of all triangular prisms; Step 705: Sum the volumes of all triangular prisms to obtain the volume V1 of the stockpile as calculated by the first set of binocular visible light cameras; Step 706: Perform the operations on the three-dimensional mesh surface generated by the first set of binocular visible light cameras in steps 701 to 705 on the three-dimensional mesh surface generated by the second set of binocular visible light cameras to obtain the volume V2 of the material pile as captured and calculated by the second set of binocular visible light cameras. Step 707: Calculate the total volume of the stockpile, V = V1 + V2.

Images