Online measuring method, measuring scale and equipment for loaded materials based on intelligent video

Abstract

The present invention relates to the technical field of material conveying and metering, and in particular to an online metering method, metering scale and equipment for conveyed materials based on intelligent video. The method of the present invention comprises: keeping the positions of the laser generator and the camera fixed, respectively collecting the first image and the second image of the calibration plate before and after the laser generator is turned on to form a set of calibration images; moving the position of the calibration plate multiple times, and collecting a total of N sets of calibration images; respectively calculating the 3D point coordinates of the laser lines corresponding to the N sets of calibration images in the camera coordinate system, and then fitting the light plane equation in the camera coordinate system; respectively taking the third image and the fourth image when the conveyor belt is empty and carrying materials, and respectively calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system in combination with the light plane equation; calculating the area enclosed by the lower laser line and the upper laser line, and metering the material in combination with the conveyor belt running speed. The present invention can maintain high metering accuracy under complex working conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of material conveying and metering, and in particular to an online metering method, a metering scale and equipment for loaded materials based on intelligent video. Background Art

[0002] Online material measurement technology is of great significance in industrial production, especially in the mining, coal, logistics and other industries. Accurate material measurement can improve production efficiency, optimize resource allocation, and provide a reliable basis for enterprise cost accounting.

[0003] The industry employs a variety of technologies to measure materials on conveyor belts online. For example, mechanical sensors are used in conjunction with conveyor belts to achieve dynamic weighing. Pressure sensors installed beneath the conveyor belt sense the material weight. However, this measurement method is susceptible to factors such as belt tension and material impact, resulting in low measurement accuracy. Other methods use ultrasonic or laser distance measuring devices to indirectly calculate material volume and, therefore, weight, by measuring changes in material height. However, this method struggles to maintain accurate measurement when the material is piled in irregular shapes.

[0004] Therefore, it is urgent to propose a high-precision measurement method that can adapt to complex working conditions. Summary of the Invention

[0005] In order to solve the above problems in the prior art, the present invention proposes an online metering method, metering scale and equipment for loaded materials based on intelligent video, which can maintain high metering accuracy under complex working conditions.

[0006] In a first aspect of the present invention, an online metering method for loaded materials based on intelligent video is provided, the metering method comprising:

[0007] The Zhang Zhengyou calibration method is used to obtain the camera's intrinsic parameter matrix and distortion matrix;

[0008] Keeping the positions of the laser generator and the camera fixed, capturing a first image of the calibration plate when the laser generator is turned off, and capturing a second image of the calibration plate when the laser is turned on, thereby forming a set of calibration images;

[0009] Moving the calibration plate multiple times and repeatedly acquiring the first image and the second image to obtain N sets of calibration images;

[0010] For each set of the calibration images, fitting the 3D point coordinates of the laser line in the second image in the camera coordinate system;

[0011] Fitting the light plane equation in the camera coordinate system according to the 3D point coordinates of the laser line corresponding to the N groups of calibration images in the camera coordinate system;

[0012] Taking a third image and a fourth image when the conveyor belt is empty and loaded with materials, respectively, and calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system respectively by combining the light plane equation;

[0013] The material transported within a specified time period is measured by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed.

[0014] Preferably, the step of “fitting the 3D point coordinates of the laser line in the second image captured in the camera coordinate system for each set of the calibration images” includes:

[0015] For a set of calibration images, determining a spatial equation of the calibration plate plane in the camera coordinate system when the image is captured based on the first image, the intrinsic parameter matrix, and the distortion matrix;

[0016] The laser line is extracted according to the second image in the same set of calibration images, and the 3D point coordinates of the laser line in the camera coordinate system are calculated by combining the spatial equation, the intrinsic parameter matrix and the distortion matrix.

[0017] Preferably, the step of “for a set of calibration images, determining, based on the first image, the intrinsic parameter matrix, and the distortion matrix, a spatial equation of the calibration plate plane in the camera coordinate system when the image is captured” includes:

[0018] Extracting pixel coordinates of corner points of the calibration plate from the first image, performing distortion correction according to the distortion matrix, and then calculating 2D point coordinates of the corner points of the calibration plate in a pixel coordinate system;

[0019] Defining the 3D coordinates of the corner points of the calibration plate in the world coordinate system;

[0020] According to the 2D point coordinates of the corner points of the calibration plate in the pixel coordinate system and the 3D point coordinates in the world coordinate system, combined with the intrinsic parameter matrix and the distortion matrix, the extrinsic parameters of the camera are solved by the PnP algorithm;

[0021] Using the external parameters, converting the 3D point coordinates of the corner points of the calibration plate in the world coordinate system into the 3D point coordinates in the camera coordinate system;

[0022] According to the 3D point coordinates of the corner points of the calibration plate in the camera coordinate system, the spatial equation of the calibration plate plane in the camera coordinate system is fitted.

[0023] Preferably, the step of “extracting a laser line according to the second image in the same set of calibration images, and calculating the 3D point coordinates of the laser line in the camera coordinate system by combining the spatial equation, the intrinsic parameter matrix and the distortion matrix” includes:

[0024] Preprocessing the second image and extracting the laser line using the HilditchThin algorithm;

[0025] Back-projecting the coordinates of each 2D point of the laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain the first ray equation in the camera coordinate system;

[0026] The intersection of the first ray equation and the space equation of the calibration plate plane is obtained to obtain the 3D point coordinates of the laser line in the camera coordinate system.

[0027] Preferably, the step of "respectively capturing the third image and the fourth image when the conveyor belt is empty and when carrying materials, and respectively calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system in combination with the light plane equation" includes:

[0028] Taking a third image of the laser irradiating the conveyor belt when the conveyor belt is empty;

[0029] Preprocessing the third image and extracting the lower laser line using the HilditchThin algorithm;

[0030] For each 2D point coordinate of the lower laser line in the pixel coordinate system, back-project the coordinates using the intrinsic parameter matrix to obtain a second ray equation in the camera coordinate system;

[0031] Finding the intersection of the second ray equation and the light plane equation to obtain the 3D point coordinates of the lower laser line in the camera coordinate system;

[0032] capturing a fourth image of the laser irradiating the material while the conveyor belt is carrying the material;

[0033] Preprocessing the fourth image and extracting the upper laser line using the HilditchThin algorithm;

[0034] For each 2D point coordinate of the upper laser line in the pixel coordinate system, back-project the coordinates using the intrinsic parameter matrix to obtain a third ray equation in the camera coordinate system;

[0035] The intersection of the third ray equation and the light plane equation is calculated to obtain the 3D point coordinates of the upper laser line in the camera coordinate system.

[0036] Preferably, the step of “measuring the material transported within a specified time period by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed” includes:

[0037] The cross-sectional area of ​​the material on the conveyor belt is obtained by calculating the area enclosed by the lower laser line and the upper laser line;

[0038] Obtain the conveyor belt running speed through the speed sensor;

[0039] The weight of the material transported within a specified time period is calculated based on the running speed and the cross-sectional area.

[0040] Preferably, the laser generator is pre-installed just above the conveyor belt so that the laser can irradiate the conveyor belt vertically;

[0041] The camera is pre-installed at a set tilt angle so that the camera can capture an image of the contact area between the laser and the conveyor belt.

[0042] In a second aspect of the present invention, an online weighing scale for conveying materials based on intelligent video is proposed, which measures the materials carried on the conveyor belt according to the method described above.

[0043] According to a third aspect of the present invention, a computer-readable storage device is provided, storing a computer program that can be loaded by a processor and execute the method described above.

[0044] The present invention has the following beneficial effects:

[0045] This invention uses multiple sets of calibration images to fit the light plane equation. It then calculates the cross-sectional area of ​​the material enclosed by the upper and lower laser lines as it passes through the light plane, thereby measuring the material within a specified time period. This intelligent vision combined with laser measurement technology is applicable to bulk materials of varying shapes and particle sizes (such as ore, coal, and grain), unaffected by physical properties, belt tension, and impact, achieving high measurement accuracy even under complex operating conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a schematic diagram of the main steps of an embodiment of the method for online metering of loaded materials based on intelligent video in the present invention;

[0047] Figure 2 Schematic diagram of the installation position of the laser generator and the camera in an embodiment of the present invention;

[0048] Figure 3 (a) and (b) are a set of calibration images collected before and after the laser generator is turned on. DETAILED DESCRIPTION

[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] It should be noted that, in the description of the present invention, the terms "first" and "second" are merely for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore should not be understood as limiting the present invention. In addition, the term "and / or" in the present invention is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document, unless otherwise specified, generally indicates that the associated objects are in an "or" relationship.

[0052] Figure 1 This is a schematic diagram of the main steps of an embodiment of the method for online metering of loaded materials based on intelligent video in the present invention. Figure 1 As shown, the measurement method includes steps S10-S70:

[0053] Step S10: Use Zhang Zhengyou calibration method to obtain the camera's intrinsic parameter matrix and distortion matrix.

[0054] Step S20: Keep the positions of the laser generator and the camera fixed, capture a first image of the calibration plate when the laser generator is turned off, and capture a second image of the laser irradiating the calibration plate after the laser generator is turned on, thereby forming a set of calibration images.

[0055] Figure 2 FIG is a schematic diagram of the installation position of the laser generator and the camera in the embodiment of the present invention. Figure 2 As shown, the laser generator is pre-installed directly above the conveyor belt so that the laser can irradiate the conveyor belt vertically; the camera is pre-installed at a set tilt angle so that the camera can capture images of the contact area between the laser and the conveyor belt.

[0056] Figure 3 (a) and (b) are a set of calibration images collected before and after the laser generator is turned on.

[0057] Step S30: Move the calibration plate multiple times and repeatedly capture the first image and the second image to obtain N sets of calibration images, where N is a preset value.

[0058] Step S40: For each set of calibration images, fit the 3D point coordinates of the laser line in the second image in the camera coordinate system.

[0059] Specifically, this step may include steps S41-S42:

[0060] Step S41: For a set of calibration images, the spatial equation of the calibration plate plane in the camera coordinate system when the image was captured is obtained based on the first image, the camera's intrinsic parameter matrix, and the distortion matrix. This step may specifically include steps S411-S415:

[0061] S411 , extracting pixel coordinates of corner points of the calibration plate from the first image, performing distortion correction according to the distortion matrix, and then calculating the 2D point coordinates of the corner points of the calibration plate in the pixel coordinate system.

[0062] S412: Define the 3D coordinates of the corner points of the calibration plate in the world coordinate system.

[0063] S413. According to the 2D point coordinates of the corner points of the calibration plate in the pixel coordinate system and the 3D point coordinates in the world coordinate system, the camera's extrinsic parameters are solved by the PnP algorithm in combination with the intrinsic parameter matrix and the distortion matrix.

[0064] S414: Using external parameters, convert the 3D point coordinates of the corner points of the calibration plate in the world coordinate system into 3D point coordinates in the camera coordinate system.

[0065] S415 , fitting the spatial equation of the calibration plate plane in the camera coordinate system according to the 3D point coordinates of the corner points of the calibration plate in the camera coordinate system.

[0066] In this embodiment, a checkerboard calibration plate is used to extract multiple corner points from the first image and use them to fit the space equation after coordinate transformation.

[0067] Step S42: Extract the laser line from the second image in the same set of calibration images, and calculate the 3D point coordinates of the laser line in the camera coordinate system by combining the calibration plate plane space equation, the intrinsic parameter matrix, and the distortion matrix. This step may specifically include steps S421-S423:

[0068] S421 , pre-processing the second image and extracting the laser line using the HilditchThin algorithm.

[0069] The HilditchThin algorithm is a classic binary image thinning algorithm that is used to gradually remove edge pixels from connected areas (such as text and lines) in a binary image, ultimately obtaining a skeleton with a single-pixel width.

[0070] S422 , back-projecting each 2D point coordinate of the laser line in the pixel coordinate system using an intrinsic parameter matrix to obtain a first ray equation in the camera coordinate system.

[0071] S423 . Calculate the intersection of the first ray equation and the spatial equation of the calibration plate plane to obtain the 3D point coordinates of the laser line in the camera coordinate system.

[0072] Step S50 : fitting the light plane equation in the camera coordinate system according to the 3D point coordinates of the laser line corresponding to the N sets of calibration images in the camera coordinate system.

[0073] Step S60: Take a third image and a fourth image when the conveyor belt is empty and when it is carrying materials, respectively, and calculate the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system respectively by combining the light plane equation.

[0074] Because the cross-sectional area of ​​the material needs to be determined in this invention, the intersection of the conveyor belt and the light plane when empty is called the lower laser line, and the intersection of the material and the light plane when loaded is called the upper laser line. The area enclosed by the upper and lower laser lines is the cross-sectional area of ​​the material. The shape of the upper laser line is related to the shape of the accumulated material, while the shape of the lower laser line is related to the surface shape of the conveyor belt.

[0075] Specifically, this step includes steps S61-S68:

[0076] Step S61 : capturing a third image of the laser irradiating the conveyor belt when the conveyor belt is unloaded.

[0077] Step S62: pre-process the third image and extract the lower laser line using the HilditchThin algorithm. The pre-processing includes traditional image processing such as grayscale conversion, thresholding, erosion and dilation, and connected domain analysis.

[0078] Step S63: Back-project the coordinates of each 2D point of the lower laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain the second ray equation in the camera coordinate system.

[0079] Step S64: Calculate the intersection of the second ray equation and the light plane equation to obtain the 3D point coordinates of the lower laser line in the camera coordinate system.

[0080] Step S65 : capturing a fourth image of the laser irradiating the material while the conveyor belt is carrying the material.

[0081] Step S66: pre-process the fourth image and use the HilditchThin algorithm to extract the upper laser line.

[0082] Step S67: Perform back-projection on each 2D point coordinate of the laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain the third ray equation in the camera coordinate system.

[0083] Step S68: Calculate the intersection of the third ray equation and the light plane equation to obtain the 3D point coordinates of the upper laser line in the camera coordinate system.

[0084] Step S70: Measure the material transported within a specified time period by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed.

[0085] Specifically, this step may include steps S71-S73:

[0086] Step S71: Calculate the area enclosed by the lower laser line and the upper laser line to obtain the cross-sectional area of ​​the material on the conveyor belt, as shown in formula (1):

[0087] (1)

[0088] Where S is the cross-sectional area of ​​the material, and They are the curves of the upper laser line and the lower laser line respectively.

[0089] Step S72: Obtain the conveyor belt running speed through the speed sensor.

[0090] Step S73: Calculate the weight of the material transported within a specified time period based on the running speed and cross-sectional area, as shown in formula (2):

[0091] (2)

[0092] in, is the weight of the material transported, is the material density, S is the cross-sectional area of ​​the material, is the conveyor belt speed, Specifies the length of the time period.

[0093] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0094] Furthermore, based on the above method embodiment, the present invention also provides an embodiment of an online weighing scale for conveying materials based on intelligent video. The weighing scale of this embodiment measures the materials carried on the conveyor belt according to the method described above.

[0095] Furthermore, the present invention also provides an embodiment of a computer-readable storage device. The storage device of this embodiment stores a computer program that can be loaded by a processor and execute the method described above.

[0096] The computer-readable storage device may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes.

[0097] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0098] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is clearly not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent modifications or substitutions to the relevant technical features, and the technical solutions after such modifications or substitutions will fall within the scope of protection of the present invention.

Claims

1. An online measuring method for loaded materials based on intelligent video, characterized in that: The measurement method includes: The Zhang Zhengyou calibration method is used to obtain the camera's intrinsic parameter matrix and distortion matrix; Keeping the positions of the laser generator and the camera fixed, capturing a first image of the calibration plate when the laser generator is turned off, and capturing a second image of the calibration plate when the laser is turned on, thereby forming a set of calibration images; Moving the calibration plate multiple times and repeatedly acquiring the first image and the second image to obtain N sets of calibration images; For each set of the calibration images, fitting the 3D point coordinates of the laser line in the second image in the camera coordinate system; Fitting the light plane equation in the camera coordinate system according to the 3D point coordinates of the laser line corresponding to the N sets of calibration images in the camera coordinate system; wherein N is a preset value; Taking a third image and a fourth image when the conveyor belt is empty and loaded with materials, respectively, and calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system respectively by using the light plane equation; By calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining it with the conveyor belt running speed, the material transported within a specified time period is measured; in, The steps of "respectively capturing a third image and a fourth image when the conveyor belt is empty and when carrying materials, and respectively calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system by combining the light plane equation" include: Taking a third image of the laser irradiating the conveyor belt when the conveyor belt is empty; Preprocessing the third image and extracting the lower laser line using the HilditchThin algorithm; For each 2D point coordinate of the lower laser line in the pixel coordinate system, back-project the coordinates using the intrinsic parameter matrix to obtain a second ray equation in the camera coordinate system; Finding the intersection of the second ray equation and the light plane equation to obtain the 3D point coordinates of the lower laser line in the camera coordinate system; capturing a fourth image of the laser irradiating the material while the conveyor belt is carrying the material; Preprocessing the fourth image and extracting the upper laser line using the HilditchThin algorithm; For each 2D point coordinate of the upper laser line in the pixel coordinate system, back-project the coordinates using the intrinsic parameter matrix to obtain a third ray equation in the camera coordinate system; The intersection of the third ray equation and the light plane equation is calculated to obtain the 3D point coordinates of the upper laser line in the camera coordinate system.

2. The method for online metering of loaded materials based on intelligent video according to claim 1, characterized in that: The step of “fitting, for each set of the calibration images, the 3D point coordinates of the laser line in the second image captured in the camera coordinate system” includes: For a set of calibration images, determining a spatial equation of the calibration plate plane in the camera coordinate system when the image is captured based on the first image, the intrinsic parameter matrix, and the distortion matrix; The laser line is extracted according to the second image in the same set of calibration images, and the 3D point coordinates of the laser line in the camera coordinate system are calculated by combining the spatial equation, the intrinsic parameter matrix and the distortion matrix.

3. The method for online metering of loaded materials based on intelligent video according to claim 2, characterized in that: The step of “for a set of calibration images, determining, based on the first image, the intrinsic parameter matrix, and the distortion matrix, a spatial equation of the calibration plate plane in the camera coordinate system when the image is captured” includes: Extracting pixel coordinates of corner points of the calibration plate from the first image, performing distortion correction according to the distortion matrix, and then calculating 2D point coordinates of the corner points of the calibration plate in a pixel coordinate system; Defining the 3D coordinates of the corner points of the calibration plate in the world coordinate system; According to the 2D point coordinates of the corner points of the calibration plate in the pixel coordinate system and the 3D point coordinates in the world coordinate system, combined with the intrinsic parameter matrix and the distortion matrix, the extrinsic parameters of the camera are solved by the PnP algorithm; Using the external parameters, convert the 3D point coordinates of the corner points of the calibration plate in the world coordinate system into the 3D point coordinates in the camera coordinate system; According to the 3D point coordinates of the corner points of the calibration plate in the camera coordinate system, the spatial equation of the calibration plate plane in the camera coordinate system is fitted.

4. The method for online metering of loaded materials based on intelligent video according to claim 2, characterized in that: The step of “extracting a laser line according to the second image in the same set of calibration images, and calculating the 3D point coordinates of the laser line in the camera coordinate system by combining the spatial equation, the intrinsic parameter matrix, and the distortion matrix” includes: Preprocessing the second image and extracting the laser line using the HilditchThin algorithm; Back-projecting the coordinates of each 2D point of the laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain the first ray equation in the camera coordinate system; The intersection of the first ray equation and the space equation of the calibration plate plane is obtained to obtain the 3D point coordinates of the laser line in the camera coordinate system.

5. The method for online metering of loaded materials based on intelligent video according to claim 1, characterized in that: The step of "measuring the material transported within a specified time period by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed" includes: The cross-sectional area of ​​the material on the conveyor belt is obtained by calculating the area enclosed by the lower laser line and the upper laser line; Obtain the conveyor belt running speed through the speed sensor; The weight of the material transported within a specified time period is calculated based on the running speed and the cross-sectional area.

6. The method for online weighing of loaded materials based on intelligent video according to claim 1, characterized in that: The laser generator is pre-installed directly above the conveyor belt so that the laser can be irradiated vertically onto the conveyor belt; The camera is pre-installed at a set tilt angle so that the camera can capture an image of the contact area between the laser and the conveyor belt.

7. An online weighing scale for loaded materials based on intelligent video, characterized in that: The weighing scale measures the material carried on the conveyor belt according to the method as described in any one of claims 1 to 6.

8. A computer-readable storage device, characterized in that: The computer program is stored and can be loaded by a processor to execute the method according to any one of claims 1 to 6.

Images