On-load material online metering method, metering scale and equipment based on intelligent video

Abstract

The invention relates to the technical field of material conveying and metering, in particular to an on-load material online metering method based on an intelligent video, a metering scale and equipment. The method comprises the steps that the positions of a laser generator and a camera are kept unchanged, and a first image and a second image of a calibration plate are collected before and after the laser generator is started to form a set of calibration images; moving the position of the calibration plate for multiple times, and collecting N groups of calibration images; respectively calculating 3D point coordinates of the laser rays corresponding to the N groups of calibration images in a camera coordinate system, and further fitting a light plane equation in the camera coordinate system; respectively shooting a third image and a fourth image when the conveying belt is in a no-load state and carries materials, and respectively solving 3D point coordinates of the lower laser ray and the upper laser ray in the camera coordinate system by combining the light plane equation; and the area defined by the lower laser lines and the upper laser lines is calculated, and the materials are metered in combination with the running speed of the conveying belt. According to the invention, high metering precision can be maintained under complex working conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of material conveying and metering, and particularly to an on-line metering method, a weighing scale and a device for loaded materials based on intelligent video. Background Art

[0002] The on-line metering technology of materials is of great significance in industrial production. Especially in industries such as minerals, coal, and logistics, accurate material metering can improve production efficiency, optimize resource allocation, and provide a reliable basis for the cost accounting of enterprises.

[0003] In the industry, a variety of technical means are adopted to perform on-line metering of materials on the conveyor belt. For example, a mechanical sensor is used in cooperation with the conveyor belt to achieve dynamic weighing, and a pressure sensor installed under the conveyor belt senses the weight of the material. However, this measurement method is easily affected by factors such as belt tension and material impact, resulting in low metering accuracy; there is also the use of ultrasonic or laser ranging devices, which indirectly calculate the material volume by measuring the change in material height, and then estimate the weight. This method is difficult to ensure metering accuracy when the shape of the material accumulation is irregular.

[0004] Therefore, there is an urgent need to propose a high-precision metering method that can adapt to complex working conditions. Summary of the Invention

[0005] In order to solve the above problems in the prior art, the present invention proposes an on-line metering method, a weighing scale and a device for loaded materials based on intelligent video, which can maintain high metering accuracy under complex working conditions.

[0006] In the first aspect of the present invention, an on-line metering method for loaded materials based on intelligent video is proposed. The metering method includes: Obtaining the internal parameter matrix and distortion matrix of the camera by using the Zhang Zhengyou calibration method; Keeping the positions of the laser generator and the camera fixed, collecting the first image of the calibration board when the laser generator is turned off, and collecting the second image of the laser irradiated on the calibration board after the laser generator is turned on, so as to form a set of calibration images; Moving the position of the calibration board multiple times, and repeating the operation of collecting the first image and the second image, and a total of N sets of the calibration images are obtained; For each set of the calibration images, fitting out the 3D point coordinates of the laser line in the second image in the camera coordinate system; According to the 3D point coordinates of the laser lines corresponding to the N sets of the calibration images in the camera coordinate system, fitting out the optical plane equation in the camera coordinate system; The third image and the fourth image are respectively taken when the conveyor belt is empty and when the conveyor belt is carrying materials, and the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system are respectively calculated by combining the light plane equation; The material transported within a specified time period is measured by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed.

[0007] Preferably, the step of “for each group of the calibration images, fitting the 3D point coordinates of the laser line in the second image in the camera coordinate system” comprises: For a certain group of calibration images, according to the first image, the intrinsic parameter matrix and the distortion matrix, a spatial equation of the calibration plate plane in the camera coordinate system when the image is taken is obtained; The laser line is extracted according to the second image in the same group of the calibration images, and the 3D point coordinates of the laser line in the camera coordinate system are calculated by combining the space equation, the intrinsic parameter matrix and the distortion matrix.

[0008] Preferably, the step of “for a group of the calibration images, according to the first image, the intrinsic parameter matrix and the distortion matrix, obtaining the spatial equation of the calibration plate plane in the camera coordinate system when the image is captured” comprises: Extracting pixel coordinates of corner points of the calibration plate from the first image, and performing distortion correction according to the distortion matrix, thereby calculating 2D point coordinates of the corner points of the calibration plate in a pixel coordinate system; Defining the 3D point coordinates of the corner points of the calibration plate in the world coordinate system; According to the 2D point coordinates of the corner points of the calibration plate in the pixel coordinate system and the 3D point coordinates in the world coordinate system, combined with the intrinsic parameter matrix and the distortion matrix, the extrinsic parameters of the camera are solved by the PnP algorithm; Using the external parameters, converting the 3D point coordinates of the corner points of the calibration plate in the world coordinate system into the 3D point coordinates in the camera coordinate system; According to the 3D point coordinates of the corner points of the calibration plate in the camera coordinate system, the spatial equation of the calibration plate plane in the camera coordinate system is fitted.

[0009] Preferably, the step of “extracting a laser line according to the second image in the same group of calibration images, and calculating the 3D point coordinates of the laser line in the camera coordinate system by combining the space equation, the intrinsic parameter matrix and the distortion matrix” comprises: Preprocessing the second image and extracting the laser line using the HilditchThin algorithm; The coordinates of each 2D point of the laser line in the pixel coordinate system are back-projected using the intrinsic parameter matrix to obtain a first ray equation in the camera coordinate system; The intersection of the first ray equation and the space equation of the calibration plate plane is obtained to obtain the 3D point coordinates of the laser line in the camera coordinate system.

[0010] Preferably, the step of "respectively capturing the third image and the fourth image when the conveyor belt is empty and when carrying materials, and respectively calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system in combination with the light plane equation" includes: Taking a third image of the laser irradiating the conveyor belt when the conveyor belt is empty; Preprocessing the third image and extracting the lower laser line using the HilditchThin algorithm; For each 2D point coordinate of the lower laser line in the pixel coordinate system, back-projection is performed using the intrinsic parameter matrix to obtain a second ray equation in the camera coordinate system; Find the intersection of the second ray equation and the light plane equation to obtain the 3D point coordinates of the lower laser line in the camera coordinate system; capturing a fourth image of the laser irradiating the material when the conveyor belt carries the material; Preprocessing the fourth image and extracting the upper laser line using the HilditchThin algorithm; For each 2D point coordinate of the upper laser line in the pixel coordinate system, back-projection is performed using the intrinsic parameter matrix to obtain a third ray equation in the camera coordinate system; The intersection of the third ray equation and the light plane equation is calculated to obtain the 3D point coordinates of the upper laser line in the camera coordinate system.

[0011] Preferably, the step of "calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and measuring the material transported within a specified time period in combination with the conveyor belt running speed" includes: The cross-sectional area of ​​the material on the conveyor belt is obtained by calculating the area enclosed by the lower laser line and the upper laser line; Obtain the conveyor belt running speed through the speed sensor; The weight of the material transported within a specified time period is calculated based on the running speed and the cross-sectional area.

[0012] Preferably, the laser generator is pre-installed just above the conveyor belt so that the laser can irradiate the conveyor belt vertically; The camera is pre-installed at a set tilt angle so that the camera can capture an image of the contact area between the laser and the conveyor belt.

[0013] In a second aspect of the present invention, an on-line weighing scale for loaded materials based on intelligent video is proposed, and the weighing scale measures the materials carried on the conveyor belt according to the method described above.

[0014] In a third aspect of the present invention, a computer-readable storage device is proposed, which stores a computer program that can be loaded and executed by a processor according to the method described above.

[0015] The present invention has the following beneficial effects: The present invention fits the light plane equation through multiple groups of calibration images, and then calculates the cross-sectional area of the material enclosed by the upper laser line and the lower laser line when the material passes through the light plane, so as to measure the material within a specified time period. Through the above intelligent vision + laser measurement technology, it is applicable to bulk materials of different shapes and particle sizes (such as ores, coals, grains, etc.), and is not affected by the physical properties of the materials, belt tension, material impact, etc., realizing high weighing accuracy under complex working conditions. Description of the Drawings

[0016] Figure 1 is a schematic diagram of the main steps of an embodiment of the on-line weighing method for loaded materials based on intelligent video in the present invention; Figure 2 is a schematic diagram of the installation positions of the laser generator and the camera in an embodiment of the present invention; Figure 3 In (a) and (b) in are a group of calibration images collected before and after the laser generator is turned on, respectively. Detailed Embodiments

[0017] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present invention and are not intended to limit the protection scope of the present invention.

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] It should be noted that in the description of the present invention, the terms "first" and "second" are only for convenience of description, rather than indicating or implying the relative importance of the devices, elements or parameters, and thus should not be construed as a limitation to the present invention. In addition, the term "and / or" in the present invention is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0020] Figure 1 is a schematic diagram of the main steps of an embodiment of the on-line metering method for loaded materials based on intelligent video in the present invention. As Figure 1 shown, the metering method includes steps S10 - S70: Step S10: Obtain the internal parameter matrix and distortion matrix of the camera using the Zhang Zhengyou calibration method.

[0021] Step S20: Keep the positions of the laser generator and the camera fixed. When the laser generator is turned off, collect the first image of the calibration board, and when the laser generator is turned on, collect the second image of the laser irradiated on the calibration board, thereby forming a set of calibration images.

[0022] Figure 2 is a schematic diagram of the installation positions of the laser generator and the camera in an embodiment of the present invention. As Figure 2 shown, the laser generator is pre-installed directly above the conveyor belt so that the laser can vertically irradiate the conveyor belt; the camera is pre-installed at a set tilt angle so that the camera can capture the image of the area where the laser contacts the conveyor belt.

[0023] Figure 3 (a) and (b) in are a set of calibration images collected before and after the laser generator is turned on, respectively.

[0024] Step S30: Move the position of the calibration board multiple times and repeat the operation of collecting the first image and the second image to obtain a total of N sets of calibration images. Among them, N is a preset value.

[0025] Step S40: For each set of calibration images, fit the 3D point coordinates of the laser line in the second image in the camera coordinate system.

[0026] Specifically, this step may include steps S41 - S42: Step S41: For a certain set of calibration images, according to the first image, the internal parameter matrix and the distortion matrix of the camera, find the spatial equation of the calibration board plane in the camera coordinate system when this image is taken. This step may specifically include steps S411 - S415: S411. Extract the pixel coordinates of the calibration board corner points from the first image, perform distortion correction according to the distortion matrix, and then calculate the 2D point coordinates of the calibration board corner points in the pixel coordinate system.

[0027] S412. Define the 3D point coordinates of the calibration board corner points in the world coordinate system.

[0028] S413. According to the 2D point coordinates of the calibration board corner points in the pixel coordinate system and the 3D point coordinates in the world coordinate system, combined with the internal parameter matrix and the distortion matrix, solve the external parameters of the camera through the PnP algorithm.

[0029] S414. Use the external parameters to convert the 3D point coordinates of the calibration board corner points in the world coordinate system into the 3D point coordinates in the camera coordinate system.

[0030] S415. According to the 3D point coordinates of the calibration board corner points in the camera coordinate system, fit the spatial equation of the calibration board plane in the camera coordinate system.

[0031] In this embodiment, a checkerboard calibration board is used, and multiple corner points can be extracted from the first image for fitting the spatial equation after coordinate transformation.

[0032] Step S42. Extract the laser line from the second image in the same group of calibration images, and combine the spatial equation of the calibration board plane, the internal parameter matrix and the distortion matrix to find the 3D point coordinates of the laser line in the camera coordinate system. This step can specifically include steps S421 - S423: S421. Preprocess the second image and use the HilditchThin algorithm to extract the laser line.

[0033] The HilditchThin algorithm is a classic binary image thinning algorithm, which is used to gradually strip the edge pixels of the connected regions (such as text, lines, etc.) in the binary image, and finally obtain a single-pixel-width skeleton.

[0034] S422. For each 2D point coordinate of the laser line in the pixel coordinate system, use the internal parameter matrix for back-projection to obtain the first ray equation in the camera coordinate system.

[0035] S423. Find the intersection point of the first ray equation and the spatial equation of the calibration board plane to obtain the 3D point coordinates of the laser line in the camera coordinate system.

[0036] Step S50. According to the 3D point coordinates of the laser lines corresponding to N groups of calibration images in the camera coordinate system, fit the light plane equation in the camera coordinate system.

[0037] Step S60, taking a third image and a fourth image when the conveyor belt is empty and carrying materials, respectively, and calculating the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system in combination with the light plane equation.

[0038] Since the cross-sectional area of ​​the material needs to be calculated in the present invention, the boundary line between the conveyor belt and the light plane when empty is called the lower laser line, and the boundary line between the material and the light plane when carrying the material is called the upper laser line. The area enclosed by the upper and lower laser lines is the cross-sectional area of ​​the material. Among them, the shape of the upper laser line is related to the shape of the material after accumulation, and the shape of the lower laser line is related to the surface shape of the conveyor belt.

[0039] Specifically, this step includes steps S61-S68: Step S61, capturing a third image of the conveyor belt irradiated with laser light when the conveyor belt is unloaded.

[0040] Step S62: pre-process the third image and extract the lower laser line using the HilditchThin algorithm. The pre-processing here includes: grayscale, thresholding, erosion and expansion, connected domain analysis and other traditional image processing.

[0041] Step S63: perform back-projection on each 2D point coordinate of the lower laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain a second ray equation in the camera coordinate system.

[0042] Step S64, find the intersection of the second ray equation and the light plane equation to obtain the 3D point coordinates of the lower laser line in the camera coordinate system.

[0043] Step S65, capturing a fourth image of the laser irradiating the material when the conveyor belt is carrying the material.

[0044] Step S66, preprocess the fourth image and use the HilditchThin algorithm to extract the upper laser line.

[0045] Step S67: perform back-projection on each 2D point coordinate of the laser line in the pixel coordinate system using the intrinsic parameter matrix to obtain the third ray equation in the camera coordinate system.

[0046] Step S68, find the intersection of the third ray equation and the light plane equation to obtain the 3D point coordinates of the upper laser line in the camera coordinate system.

[0047] Step S70: Measure the materials transported within a specified time period by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the conveyor belt running speed.

[0048] Specifically, this step may include steps S71-S73: Step S71: Obtain the cross-sectional area of the material on the conveyor belt by calculating the area enclosed by the lower laser line and the upper laser line, as shown in formula (1): (1) where S is the cross-sectional area of the material, and are the curves of the upper laser line and the lower laser line respectively.

[0049] Step S72: Obtain the running speed of the conveyor belt through a speed sensor.

[0050] Step S73: Calculate the weight of the material transported within a specified time period according to the running speed and the cross-sectional area, as shown in formula (2): (2) where is the weight of the material transported, is the material density, S is the cross-sectional area of the material, is the running speed of the conveyor belt, is the length of the specified time period.

[0051] In the above embodiments, although the steps are described in the above sequential order, those skilled in the art can understand that in order to achieve the effects of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are all within the protection scope of the present invention.

[0052] Furthermore, based on the above method embodiments, the present invention also provides an embodiment of an on-line weighing scale for loaded materials based on intelligent video. The weighing scale in this embodiment weighs the materials carried on the conveyor belt according to the method described above.

[0053] Even further, the present invention also provides an embodiment of a computer-readable storage device. The computer program capable of being loaded and executed by a processor as described above is stored in the storage device of this embodiment.

[0054] The computer-readable storage device may include: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0055] Those skilled in the art should be able to realize that the method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0056] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An on-line metering method for loaded materials based on intelligent video, characterized in that, The measurement method includes: Obtaining the internal parameter matrix and distortion matrix of the camera using the Zhang-Zhengyou calibration method; Keeping the positions of the laser generator and the camera fixed, collecting the first image of the calibration board when the laser generator is off, and collecting the second image of the laser irradiating on the calibration board after the laser generator is turned on, thereby forming a set of calibration images; Moving the position of the calibration board multiple times and repeating the operation of collecting the first image and the second image, and a total of N sets of the calibration images are obtained; For each set of the calibration images, fitting the 3D point coordinates of the laser line in the second image taken in the camera coordinate system; According to the 3D point coordinates of the laser lines corresponding to the N sets of the calibration images in the camera coordinate system, fitting the light plane equation in the camera coordinate system; where N is a preset value; Taking the third image and the fourth image respectively when the conveyor belt is empty and carrying materials, and combining the light plane equation to find the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system respectively; By calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining the running speed of the conveyor belt, measuring the materials transported within a specified time period.

2. The on-line metering method of loaded materials based on intelligent video according to claim 1, characterized in that The step of "for each set of the calibration images, fitting the 3D point coordinates of the laser line in the second image taken in the camera coordinate system" includes: For a certain set of the calibration images, according to the first image, the internal parameter matrix and the distortion matrix, finding the spatial equation of the calibration board plane in the camera coordinate system when taking this image; Extracting the laser line from the second image in the same set of the calibration images, and combining the spatial equation, the internal parameter matrix and the distortion matrix to find the 3D point coordinates of this laser line in the camera coordinate system.

3. The online metering method of the loaded material based on intelligent video according to claim 2, characterized in that, The step of "for a certain set of the calibration images, according to the first image, the internal parameter matrix and the distortion matrix, finding the spatial equation of the calibration board plane in the camera coordinate system" includes: Extracting the pixel coordinates of the calibration board corner points from the first image, performing distortion correction according to the distortion matrix, and further calculating the 2D point coordinates of the calibration board corner points in the pixel coordinate system; Defining the 3D point coordinates of the calibration board corner points in the world coordinate system; According to the 2D point coordinates of the calibration board corner points in the pixel coordinate system and the 3D point coordinates in the world coordinate system, combining the internal parameter matrix and the distortion matrix, and solving the external parameters of the camera through the PnP algorithm; Using the external parameters to convert the 3D point coordinates of the calibration board corner points in the world coordinate system into the 3D point coordinates in the camera coordinate system; According to the 3D point coordinates of the calibration board corner points in the camera coordinate system, fitting the spatial equation of the calibration board plane in the camera coordinate system.

4. The online metering method of the loaded material based on intelligent video according to claim 2, wherein The step of "extracting the laser line from the second image in the same set of the calibration images, and combining the spatial equation, the internal parameter matrix and the distortion matrix to find the 3D point coordinates of this laser line in the camera coordinate system" includes: Preprocessing the second image and using the HilditchThin algorithm to extract the laser line; For each 2D point coordinate of the laser line in the pixel coordinate system, back-projection is performed using the internal parameter matrix to obtain the first ray equation in the camera coordinate system; Find the intersection point of the first ray equation and the spatial equation of the calibration plate plane to obtain the 3D point coordinate of the laser line in the camera coordinate system.

5. The online metering method for loaded materials based on intelligent video according to claim 1, wherein, The steps of "taking a third image and a fourth image respectively when the conveyor belt is empty and carrying materials, and respectively obtaining the 3D point coordinates of the lower laser line and the upper laser line in the camera coordinate system in combination with the light plane equation" include: Take a third image of the laser irradiating the conveyor belt when the conveyor belt is empty; Preprocess the third image and use the HilditchThin algorithm to extract the lower laser line; For each 2D point coordinate of the lower laser line in the pixel coordinate system, back-projection is performed using the internal parameter matrix to obtain the second ray equation in the camera coordinate system; Find the intersection point of the second ray equation and the light plane equation to obtain the 3D point coordinate of the lower laser line in the camera coordinate system; Take a fourth image of the laser irradiating the material when the conveyor belt is carrying materials; Preprocess the fourth image and use the HilditchThin algorithm to extract the upper laser line; For each 2D point coordinate of the upper laser line in the pixel coordinate system, back-projection is performed using the internal parameter matrix to obtain the third ray equation in the camera coordinate system; Find the intersection point of the third ray equation and the light plane equation to obtain the 3D point coordinate of the upper laser line in the camera coordinate system.

6. The online metering method of the loaded material based on intelligent video according to claim 1, wherein The steps of "measuring the materials transported within a specified time period by calculating the area enclosed by the lower laser line and the upper laser line in the camera coordinate system and combining with the running speed of the conveyor belt" include: Obtain the cross-sectional area of the material on the conveyor belt by calculating the area enclosed by the lower laser line and the upper laser line; Obtain the running speed of the conveyor belt through a speed sensor; Calculate the weight of the materials transported within the specified time period according to the running speed and the cross-sectional area.

7. The online metering method for on-load materials based on intelligent video according to claim 1, characterized in that The laser generator is pre-installed directly above the conveyor belt so that the laser can vertically irradiate the conveyor belt; The camera is pre-installed at a set inclination angle so that the camera can capture an image of the area where the laser contacts the conveyor belt.

8. An on-line weighing scale for loaded materials based on intelligent video, characterized in that, The weighing scale measures the materials carried on the conveyor belt according to the method described in any one of claims 1-7.

9. A computer-readable storage device, characterized in that, A computer program is stored that can be loaded and executed by a processor according to the method described in any one of claims 1-7.

Images