Coal Mining Machine Equipment Positioning Method and Device Based on Laser Rangefinder and Machine Vision

Abstract

The present invention provides a positioning method for coal mining equipment based on a laser rangefinder and machine vision, which includes an industrial camera for real-time tracking of the coal mining equipment. Among them, an inclinometer and a laser rangefinder are fixed on the industrial camera, and the laser rangefinder and the industrial camera point in the same direction; the laser rangefinder measures the distance to a 360-degree prism fixed on the coal mining equipment to obtain the distance between the laser rangefinder and the 360-degree prism; in response to the movement of the coal mining equipment, the inclinometer measures the inclination angles in the real-time horizontal direction and the vertical direction; and the real-time position of the coal mining equipment is calculated. The method of the present invention has a low cost, good environmental adaptability, and high positioning accuracy. The present invention also provides a positioning device for coal mining equipment based on a laser rangefinder and machine vision.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mining equipment, and in particular, to a positioning method and device for coal mining equipment based on a laser rangefinder and machine vision. Background Art

[0002] The heading face in a coal mine underground is one of the most important working faces. The environment of the working face where coal mining equipment, such as a roadheader, is located is complex and there are many workers. A large amount of dust and noise are generated during the cutting process of the roadheader, and the temperature and humidity of the working face are relatively high, which endangers the physical health of the workers. Therefore, it is an inevitable trend to realize the automation of the roadheader and the intelligence of the roadheader working face. Among them, the autonomous navigation of the roadheader is crucial.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: The traditional laser pointing instrument navigation technology is simple to operate, the machine vision technology has high positioning accuracy and low cost, and the inertial navigation technology has strong anti-interference ability and high autonomy. However, the traditional laser pointing instrument is easily blocked, the machine vision is greatly affected by the environment, and the inertial navigation has cumulative errors. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to some extent.

[0005] To this end, an object of the present invention is to provide a positioning method and device for coal mining equipment based on a laser rangefinder and machine vision.

[0006] In a first aspect, a positioning method for coal mining equipment based on a laser rangefinder and machine vision includes:

[0007] An industrial camera performs real-time tracking on the coal mining equipment. An inclinometer and a laser rangefinder are fixed on the industrial camera, and the laser rangefinder and the industrial camera point in the same direction;

[0008] The laser rangefinder measures the distance to a 360-degree prism fixed on the coal mining equipment to obtain the distance between the laser rangefinder and the 360-degree prism;

[0009] In response to the movement of the coal mining equipment, the inclinometer measures the inclination angle in the real-time horizontal direction and the inclination angle in the vertical direction;

[0010] The real-time position of the coal mining equipment is calculated based on the distance between the laser rangefinder and the 360-degree prism, the inclination angle in the real-time horizontal direction measured by the inclinometer, and the inclination angle in the vertical direction.

[0011] According to an embodiment of the present invention, the positioning method for coal mining equipment based on a laser rangefinder and machine vision further includes:

[0012] The server obtains the image collected by the industrial camera, identifies the coal mining equipment in the image by using the frame difference method, and controls the industrial camera to deflect to point to the coal mining equipment.

[0013] According to an embodiment of the present invention, the positioning method of the coal mining equipment based on the laser rangefinder and machine vision further includes:

[0014] Subtract the corresponding pixel values of adjacent frame images to obtain a difference image, and binarize the difference image;

[0015] If the change in the corresponding pixel value is less than the threshold, this is a background pixel, otherwise this is a pixel of the coal mining equipment.

[0016] According to an embodiment of the present invention, the positioning method of the coal mining equipment based on the laser rangefinder and machine vision further includes:

[0017] The regulations of the roadway coordinate system are as follows: the Y-axis direction is the roadway advancing direction, the X direction is horizontally to the right perpendicular to the Y-axis, and the Z-axis is vertically upward perpendicular to the horizontal plane;

[0018] Set the initial position of the laser rangefinder in the roadway coordinate system as P(X0, Y0, Z0), and the position of the prism as T(X t , Y t , Z t );

[0019] The position of the 360-degree prism on the coal mining equipment is calculated according to the following formula:

[0020]

[0021] In the formula, θ is the horizontal direction angle between the laser rangefinder and the 360-degree prism, φ is the vertical direction angle between the laser rangefinder and the 360-degree prism, and S is the oblique distance between the laser rangefinder and the 360-degree prism.

[0022] The real-time position of the 360-degree prism is the real-time position of the coal mining equipment.

[0023] The second object of the present invention is to propose a positioning device for coal mining equipment based on a laser rangefinder and machine vision, including:

[0024] An industrial camera for real-time tracking of the coal mining equipment, wherein a 360-degree prism is rigidly connected to the coal mining equipment;

[0025] A laser rangefinder installed above the industrial camera for measuring the distance between the laser rangefinder and the 360-degree prism, wherein the laser rangefinder and the industrial camera point in the same direction;

[0026] An inclinometer, installed above the laser rangefinder, is used to measure the horizontal angle between the laser rangefinder and the coal mining equipment and the vertical angle between the laser rangefinder and the 360-degree prism.

[0027] A server is used to obtain the images collected by the industrial camera, identify the coal mining equipment in the images using the frame difference method, control the industrial camera to deflect and point to the coal mining equipment, and calculate the real-time position of the coal mining equipment.

[0028] According to an embodiment of the present invention, the positioning device for coal mining equipment based on a laser rangefinder and machine vision further includes a bearing platform. The industrial camera is rotatably arranged above the bearing platform. The laser rangefinder is rigidly connected to the industrial camera, and the inclinometer is installed above the laser rangefinder.

[0029] According to an embodiment of the present invention, the industrial camera and the laser rangefinder are connected by a ball joint.

[0030] According to an embodiment of the present invention, the industrial camera is provided with an explosion-proof housing.

[0031] According to an embodiment of the present invention, the coal mining equipment is at least one of a roadheader, a roadheader-anchoring machine, and a shearer.

[0032] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, in which:

[0034] Figure 1 is a schematic flow chart of a method for positioning coal mining equipment based on a laser rangefinder and machine vision proposed in an embodiment of the present invention.

[0035] Figure 2 is a schematic structural diagram of a positioning device for coal mining equipment based on a laser rangefinder and machine vision proposed in an embodiment of the present invention.

[0036] Figure 3 is a schematic flow chart of realizing real-time tracking of a coal mining machine using the frame difference method proposed in an embodiment of the present invention.

[0037] Figure 4 is a schematic principle diagram of measuring the real-time position of a coal mining machine using a laser rangefinder proposed in an embodiment of the present invention.

[0038] Reference numerals:

[0039] 1 - 360 - degree prism, 2 - roadheader, 3 - inclinometer, 4 - laser rangefinder, 5 - industrial camera, 6 - carrier table. Detailed implementation manners

[0040] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as a limitation of the present invention. On the contrary, the embodiments of the present invention include all changes, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0041] Figure 1 It is a schematic flowchart of a positioning method for coal mining equipment based on a laser rangefinder and machine vision proposed in an embodiment of the present invention.

[0042] In a first aspect, in an embodiment of the present invention, referring to Figure 1 、 Figure 2 , a positioning method for coal mining equipment based on a laser rangefinder and machine vision is provided, including the following steps:

[0043] S102, the industrial camera 5 performs real - time tracking on the coal mining equipment. Among them, an inclinometer 3 and a laser rangefinder 4 are fixed on the industrial camera 5, and the laser rangefinder 4 and the industrial camera 5 point in the same direction.

[0044] S104, the laser rangefinder 4 measures the distance to the 360 - degree prism 1 fixed on the coal mining equipment to obtain the distance between the laser rangefinder 4 and the 360 - degree prism 1.

[0045] S106, in response to the movement of the coal mining equipment, the inclinometer 3 measures the inclination angle in the real - time horizontal direction and the inclination angle in the vertical direction.

[0046] In this step, since the industrial camera performs tracking on the coal mining equipment, the laser rangefinder and the industrial camera rotate in the horizontal and vertical directions together with the coal mining equipment.

[0047] S108, calculate the real - time position of the coal mining equipment according to the distance between the laser rangefinder 4 and the 360 - degree prism 1, the inclination angle in the real - time horizontal direction measured by the inclinometer 3, and the inclination angle in the vertical direction.

[0048] In this step, the coal mining equipment can be the roadheader 2. Initially, the direction of the laser beam can be the axis direction of the roadway. At this time, both the horizontal angle and the vertical angle of the inclinometer are set to zero. When the coal mining equipment moves, the position of the 360-degree prism changes, and the industrial camera performs target tracking. The laser rangefinder and the industrial camera rotate horizontally and vertically together. At this time, the horizontal angle and the vertical angle of the laser rangefinder measured by the inclinometer are the horizontal angle relative to the axis direction of the roadway and the vertical angle relative to the horizontal plane.

[0049] Through the above steps, by combining the industrial camera and the laser indicator, the real-time position of the coal mining equipment can be obtained, avoiding the problems in the prior art that machine vision positioning is easily affected by the environment and the laser indicator needs manual adjustment, and realizing the automation and intelligence of the positioning of the coal mining equipment.

[0050] Identifying the coal mining equipment in the images collected by the industrial camera can be determined according to actual needs. There are many ways of image processing and recognition. In this embodiment, the following optional implementation methods are provided.

[0051] Step S102, the industrial camera 5 performing real-time tracking on the coal mining equipment includes:

[0052] Step S1021, the server side obtains the images collected by the industrial camera 5, uses the frame difference method to identify the coal mining equipment in the images, and controls the industrial camera 5 to deflect to point to the coal mining equipment.

[0053] The device for controlling the deflection of the industrial camera 5 can be implemented by using a driving mechanism in the prior art.

[0054] Specifically, referring to Figure 3 , subtract the corresponding pixel values of adjacent frame images to obtain a difference image, and binarize the difference image.

[0055] For example, let the current frame image be f k , the previous frame image be f k-1 , and the gray values of the two consecutive frame images be set as f k (x, y) and f k-1 (x, y). Calculate the absolute value A k (x, y) of the difference between the gray values of the two frame images:

[0056] A k (x, y) = f k (x, y) - f k-1 (x, y)

[0057] If the change in the corresponding pixel value is less than the threshold, this is a background pixel; otherwise, this is a pixel of the coal mining equipment.

[0058] The obtained difference value A k(x, y) is compared with a pre-set threshold T. When A k (x, y) is greater than the threshold T, it is a coal mining machine equipment. When A k (x, y) is less than the threshold T, it is the background, as shown in the following formula.

[0059]

[0060] In step S106, for standardized measurement, referring to Figure 4 , the regulations of the roadway coordinate system are as follows: the Y-axis direction is the roadway advancing direction, the X-direction is horizontally to the right perpendicular to the Y-axis, and the Z-axis is vertically upward from the horizontal plane;

[0061] Set the initial position of the laser rangefinder in the roadway coordinate system as P(X0, Y0, Z0), and the position of the prism as T(X t , Y t , Z t );

[0062] The position of the 360-degree prism on the coal mining machine equipment is calculated according to the following formula:

[0063]

[0064] In the formula, θ is the horizontal direction angle between the laser rangefinder and the 360-degree prism, φ is the vertical direction angle between the laser rangefinder and the 360-degree prism, and S is the oblique distance between the laser rangefinder and the 360-degree prism.

[0065] The real-time position of the 360-degree prism is the real-time position of the coal mining machine equipment, and the position of the 360-degree prism should enable the laser rangefinder to shine on it.

[0066] It can be seen from the above embodiments that the coal mining machine equipment positioning method based on a laser rangefinder and machine vision provided by the embodiments of the present invention uses machine vision for real-time tracking of the coal mining machine equipment and a laser rangefinder for position detection of the coal mining machine equipment. The laser rangefinder has relatively good environmental adaptability and high accuracy, and can perform real-time detection of the position of the coal mining machine equipment. The combination of the two has a lower cost, good environmental adaptability, and high positioning accuracy, which helps to realize the automation and intelligence of the coal mining machine equipment.

[0067] For the above purposes, the second aspect of the embodiments of the present invention provides a coal mining equipment positioning device based on a laser rangefinder and machine vision. In one embodiment of the present invention, the coal mining equipment positioning device based on a laser rangefinder and machine vision includes a server, an inclinometer 3, a laser rangefinder 4, and an industrial camera 5. The industrial camera 5 performs real-time tracking on the coal mining equipment, wherein a 360-degree prism is rigidly connected to the coal mining equipment; the laser rangefinder 4 is installed above the industrial camera 5 and is used to measure the distance between the laser rangefinder 4 and the 360-degree prism. The laser rangefinder 4 and the industrial camera 5 point in the same direction; the inclinometer 3 is installed above the laser rangefinder 4 and is used to measure the horizontal angle between the laser rangefinder 4 and the coal mining equipment and the vertical angle between the laser rangefinder 4 and the 360-degree prism 1; the server is used to obtain the image collected by the industrial camera 5, identify the coal mining equipment in the image by using the frame difference method, control the industrial camera 5 to deflect to point to the coal mining equipment, and calculate the real-time position of the coal mining equipment.

[0068] It should be noted that the coal mining equipment is at least one of a roadheader 2, a continuous miner, and a shearer.

[0069] In some embodiments, the coal mining equipment positioning device based on a laser rangefinder and machine vision further includes a carrier 6. The industrial camera 5 is rotatably arranged above the carrier 6. The laser rangefinder 4 is rigidly connected to the industrial camera 5, and the inclinometer 3 is installed above the laser rangefinder 4.

[0070] As a possible implementation, the industrial camera 5 and the laser rangefinder 4 are connected by a ball joint. The overall structure of the laser rangefinder 4 and the industrial camera 5 can rotate 360 degrees in the horizontal plane and can rotate up and down by -90° to 90° in the vertical direction. When the coal mining equipment undergoes forward, roll, and pitch changes, the 360-degree prism changes in the same way as the coal mining equipment. To meet the requirements of safe underground operation, the industrial camera 5 is provided with an explosion-proof housing.

[0071] The coal mining equipment positioning method based on a laser rangefinder and machine vision can achieve the same or similar effects as any of the foregoing method embodiments corresponding thereto.

[0072] It should be noted that in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0073] Any process or method description depicted in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code that includes one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present invention includes additional implementations, where functions may be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0074] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0075] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A positioning method for coal mining equipment based on a laser rangefinder and machine vision, characterized in that, Including: An industrial camera (5) performs real-time tracking on a coal mining machine equipment. Among them, an inclinometer (3) and a laser rangefinder (4) are fixed on the industrial camera (5), and the laser rangefinder (4) and the industrial camera (5) point in the same direction; The laser rangefinder (4) measures the distance to a 360-degree prism (1) fixed on the coal mining machine equipment, and obtains the distance between the laser rangefinder (4) and the 360-degree prism (1); In response to the movement of the coal mining machine equipment, the inclinometer (3) measures the inclination angle in the real-time horizontal direction and the inclination angle in the vertical direction; The real-time position of the coal mining machine equipment is calculated based on the distance between the laser rangefinder (4) and the 360-degree prism (1), the inclination angle in the real-time horizontal direction measured by the inclinometer (3), and the inclination angle in the vertical direction; The server side acquires the image collected by the industrial camera (5), uses the frame difference method to identify the coal mining machine equipment in the image, and controls the industrial camera (5) to deflect to point to the coal mining machine equipment; Subtract the corresponding pixel values of adjacent frame images to obtain a difference image, and binarize the difference image; If the change in the corresponding pixel value is less than the threshold, this is a background pixel, otherwise this is a pixel of the coal mining machine equipment.

2. The coal mining equipment positioning method based on a laser rangefinder and machine vision according to claim 1, wherein, Including: The regulations of the roadway coordinate system are as follows: The Y-axis direction is the roadway advancing direction, the X direction is horizontally to the right perpendicular to the Y-axis, and the Z-axis is vertically upward from the horizontal plane; Set the initial position of the laser rangefinder in the roadway coordinate system as P(X0, Y0, Z0), and the position of the prism as T(X t , Y t , Z t ); The position of the 360-degree prism on the coal mining machine equipment is calculated according to the following formula: In the formula, θ is the horizontal direction angle between the laser rangefinder and the 360-degree prism, φ is the vertical direction angle between the laser rangefinder and the 360-degree prism, and S is the oblique distance between the laser rangefinder and the 360-degree prism; The real-time position of the 360-degree prism is the real-time position of the coal mining machine equipment.

3. A positioning device for coal mining equipment based on a laser rangefinder and machine vision, characterized in that, Including: An industrial camera (5) performs real-time tracking on a coal mining machine equipment. Among them, a 360-degree prism is rigidly connected to the coal mining machine equipment; A laser rangefinder (4) is installed above the industrial camera (5) and is used to measure the distance between the laser rangefinder (4) and the 360-degree prism. Among them, the laser rangefinder (4) and the industrial camera (5) point in the same direction; An inclinometer (3) is installed above the laser rangefinder (4) and is used to measure the horizontal direction angle between the laser rangefinder (4) and the coal mining machine equipment and the vertical direction angle between the laser rangefinder (4) and the 360-degree prism (1); The server side is used to acquire the image collected by the industrial camera (5), use the frame difference method to identify the coal mining machine equipment in the image, control the industrial camera (5) to deflect to point to the coal mining machine equipment, and calculate the real-time position of the coal mining machine equipment; Subtract the corresponding pixel values of adjacent frame images to obtain a difference image, and binarize the difference image; If the change in the corresponding pixel value is less than the threshold, this is a background pixel, otherwise this is a pixel of the coal mining machine equipment.

4. The coal mining equipment positioning device based on a laser rangefinder and machine vision according to claim 3, characterized in that, It further includes a carrier table (6). The industrial camera (5) is rotatably arranged above the carrier table (6). The laser rangefinder (4) is rigidly connected to the industrial camera (5), and the inclinometer (3) is installed above the laser rangefinder (4).

5. The coal mining equipment positioning device based on a laser rangefinder and machine vision according to claim 3, wherein, The industrial camera (5) and the laser rangefinder (4) are connected by a ball joint.

6. The coal mining equipment positioning device based on a laser rangefinder and machine vision according to claim 3, characterized in that, The industrial camera (5) is provided with an explosion-proof housing.

7. The coal mining equipment positioning device based on a laser rangefinder and machine vision according to claim 3, characterized in that, The coal mining equipment is at least one of a roadheader, a roadheader-anchoring machine, and a shearer.

Images