Image format conversion system and method for coal pile modeling in shed

Abstract

The present invention provides an image format conversion system for modeling a coal pile in a shed, comprising: a processing unit and a camera, wherein the camera is arranged on the end of a bucket wheel machine arm through a pan / tilt platform; wherein the camera is interactively connected with the processing unit. A method for image format conversion for modeling a coal pile in a shed is also provided. According to an embodiment of the present invention, the problem that an image with three-dimensional coordinate information cannot be obtained before three-dimensional reconstruction of a coal pile in a shed is solved, which is beneficial to the three-dimensional reconstruction of a coal pile in a shed.

Description

Technical Field

[0001] The invention relates to the technical field of image acquisition, and in particular to an image format conversion system for modeling coal piles in a shed and a method using the system. Background Art

[0002] Coal is the main energy source in my country, and its related industries are driving the national economic development. As coal mining technology gradually matures, coal production has also greatly increased, and a number of super-large coal mines with tens of millions of tons have emerged. These high-yield coal mines need to use large coal storage yards to store coal. With the overproduction of coal in my country and the country's transformation of energy strategy, the key work of coal mining enterprises has also expanded to formulating reasonable coal production plans, reasonably controlling coal production and storage according to demand, increasing corporate profits, and reducing corporate losses.

[0003] In addition to coal mining enterprises, in the production and operation activities of enterprises that need to store coal, such as thermal power plants, timely inventory of coal storage is an important part of these enterprises' production and operation activities. Generally, these enterprises have set up departments and personnel specifically responsible for inventorying coal storage, so as to obtain the coal storage in the coal storage in time and calculate the economic benefits. How to quickly and efficiently inventory coal storage and achieve accurate measurement of coal storage in coal storage has always been a difficult problem in the coal industry.

[0004] At present, the widely used coal counting methods mainly include manual tape measurement method and laser measurement method. In recent years, with the development of drone technology and 3D reconstruction technology, the image sequence of the open-air coal storage yard is collected by drone, and then the 3D reconstruction algorithm based on the image sequence is used to generate a 3D real-life model of the coal pile in the open-air coal storage yard, which can better solve the difficulties of coal counting by manual measurement and laser measurement methods.

[0005] Moreover, drones have the advantages of small size, low price and easy operation, and have broad development prospects in 3D modeling. Operators can independently plan flights to cope with different environments, which greatly meets the operational needs of different fields. As a research hotspot in recent years, drone technology combined with computer vision technology has developed rapidly in data acquisition and image processing, and has made great progress, expanding the application field of computer vision.

[0006] However, the current modeling algorithms for coal piles in coal yards of thermal power plants all perform 3D reconstruction of open-air coal piles, while commercial 3D reconstruction software requires image data with GPS information. If the coal pile is blocked from above by buildings such as coal sheds of thermal power plants, drones can only collect image information with GPS (longitude, latitude, elevation) outside the shed, but cannot obtain image information with GPS (longitude, latitude, elevation) of the coal pile inside the shed, which makes subsequent 3D reconstruction impossible.

[0007] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0008] In view of this, the present invention aims to provide an improved technical solution of an image format conversion system and method that can be used for modeling coal piles in a shed.

[0009] In one aspect, an image format conversion system for coal pile modeling in a shed is provided, comprising:

[0010] a processing unit; and

[0011] A camera, wherein the camera is arranged on the end of the bucket wheel machine arm through a pan / tilt platform;

[0012] Wherein, the camera is interactively connected with the processing unit.

[0013] In one embodiment, the processing unit includes: an ARM development board, a communication module for transmitting information, a storage module and an image data module; the communication module, the storage module and the image data module are all arranged on the ARM development board; the communication module is connected to the storage module.

[0014] In one embodiment, the processing unit further includes: a power module for providing power, and the power module is arranged on the ARM development board.

[0015] In one embodiment, the power module includes: a power chip, a power indicator light and a power adapter; the power chip is connected to the power indicator light and the power adapter, and the power adapter is also connected to a battery in the power module.

[0016] In one embodiment, the communication module is an onboard WiFi and Bluetooth two-in-one module.

[0017] In another aspect, there is also provided an image format conversion method for modeling a coal pile in a shed, using the image format conversion system as described above, characterized in that it comprises the following steps:

[0018] S101: Select a reference point outside the shed and select an auxiliary calculation point on the coal yard inside the shed;

[0019] S102: Acquire three-dimensional coordinate information of the reference point;

[0020] S103: using a camera to collect images of the coal pile in the shed and corresponding three-dimensional information;

[0021] S104: converting the three-dimensional information into three-dimensional coordinate information with the reference point as the origin; and

[0022] S105: Convert the image into an image with three-dimensional coordinate information.

[0023] In one embodiment, in step S103, the angle at which the camera captures images matches the angle at which the drone outside the studio captures images.

[0024] In one embodiment, the three-dimensional coordinate information includes: longitude, latitude and altitude.

[0025] In one embodiment, the calculation formula for the conversion in step S104 is:

[0026] When 0°≤α<90°, X0=aL cosβcosα, Y0=L cosβsinα, Z0=L sinβ;

[0027] When 90°≤α<180°, X0=a+L cosβcos(π-α), Y0=L cosβsin(π-α), Z0=L sinβ;

[0028] When 180°≤α<270°, X0=a+L cosβcos(α-π), Y0=-L cosβsin(α-π), Z0=L sinβ;

[0029] When 270°≤α<360°, X0=aL cosβcos(2π-α), Y0=-L cosβsin(2π-α), Z0=Lsinβ;

[0030] L is the arm length of the bucket wheel machine, a is the displacement of the bucket wheel machine, α is the angle formed by the arm of the bucket wheel machine and the reference point of the bucket wheel machine track, β is the pitch angle, m is the length of the coal pile, n is the width, and n0 is the width of the bucket wheel machine track from the edge of the coal pile.

[0031] In one embodiment, the calculation formula for the conversion in step S105 is:

[0032] When 0°≤α<180°, H=h1+Z0 / K;

[0033] When 180°≤α<360°, H=h1+Z0 / K.

[0034] (m1, n1, h1), (m2, n2, h2) are the GPS coordinates (longitude, latitude, elevation) of the reference point and the auxiliary calculation point, respectively. K is the scale of the drone image, and (M, N, H) are the converted image coordinates.

[0035] According to the implementation mode of the present invention, the method provided by the present invention selects a reference point with GPS coordinate information outside the shed, and converts the collected image without three-dimensional coordinate information in the shed into an image with three-dimensional coordinate information with the reference point as the origin. The method provided by the present invention is not only simple but also conducive to the subsequent three-dimensional reconstruction of the coal pile in the shed; the bucket wheel machine in the image format conversion system provided by the present invention is a common facility in the coal yard. The bucket wheel machine is combined with the camera to complete the image acquisition of the coal pile. The processing unit can control and adjust the direction and image acquisition of the camera, and realizes the processing of image data. The image format conversion system solves the problem that it is impossible to obtain an image with three-dimensional coordinate information of the coal pile in the shed before the three-dimensional reconstruction of the coal pile in the shed, which is conducive to the subsequent three-dimensional reconstruction of the coal pile in the shed. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 A schematic diagram of a coal yard in a shed according to an embodiment of the present invention is shown;

[0038] Figure 2 A schematic diagram of the position of a bucket wheel machine arm and a camera according to an embodiment of the present invention is shown;

[0039] Figure 3 A schematic diagram of a processing unit according to an embodiment of the present invention is shown;

[0040] Figure 4 A schematic diagram of a coal yard coordinate system in a shed including reference points according to an embodiment of the present invention is shown;

[0041] Figure 5 A schematic diagram of paths for collecting images inside and outside a studio according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0043] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0044] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0046] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0047] Figure 1 A schematic diagram of a coal yard in a shed according to an embodiment of the present invention is shown. Figure 2FIG. 2 shows a schematic diagram of the positions of the bucket wheel machine arm 2 and the camera 3 according to an embodiment of the present invention. Figure 1-2 As shown, an image format conversion system for modeling coal piles in a shed includes a bucket wheel machine 1 set on a bucket wheel machine track 11, and also includes: a processing unit (not shown), a camera 3; the camera 3 is set on the end of the bucket wheel machine arm 2 of the bucket wheel machine 1 through a pan / tilt 31, and the bucket wheel machine 1 moves along the bucket wheel machine track 11. The camera 3 on the end of the bucket wheel machine arm 2 collects image data of the coal pile in the coal yard in the shed as the bucket wheel machine 1 moves, and the bucket wheel machine arm 2 adjusts the angle with the ground during the process of collecting image data; the camera 3 is interactively connected to the processing unit, and the processing unit can receive image data collected by the camera 3.

[0048] Figure 3 FIG. 2 shows a schematic diagram of a processing unit according to an embodiment of the present invention. Figure 3 As shown, the processing unit includes: an ARM development board, a communication module for transmitting information, a storage module and an image data module for collecting and processing image data; the communication module, the storage module and the image data module are all arranged on the ARM development board; and the communication module is connected to the storage module.

[0049] Preferably, the processing unit uses a Freescale Cortex-A9 series core board, the processor is Freescale i.MX6Q, the main frequency is 1 GHz; the RAM is 1 GB DDR3; and the ROM is 8 GB EMMC.

[0050] The processing unit also includes: a power module for providing power, which is also arranged on the ARM development board. The power module includes: a power chip, a power indicator light and a power adapter; the power chip maintains a stable output of voltage and current, and the power chip is connected to the power indicator light and the power adapter; the power adapter is also connected to the battery in the power module, and the battery can be charged through the power adapter; the power indicator light is used to display whether the power module is connected and whether the processing unit fails. Preferably, the power indicator light is an LED indicator light.

[0051] The communication module can receive the information of displacement, horizontal angle and pitch angle of the bucket wheel machine 1 collected and transmitted by the camera 3, as well as the image information of the coal pile in the coal yard in the shed. Figure 4 FIG. 1 shows a schematic diagram of a coal yard coordinate system in a shed including reference points according to an embodiment of the present invention. Figure 4 In the coordinate system shown, the x-axis is the axis of the bucket wheel machine track 11, the z-axis is perpendicular to the horizontal plane, the origin is the starting movement position of the bucket wheel machine 1, and the reference point is in the negative direction of the x-axis. The horizontal angle is: the angle formed by the projection line of the bucket wheel machine arm 2 on the horizontal plane and the negative x-axis, and the pitch angle is: the angle between the bucket wheel machine arm 2 and the horizontal plane.

[0052] Preferably, the communication module is an onboard WiFi and Bluetooth two-in-one module, and the model is RealTek RTL8723AS Combo Module, IEEE 802.11b / g / n 2.4GHz 1T1R WiFi with Bluetooth v2.1+EDR / Bluetooth3.0 / 3.0+HS / 4.0.

[0053] The storage module includes RAM, ROM and SD card interface, wherein the SD card interface is compatible with the SDXC (UHS-I) standard and can support SD cards up to 512GB. The storage module can receive image information and three-dimensional coordinate information transmitted by the communication module.

[0054] The image data module extracts the image information and the three-dimensional coordinate information stored in the storage module, and processes the information into image information in a target image data format.

[0055] In addition, in drone aerial photogrammetry, the overlap between photos along the same route is called heading overlap; the overlap between photos of adjacent routes is called lateral overlap. Heading overlap and lateral overlap are necessary conditions for stereoscopic observation, stereo measurement, and photo connection. That is, collecting images that meet the heading overlap and lateral overlap requirements is the prerequisite for 3D reconstruction.

[0056] Therefore, when collecting images in the shed, the tilt angle of the camera 3 located at the end of the bucket wheel machine arm 2 should be consistent with the tilt angle of the outdoor drone camera lens. Figure 5 FIG. 1 shows a schematic diagram of the path of collecting images inside and outside the studio according to an embodiment of the present invention. Figure 5 As shown, the path and shooting angle of the camera 3 in the studio when collecting images must meet the requirements of image overlap.

[0057] The degree of overlap of adjacent images on the same route is called heading overlap; the degree of overlap of images between adjacent routes is called lateral overlap. The general commercial 3D modeling software requires 60% heading overlap and 40% lateral overlap. When the shape of the coal pile changes greatly, the image overlap can be appropriately increased to ensure the reconstruction accuracy.

[0058] The working process of the image format conversion system for modeling coal piles in the shed is as follows: the bucket wheel machine 1 starts from the starting position of the bucket wheel machine track 11 and moves along the bucket wheel machine track 11, the bucket wheel machine arm 2 moves with the movement of the bucket wheel machine 1, and the camera 3 moves with the movement of the bucket wheel machine arm 2; during the movement, the horizontal angle and pitch angle of the bucket wheel machine arm 2 are adjusted, and at the same time, the processing unit controls the pan / tilt 31 to adjust the shooting angle of the lens of the camera 3, so that the angle of the image collected by the camera 3 matches the angle of the image collected by the drone; during the movement, the camera 3 collects the displacement, horizontal angle and pitch angle information of the bucket wheel machine 1, as well as the image information of the coal pile in the coal yard in the shed; the camera 3 transmits the collected information to the communication module in the processing unit, the communication module receives the data and transmits the data to the storage unit, and the image data module extracts the data stored in the storage unit and converts the image into an image format with three-dimensional coordinate information.

[0059] A method for image format conversion for modeling coal piles in a shed is also provided, which uses the image format conversion system as described above and includes the following steps:

[0060] S101: Select a reference point outside the shed and select an auxiliary calculation point on the coal yard inside the shed

[0061] like Figure 1 As shown, a reference point is selected outside the coal yard in the shed, and the reference point is located on the extension line of the bucket wheel machine track 11. In addition, an auxiliary calculation point is selected on the coal yard in the shed. Relative to the coal pile, the auxiliary calculation point is located in the diagonal direction of the reference point. Note that the longitude and latitude of the reference point and the auxiliary calculation point are different.

[0062] S102: Obtaining three-dimensional coordinate information of the reference point

[0063] Before capturing an image, a wireless communication positioning device such as a mobile phone is used to obtain three-dimensional coordinate information of the reference point, namely, GPA coordinate information; and the obtained three-dimensional coordinate information of the reference point is sent to a processing unit.

[0064] S103: Using camera 3 to collect images of the coal pile in the shed and corresponding three-dimensional information

[0065] like Figure 2As shown, the bucket wheel machine arm 2 is adjusted to the maximum height that can be reached, and the bucket wheel machine 1 is started so that the bucket wheel machine 1 starts to move along the bucket wheel machine track 11 from one end close to the reference point, and the bucket wheel machine arm 2 moves with the movement of the bucket wheel machine 1; the camera 3 installed at the end of the bucket wheel machine arm 2 through the pan / tilt 31 moves with the movement of the bucket wheel machine arm 2; during the movement, the horizontal angle and pitch angle of the bucket wheel machine arm 2 are adjusted, and at the same time, the processing unit controls the pan / tilt 31 to adjust the shooting angle of the lens of the camera 3, so that the angle of the image collected by the camera 3 matches the angle of the image collected by the drone; the camera 3 completes the image without three-dimensional coordinate information and the corresponding three-dimensional information collection of the coal pile, and transmits the collected information to the communication module in the processing unit, and the communication module stores the received information in the storage module.

[0066] S104: Convert the three-dimensional information into three-dimensional coordinate information with the reference point as the origin

[0067] like Figure 4 In the coordinate system shown, the x-axis is the axis of the bucket wheel machine track 11, the z-axis is perpendicular to the horizontal plane, the origin is the starting movement position of the bucket wheel machine 1, and the reference point is in the negative direction of the x-axis; the image data module in the processing unit converts the three-dimensional information of the collected image into corresponding three-dimensional coordinate information with the reference point as the origin, and stores the processed information in the storage module.

[0068] S105: Convert the image into an image with three-dimensional coordinate information.

[0069] The collected image without three-dimensional coordinate information is combined with the three-dimensional coordinate information stored in the storage module through the image data module in the processing unit to generate an image with three-dimensional coordinate information.

[0070] In some embodiments, the three-dimensional coordinate information is: longitude, latitude, and elevation.

[0071] like Figure 4 As shown, in some embodiments, the calculation formula for the conversion in step S104 is:

[0072] When 0°≤α<90°, X0=aL cosβcosα, Y0=L cosβsinα, Z0=L sinβ;

[0073] When 90°≤α<180°, X0=a+L cosβcos(π-α), Y0=L cosβsin(π-α), Z0=L sinβ;

[0074] When 180°≤α<270°, X0=a+L cosβcos(α-π), Y0=-L cosβsin(α-π), Z0=L sinβ;

[0075] When 270°≤α<360°, X0=aL cosβcos(2π-α), Y0=-L cosβsin(2π-α), Z0=Lsinβ;

[0076] L is the length of bucket wheel machine arm 2, a is the displacement of bucket wheel machine 1, α is the angle formed by bucket wheel machine arm 2 and the reference point of bucket wheel machine track 11, β is the pitch angle, m is the length of coal pile, n is the width, and n0 is the width of bucket wheel machine track 11 from the edge of coal pile.

[0077] In some embodiments, the calculation formula for the conversion in step S105 is:

[0078] When 0°≤α<180°, H=h1+Z0 / K;

[0079] When 180°≤α<360°, H=h1+Z0 / K.

[0080] (m1, n1, h1), (m2, n2, h2) are the GPS coordinates (longitude, latitude, elevation) of the reference point and the auxiliary calculation point, respectively. K is the scale of the drone image, and (M, N, H) are the converted image coordinates.

[0081] According to the implementation mode of the present invention, the method provided by the present invention selects a reference point with GPS coordinate information outside the shed, and converts the collected image without three-dimensional coordinate information in the shed into an image with three-dimensional coordinate information with the reference point as the origin. The method provided by the present invention is not only simple but also conducive to the subsequent three-dimensional reconstruction of the coal pile in the shed; the bucket wheel machine 1 in the image format conversion system provided by the present invention is a common facility in the coal yard. The bucket wheel machine 1 is combined with the camera 3 to complete the image acquisition of the coal pile. The processing unit can control and adjust the direction and image acquisition of the camera 3, and realizes the processing of the image data. The image format conversion system solves the problem that the image with three-dimensional coordinate information of the coal pile in the shed cannot be obtained before the three-dimensional reconstruction of the coal pile in the shed, which is conducive to the three-dimensional reconstruction of the coal pile in the shed.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0083] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image format conversion method for modeling coal piles in a shed, using an image format conversion system, the image format conversion system comprising: Processing unit; and a camera, wherein the camera is arranged on the end of the bucket wheel machine arm through a pan / tilt platform; wherein the camera is interactively connected with the processing unit, and the method is characterized in that it comprises the following steps: S101: Select a reference point outside the shed and select an auxiliary calculation point on the coal yard inside the shed; S102: Acquire three-dimensional coordinate information of the reference point; S103: using a camera to collect images of the coal pile in the shed and corresponding three-dimensional information, wherein the angle at which the camera collects the image in step S103 matches the angle at which the drone collects the image outside the shed; S104: Converting the three-dimensional information into three-dimensional coordinate information with the reference point as the origin, specifically including: when hour, , , ; when hour, , , ; when hour, , , ; when hour, , , ; X0 is the longitude with the reference point as the origin, Y0 is the latitude with the reference point as the origin, Z0 is the elevation with the reference point as the origin, L is the arm length of the bucket wheel machine, a is the displacement of the bucket wheel machine, α is the angle formed by the arm of the bucket wheel machine and the reference point of the bucket wheel machine track, and β is the pitch angle; S105: Convert the image into an image with three-dimensional coordinate information, where the three-dimensional coordinate information is: longitude, latitude and altitude, specifically including: when hour, , , ; when hour, , , ; (m1, n1, h1), (m2, n2, h2) are the GPS coordinates (longitude, latitude, elevation) information of the benchmark point and the auxiliary calculation point respectively, m is the length of the coal pile, n is the width, n0 is the width of the bucket wheel machine track from the edge of the coal pile, K is the UAV image scale, and (M, N, H) are the converted image coordinates.

Images