Terrain-fused ranging method for straw burning smoke image based on monocular ptz

Abstract

The present invention relates to the technical field of image processing, and provides a terrain-fused ranging method for a straw burning smoke image based on monocular PTZ. The technical solution comprises the following steps: S1. using a camera to capture a highly identifiable scene view as a calibration image, and calibrating a pitch angle; and S2, incorporating natural terrain elevation data of a location of a camera and correcting the accuracy thereof. The beneficial effects of the present invention are: the present invention utilizes the mechanical rotation and high-resolution image acquisition capabilities of a PTZ camera and computer vision and image processing technologies to achieve accurate detection and localization of smoke in outdoor environments.

Description

Terrain ranging method based on monocular PTZ and straw burning smoke image fusion

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 28, 2023, with application number 202311832199X and invention name “Topography ranging method based on monocular PTZ and fusion of straw burning smoke images”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the technical field of image processing, and in particular to a terrain ranging method based on monocular PTZ and fusion of straw burning smoke images. Background Art

[0003] With the rapid development of China's agriculture, the disposal of crop straw has become an increasingly serious problem. Due to the limited comprehensive utilization options for straw and the prevalence of straw burning in some areas, the Chinese government has recently begun to address the issue of straw burning, introducing a series of policies to restrict and prohibit it. Straw burning damages soil structure, affects farmland quality, and can even cause safety issues such as fires. Therefore, finding efficient, stable, and cost-effective ways to monitor, identify, and locate straw burning has become a major challenge.

[0004] While ensuring wide-area coverage and high resolution, surveillance systems based on pan / tilt / zoom (PTZ) cameras also face challenges. One of these challenges is how to accurately and efficiently detect and locate smoke in outdoor environments and issue timely alerts.

[0005] Traditional smoke detection methods rely on specialized devices such as smoke sensors or photodiodes. These devices typically need to be deployed individually and struggle to achieve coverage over large areas. Furthermore, factors such as lighting and wind in outdoor environments can interfere with the accuracy of traditional smoke detection equipment.

[0006] Existing image processing positioning technologies often require obtaining a large number of internal and external camera parameters, and deriving the target distance through theoretical mathematical calculations. However, it is difficult to accurately obtain external parameters when the camera is installed, resulting in large errors in the results of the inference formula calculations and difficulty in correction. In addition, for cameras that have been deployed at high altitudes, and when external parameters are not accurately achieved or recorded during deployment, how to effectively calibrate the camera's pitch angle and measure the distance to the observed target has also become a major technical challenge. These problems are particularly prominent in cases where installation is inconvenient or where additional hardware support is not supported. Therefore, it is necessary to research more accurate and efficient methods to improve existing image processing positioning technologies.

[0007] et al., in the paper "Integration of forest fire video monitoring system and geographic information system," used a web-based geographic information system data interface to manually resolve the location of observed targets by clicking on coordinates on a map. This lack of automatic recognition using image data resulted in unstable ranging accuracy. In the paper "An unmanned aircraft system for automatic forest fire monitoring and measurement," Merino et al. proposed using drones equipped with infrared cameras to patrol fire spots and provide coordinate information. This, however, requires additional equipment and results in inefficient patrol response.

[0008] Summary of the Invention

[0009] The purpose of the present invention is to provide a monocular PTZ-based straw burning smoke image fusion terrain ranging method. By measuring partial data of a selected calibration scene, a pitch angle correction formula is derived to realize a full-scene pitch angle solution method based on the calibration image position. A two-stage ranging algorithm is used to measure the distance of images with smoke information detected by a PTZ camera using a plane solution model assuming an ideal ground plane and a fusion terrain solution model superimposed with digital elevation data. Multi-dimensional correction is performed on the PTZ camera, and a digital elevation model is introduced for fusion ranging to ultimately achieve the demand for accurate ranging.

[0010] The inventive concept of the present invention is as follows: the present invention proposes a monocular PTZ-based straw burning smoke image fusion terrain ranging method, which adopts a pitch angle calibration method to select a calibration scene and measure the elevation value of the calibration scene, the calibration distance value and the deployment height and other parameters to solve the pitch angle of the image to be measured based on the calibration image position; the straw burning smoke image fusion terrain ranging method takes the image of the detected smoke and the internal parameters and deployment parameters of the camera as input, and integrates the digital elevation model data around the camera deployment position to implement a two-stage algorithm model for ranging. The first-stage model of the straw burning smoke image fusion terrain ranging method is a plane solution model. This model abstracts a geometric model based on the camera imaging principle and integrates pitch angle, roll angle, scaling, and pixel correction algorithms to achieve straw burning smoke ranging in a plane. The second-stage model of the straw burning smoke image fusion terrain ranging method is a fusion terrain solution model. This model uses the plane solution model's ranging results to overlay DEM data to obtain terrain profile data between the camera deployment location and the straw burning smoke location, achieving straw burning smoke ranging in natural terrain. The framework diagram of the straw burning smoke image fusion terrain ranging method is shown in Figure 1.

[0011] In order to achieve the above-mentioned invention object, the present invention adopts a technical solution specifically as follows: a method for measuring distance between straw burning smoke images and terrain based on a monocular PTZ, comprising the following steps:

[0012] S1. Use a camera to view a certain iconic scene view as a calibration image and calibrate the pitch angle.

[0013] S2. Introduce the natural terrain elevation information where the camera is located and correct its accuracy.

[0014] The present invention also proposes an image calibration method based on a PTZ camera, which uses the camera to regard a scene view with high identification as a calibration image to calibrate the pitch angle. The pitch angle calibration model diagram is shown in Figure 4.

[0015] The specific steps include:

[0016] The first step is to collect an image with identification as the calibration image.

[0017] In the second step, since the optical center position of the camera remains unchanged when performing optical zoom, for any zoom factor, the horizontal distance D between the actual position on the map corresponding to the optical center position of the camera and the camera is ref The actual distance in the world coordinate system can be calculated. In the process of finding the corresponding position on the map of the center position of each calibration image, human operation may introduce certain errors, resulting in a difference between the actual distance and the measured distance of the calibration image.

[0018] In the third step, since there is an elevation difference between the camera deployment position and the calibration position, and the distance between the camera deployment position and the calibration position measured by the map is a bird's-eye view, if the camera's hanging height and the calibration distance are directly used to calculate the pitch angle according to the triangulation method, an elevation error will occur. Therefore, measuring the elevation value of the camera deployment position and the elevation value of the calibration image position can correct the error. Use the map tool to measure the distance between the camera deployment position and the found point to obtain the calibration distance D ref , and record the elevation value E of the camera deployment location base , the tilt angle T corresponding to the calibration image ref , the elevation value E of the calibration position ref .

[0019] Step 4: According to the calibration distance D ref The pitch angle of the calibration position can be calculated by comparing the camera's hanging height h The calculation formula is:

[0020] The fifth step is to determine the pitch angle corresponding to the known calibration image position. Tilt angle T ref And the tilt angle T of the image to be measured, the pitch angle of the image to be measured can be calculated. The calculation formula is:

[0021] Combining formula 1 and formula 2, we get:

[0022] When the pitch angle is corrected, a certain degree of roll angle offset occurs when the PTZ camera is deployed, which causes additional error in the correction. To avoid an increase in the error, roll angle correction is introduced.

[0023] The coordinate system POT is established with the camera's P and T values ​​as its horizontal and vertical axes, respectively. The horizontal axis is the geoid's horizontal plane, P', and the vertical axis is the geoid's vertical line, T'. There is a rotation angle ω between the two coordinate systems. When the camera moves from point M to point N, it moves along the coordinate axis POT by a distance of line segment |MN|, as shown in Figure 5.

[0024] Treat the MN segment as a vector Will Move to position (Nx-Mx, Ny-My), The starting point M is moved to the origin of the coordinate axis, and the end point N is decomposed into ΔNT and ΔNP along the coordinate axis POT. ΔNT is projected onto the OT' axis of the coordinate system P'OT' as ΔNT', and ΔNP is projected onto the OT' axis of the coordinate system P'OT' as ΔNP'. The parameter change vectorization and its projection are shown in Figure 6.

[0025] calculate The projection length ΔNT on the coordinate axis OT and the projection length ΔNP on the coordinate axis OP are:

[0026] Then calculate the projection of ON' on the coordinate axis OT' in the coordinate system P'OT' as |NT'|, and the projection of ON' on the coordinate axis OT' as |NP'|:

[0027] Consider Nx as the P value of the image to be tested, Mx as the P value of the calibration image, Ny as the T value of the position of the image to be tested, and My as the T value of the position of the calibration image. Combining Formula 4 and Formula 5, we get:

[0028] The concept of "transfer increment" is introduced. The transfer increment is the actual offset of the horizontal or vertical axis between two states with roll angle correction. The pitch angle transfer increment is the sum of the difference between the T value of the test image and the T value of the calibration image multiplied by the cosine of the roll angle and the difference between the P value of the test image and the P value of the calibration image multiplied by the sine of the roll angle.

[0029] Let the transfer increment be I, then the pitch angle transfer increment is I T for:

[0030] Then transfer increment I T Indicates that the PTZ camera has a roll angle ω from (P ref , T ref ) position to the (P, T) position.

[0031] Then the angle change from the calibration image position to the measured image position is corrected as:

[0032] Use Formula 7 and Formula 8 to correct Formula 3 and get:

[0033] Where T is the tilt angle value of the image to be measured; T ref is the tilt angle value of the calibration image;

[0034] E base E is the elevation value of the camera deployment location; ref is the elevation value of the optical center of the calibration image; D ref It is the horizontal distance between the optical center of the calibration image and the camera position.

[0035] Plane solution model design content:

[0036] First, we introduce an ideal model, assuming that the observation target is an ideal plane at the bottom of the camera mount. The known information in each input image scene includes: camera model, mounting height (h), pan angle (P), tilt angle (T), zoom factor (Z), and captured image resolution. Based on this information, we abstract and model the image vertically. In general, we set the camera center to O. Draw a line through point O that defines the vertical field of view boundary of the camera and intersects the horizon at points A and B. Let the vertical projection of the camera imaging plane be line segment A'B'. Draw a perpendicular line through point O to line AB that intersects at point D, which is the camera mounting height |OD|. Draw a line through point O that defines the camera's optical center and intersects line AB at point C. The line segment A'B' also intersects at point C'. In a real-world scenario, within the field of view (FOV), any point P on line segment AB is selected as the observation object. A ray PO is drawn through point P and intersects line segment A'B' at point P'. For ease of comparison, the camera imaging plane is symmetric about point O to construct the corresponding imaging equivalent planes containing A", B", C", and P". Subsequent analysis of triangle A'OB' is equivalent to analysis of triangle A"OB". For simplicity, ∠COB is denoted by ψ, ∠POC by η, and ∠OCD by θ. This completes the model, as shown in Figure 2.

[0037] For the system model in Figure 2, the existing problem is transformed into solving the modulus |DP| of line segment DP given the modulus |A″B″| of line segment A″B″ and the modulus |OD| of line segment OD. In RT△ODP, we have:

[0038] The value of |DP| can be obtained according to formula 10.

[0039] According to the angle relationship in Figure 2:

[0040] Combining Formula 11 and Formula 12, we get:

[0041] For the angle value η, when point P is on line segment A"C", θ=T+η; when point P is on line segment B"C", θ=T-η, that is:

[0042] To simplify the formula, the angle value η is vectorized and the vector is taken The direction is positive, so the sign can be brought into the physical quantity. Since the arctan() function is an odd function, we have:

[0043] Combining Formula 13, Formula 14, and Formula 15, we have:

[0044] At this time, P"C" refers to the value of |P"C"| and is signed. The direction is positive.

[0045] Then formula 14 is simplified to:

[0046] Combining Formula 16 and Formula 17, we get:

[0047] Combining Formula 1 and Formula 9, we get:

[0048] Simplifying the formula, we get:

[0049] Where dis = |DP|, which is the horizontal distance of the observed target from the camera; h = |OD|, which is the camera mounting height; Δy_phy = P"C", which is the actual sensor size corresponding to the distance of the observed target from the optical center on the y-axis of the imaging plane; and f = |OC"|, which is the current focal length.

[0050] For PTZ cameras, which have optical multi-zoom capabilities, the above model construction only considers the ranging model solution for the camera's fixed field of view and single focal length. Because zooming causes some parameters to change, some corrections are required to optimize the model.

[0051] The change of the optical zoom parameter Z will affect the change of the focal length, and the focal length needs to be corrected for the parameter Z.

[0052] For the optical maximum zoom parameter Z max The relationship between the focal length and the focal length is:

[0053] That is: f max =Z max f min (twenty two)

[0054] When f determines the focal length, for the minimum focal length f min The current optical zoom factor Z is related to the current focal length as follows: f = Zf min (twenty three)

[0055] Among them, Z is the zoom factor of the camera, f min is the minimum focal length of the camera.

[0056] To solve Δy_phy, it is necessary to obtain the physical dimensions of the camera's image sensor, denoted as height sensor_h and width sensor_w. The relationship between point P" and line segment A"B" in Figure 2 is equivalent to the relationship between the corresponding pixel point of the observation target P on its imaging plane and the vertical resolution of the image. At the same time, it is also mapped to the relationship between the position of the equivalent point of the observation target in the height direction of the camera image sensor and the height of the sensor. The mapping relationship is shown in Figure 7. In Figure 7, plane α is the physical dimension plane of the camera image sensor, β is the pixel resolution of the camera output image, and the pixel vertical resolution is denoted as resoultion_h. P" is the position mapping of the observation target on the corresponding plane.

[0057] Where Δy_pix is ​​the difference between the y-axis coordinate value dst_y of the observed target P" in the image coordinate system and the image center, that is:

[0058] The difference in physical size between the mapping position of point P" on the physical size plane of the image sensor and the center of the physical size plane in the height direction is Δy_phy. Equivalently mapping point P" on the pixel resolution plane and the physical size plane yields:

[0059] Combining Formula 21 and Formula 22, we get:

[0060] Where dst_y is the y-axis coordinate of the observed target in the image coordinate system; resoultion_h is the vertical resolution of the image; and sensor_h is the height of the physical size of the sensor.

[0061] Correction formula and judgment conditions:

[0062] Combining all the previous corrections, and combining Equation 11, Equation 20, Equation 23, and Equation 26, we get:

[0063] Formula 27 has many parameters, and the domain of definition must be guaranteed for each parameter. The parameters of the tan(θ) function should satisfy:

[0064] In terms of practical significance, θ is the angle between the horizontal plane and the straight line between the optical center and the observed target, and its domain should be:

[0065] Connect with the actual meaning, |TT ref |cos(ω),|PP ref |sin(ω) is a non-negative number; h, E base 、E ref and D ref is a non-negative number; resoultion_h is the vertical resolution of the image, which is a non-zero positive number; dst_y is defined in [0, resoultion_h], sensor_h is the height of the physical size of the camera sensor, which is a non-zero positive number; f min is the minimum focal length of the camera, which is a non-zero positive number; Z is defined in [0, f max / f min ] is a non-zero positive number, and the angle of orientation is positive. In summary, the constraint condition is formula 30.

[0066] Formula 30 is the judgment formula of Formula 27. When the relevant parameters meet Formula 30, the result calculated by Formula 27 is valid.

[0067] Fusion terrain solution model design content:

[0068] The limitation of Equation 11 lies in its assumption of an ideal ground plane. To overcome this, we introduce Digital Elevation Model (DEM) data from the Geographic Information System (GIS). We overlay the elevation data of corresponding locations in the model and calculate the intersection of the light path and the elevation profile curve as the result of the fused terrain model. Figure 3 shows the fused terrain solution model.

[0069] Under the premise of meeting the above conditions, the model solves the data equivalent to assuming that the target point is at the same elevation level as the camera position. However, due to terrain variations, this simplification can lead to increased data errors. To more effectively address this issue, we further introduce the natural terrain elevation information of the camera location and correct its accuracy. The specific implementation steps of the correction method are as follows:

[0070] The first step is to obtain digital elevation model images (.tif) files within a certain range around the camera's deployment location from the offset-free map source "Tiantu Source." These images are projected using the longitude and latitude projection of the WGS84 coordinate system. This ensures that the acquired images correspond to the actual geographic location, providing accurate basic data for subsequent processing and analysis.

[0071] The second step is to convert the offset longitude and latitude coordinates of the camera deployment location into the non-offset longitude and latitude coordinates of the WGS84 coordinate system through the projected coordinate system.

[0072] The third step is to load and parse the elevation data image file. The image is annotated based on the camera's offset-free WGS84 coordinate system. This creates a contour map of a certain area around the camera's deployment location. This map provides a visual representation of the surrounding elevation changes. Figure 8 shows the contour map around the camera.

[0073] The fourth step is to obtain the map azimuth of the line connecting the camera and the target when the camera is aimed at them based on the camera's initial azimuth and the P parameter (using true north as 0 degrees and constructing the coordinate system in a clockwise direction). Based on the azimuth and the camera's longitude and latitude coordinates in the unbiased WGS84 coordinate system, a straight line (hereinafter referred to as the optical path) is drawn on the contour map. This step visually displays the relative orientation of the target and the camera. The optical path when the camera is aimed at the target is shown in Figure 9.

[0074] The fifth step is to extract the elevation values ​​on the map along the line, based on the elevation data image file and the information about the line connecting the two when the camera is aligned with the target. The geographic latitude and longitude information corresponding to the sampling points on the line is converted into geographic distance information from the camera deployment point. A correlation curve is constructed between the elevation values ​​corresponding to the sampling points on the line and the distance values ​​from the sampling points on the line to the camera deployment point. This curve intuitively illustrates the terrain differences along the line at varying distances from the camera deployment point. A distance-elevation curve along the camera optical axis is shown in Figure 10.

[0075] The sixth step is to draw a model solution diagram of the triangulation distance measurement method on the correlation curve of the elevation value corresponding to the sampling point on the straight line and the distance value between the sampling point on the straight line and the camera deployment point, including drawing the camera position point according to the hanging height of the camera deployment point, drawing the observation target position point according to the observation target distance value calculated by the plane solution model, and drawing the optical path connection between the camera position point and the observation target position point. The optical path connection diagram between the camera and the observation target position point is shown in Figure 11.

[0076] In the seventh step, the interpolation method is used to calculate the intersection of the straight line where the light path is located and the elevation curve. This intersection is the distance correction value that integrates the natural terrain elevation information. Depending on the terrain, the intersection exists in two forms as shown in Figures 12 and 13.

[0077] This correction method is effective when there is a large elevation change between the camera deployment point and the observation target.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. By utilizing the advantages of PTZ cameras, the outdoor smoke high-precision alarm positioning system based on this algorithm can achieve smoke detection and positioning in a wide area, which is of great significance for improving the efficiency of security monitoring systems and reducing false alarm rates.

[0080] To overcome these problems, a high-precision outdoor smoke alarm positioning method based on PTZ cameras was proposed. This method utilizes the mechanical rotation and high-resolution image acquisition capabilities of PTZ cameras, combined with computer vision and image processing technologies, to accurately detect and locate smoke in outdoor environments.

[0081] 3. This invention is designed for PTZ cameras deployed at different heights. It assumes that the image to be measured is an image that has already undergone target detection and that the target is located in the center of the image. This assumption allows for the conversion of two-dimensional image ranging into one-dimensional ranging, specifically, research into ranging algorithms for the vertical direction of the image. The research involves a two-stage ranging algorithm that assumes a planar solution model based on an ideal ground plane and a fused terrain solution model based on superimposed digital elevation data. This algorithm analyzes images of identified smoke and calculates the horizontal distance, elevation difference, and latitude and longitude between the smoke location and the camera. It then locates smoke in the captured image, combining the rotation information of the PTZ camera to calculate the exact location of the smoke source in real space.

[0082] 4. This invention proposes a high-precision outdoor smoke alarm location method based on a PTZ camera. This method utilizes the mechanical rotation and high-resolution image acquisition capabilities of a PTZ camera, combined with computer vision and image processing technologies, to accurately detect and locate smoke in outdoor environments. The core concept of this method is to analyze images of identified smoke to calculate the horizontal distance and elevation difference between the smoke location and the camera, as well as the latitude and longitude of the smoke. Smoke is located in the captured images of identified smoke, and the precise location of the smoke source in real space is calculated based on the rotation information of the PTZ camera. Once the smoke is successfully located, the system can promptly send an alarm notification to security personnel so that they can take appropriate measures.

[0083] 5. By utilizing the advantages of PTZ cameras, the outdoor smoke high-precision alarm positioning system based on this method can achieve smoke detection and positioning in a wide area, which is of great significance for improving the efficiency of security monitoring systems and reducing false alarm rates.

[0084] 6. This invention proposes a method for cameras deployed at height. This method can calibrate some external parameters even when external parameter information is lacking. This method, combined with the PTZ camera's rotation information, allows for distance and location measurement of identified smoke targets. This method is highly robust and can accurately measure distance and position under various environmental conditions, providing precise data support for subsequent smoke target analysis and processing.

[0085] 7. The fusion terrain solution model of the present invention combines image information and terrain data. For deployed cameras, it can perform practical distance solution in natural environments without additional hardware support or excessive manual adjustments.

[0086] 8. After seven experiments, the present invention achieved an average ranging error of 85.14 meters, an error accuracy of 3.90%, and an error accuracy variance of 0.05%. Compared with the manufacturer's given solution data, the average error accuracy dropped to 7.045% and the variance dropped to 0.0984% of the manufacturer's given solution data, significantly reducing the ranging error value and improving the stability of the ranging results.

[0087] Figures in the specification

[0088] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0089] FIG1 is a framework diagram of a monocular PTZ-based method for fusing straw burning smoke images with terrain ranging;

[0090] FIG2 is a mathematical model diagram of the present invention;

[0091] FIG3 is a diagram of a terrain fusion solution model of the present invention;

[0092] FIG4 is a schematic diagram of a pitch angle calibration method according to the present invention;

[0093] FIG5 is a schematic diagram of the offset of the roll angle coordinate system of the present invention;

[0094] FIG6 is a schematic diagram of the roll angle coordinate system projection transformation according to the present invention;

[0095] FIG7 is a mapping diagram of the pixel plane and the sensor plane of the present invention;

[0096] FIG8 is a contour map of the surrounding area of ​​the camera of the present invention;

[0097] FIG9 is a diagram of the light path when the camera of the present invention is aimed at the observation target;

[0098] FIG10 is a distance-elevation curve on the optical axis of the camera of the present invention;

[0099] FIG11 is a diagram showing the optical path between the camera and the observation target position of the present invention;

[0100] FIG12 is a diagram showing the actual position intersection solution of the convex terrain according to the present invention;

[0101] FIG13 is a diagram showing the actual intersection point calculation of concave terrain according to the present invention;

[0102] FIG14 is a satellite image of camera deployment locations according to an embodiment of the present invention;

[0103] FIG15 is a diagram of smoke from burning straw according to an embodiment of the present invention;

[0104] FIG16 is a diagram showing the acquisition of measured distances according to an embodiment of the present invention;

[0105] FIG17 is a pitch angle calibration diagram according to an embodiment of the present invention;

[0106] FIG18 is a diagram illustrating obtaining a calibration distance according to an embodiment of the present invention;

[0107] FIG19 is a diagram illustrating obtaining a calibrated roll angle according to an embodiment of the present invention;

[0108] FIG20 is a contour map of an embodiment of the present invention;

[0109] FIG21 is a diagram illustrating a distance-elevation relationship solution according to an embodiment of the present invention. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0111] Example 1

[0112] Referring to Figures 1 to 21, this embodiment is based on the above-mentioned method. Here, an actual environment case is used for experiment. An actual straw burning smoke image taken in an environment with a PTZ camera is used as the image to be measured. The pitch angle calibration method of this method and the implementation of the straw burning smoke image fusion terrain ranging method are introduced in detail.

[0113] 1. Environment.

[0114] For example, a PTZ camera deployed on a China Tower in Lilinzhuang Village, Shuozhou City, Shanxi Province, China, is model SD04-TT, released by Zhejiang Dahua Technology Co., Ltd. Its latitude and longitude coordinates on AutoNavi Maps are (112.606994555, 39.297415243). Its positioning on the "91 Satellite Map Assistant" satellite map software is shown in Figure 14. A sample of straw smoke images captured by this camera, using data provided by China Tower, is shown in Figure 15. The image of straw smoke to be measured is used for the distance calculation.

[0115] The solver will be run on an Intel i5-12400 CPU platform, with Python 3.8.17 as the runtime environment, and referenced third-party libraries such as GDAL 2.3.3, geopy 2.4.0, and scipy 1.10.1.

[0116] 2. Data collection.

[0117] According to the local ground level, the camera's hanging height h is 45 meters and the installation azimuth is 281.57° north-east.

[0118] Use an online latitude and longitude conversion tool to convert the original latitude and longitude from the Amap map to the offset-free latitude and longitude (112.6005794820504, 39.296613752335965). The image state to be determined is: P = 323.6, T = -3.1, Z = 14. The actual distance between the target location and the camera deployment position obtained using the 91 Satellite Map Assistant satellite map software is 3005 meters. The results are shown in Figure 16.

[0119] 3. Pitch angle calibration.

[0120] From the data provided by China Tower Corporation, the local environment is selected as the pitch angle calibration image, as shown in Figure 17. The status of the calibration image is: P ref =325.4, T ref =2.1, Z=7. The distance between the optical center of the calibration image and the camera is obtained in the 91 Satellite Map Assistant satellite map software. The measurement results are shown in Figure 18. The result is D ref =410.142m. The elevation values ​​of the camera deployment position and the optical center position of the calibration image obtained in the DEM data are E base =1051.5m, E ref =1054.8m. ImageJ software was used to measure the roll angle of the image under test. A straight line in the image was selected for measurement. In an angular coordinate system with the right side of the horizontal direction at 0 degrees and clockwise as positive, the measured image roll angle was ω = -2.16°. The results are shown in Figure 19. Based on the currently known information, the solution is calculated using Equation 27:

[0121] The camera pitch angle ψ corresponding to the image to be measured is solved to be -0.677°.

[0122] 4. Plane solution model distance measurement.

[0123] According to the image information in Figure 15, the image resolution is 2560*1440, that is, resolution_h = 1440. According to the information of the target detection frame in the image, the coordinates of the midpoint of its lower edge in the image coordinate system are (1038,751), that is, dst_y = 751. According to the camera model, the sensor size is obtained as 5.32mm*7.18mm, that is, sensor_h = 7.18mm, and the minimum focal length f min =5.5mm. Combined with the current image zoom factor Z = 14, the following formula is used to calculate:

[0124] The solution is η = 0.115°.

[0125] Combined with the above solution result ψ=-0.677°, the result is judged by the judgment formula 30:

[0126] When the judgment conditions are met, the distance calculation begins, and is calculated using Formula 27:

[0127] The distance calculated using the plane solution model is dis = 3255.172 m. Evaluation: Actual distance is 3005 m, with an error of 250 m.

[0128] 5. Integrate terrain solution model for ranging.

[0129] Figure 20 shows a schematic diagram of the unbiased longitude and latitude elevation contours of the camera deployment location derived from the DEM data. In this figure, the points represent the camera deployment locations, and the lines represent the azimuth angles of the line connecting the camera and the target on the map. Based on this diagram, the elevation profile data corresponding to the optical path is converted into a two-dimensional coordinate system constructed with the distance from the camera deployment location as the horizontal axis and the profile elevation as the vertical axis. This displays the distance-elevation relationship (ElevationProfile). Based on the existing parameters, the camera location CamH, the equivalent plane base constructed based on the camera deployment location, and the target point p_dst_d obtained by the plane solution model are plotted on the diagram. The distance-elevation relationship diagram is shown in Figure 21. In this diagram, the point CamH corresponding to the camera location is connected to the target point p_dst_d obtained by the plane solution model. Interpolation is used to obtain the actual target point p_real after superimposing the elevation data. This method solves the abscissa value of the actual target point p_real to -2956.77 m, where the symbol represents the direction and the value represents the distance from the camera deployment location. Assessment: Actual distance 3005m, error 49m.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A monocular PTZ-based straw burning smoke image fusion terrain ranging method, characterized in that, It includes the following steps: S1. Use a camera to calibrate the pitch angle of a highly distinctive scene view regarded as a calibration image; S2. Introduce the natural terrain elevation information at the location of the camera and correct its accuracy.

2. The method for fusing terrain ranging with straw burning smoke images based on monocular PTZ according to claim 1, wherein The step S1 includes the following steps: S11. Collect an image with high distinctiveness as the calibration image; S12. Since the optical center position of the camera remains unchanged during optical zooming, for any zoom ratio, the horizontal distance D between the actual position on the map corresponding to the optical center position of the camera and the camera ref is used to calculate the true distance in the world coordinate system. During the process of finding the corresponding position on the map at the center position of each calibration image, manual operation will introduce certain errors, resulting in a difference between the actual distance and the measured distance of the calibration image; S13. Since there is an elevation difference between the camera deployment position and the calibration position, and the distance between the camera deployment position and the calibration position measured by the map is a top-down distance measurement, if the hanging height of the camera and the calibration distance measurement are directly used, the pitch angle calculated according to the triangulation distance measurement method will generate an elevation error. Measure the elevation value of the camera deployment position and the elevation value of the calibration image position to correct the error. Use the map tool to measure the distance between the camera deployment position and the found point to obtain the calibration distance D ref , record the elevation value E of the camera deployment position base , the tilt angle T corresponding to the calibration image ref , the elevation value E of the calibration position ref ; S14. Calculate the pitch angle of the calibration position based on the calibrated distance D ref and the hanging height h of the camera The calculation formula is as follows: S15. According to the pitch angle corresponding to the known calibration position Calibrate the tilt angle T corresponding to the image ref and the tilt angle T of the image to be measured, and calculate the pitch angle of the position of the image to be measured. The calculation formula is as follows: By combining Equation (1) and Equation (2), we get: When correcting the pitch angle above, when the PTZ camera is deployed with a roll angle offset, this brings additional errors to the above correction. To avoid the increase of errors, roll angle correction is introduced.

3. The method for fusing terrain ranging of straw burning smoke images based on monocular PTZ according to claim 1 or 2, characterized in that, The step S2 includes the following steps: The first step, obtain the digital elevation model image (.tif) files within the range around the camera deployment location from the map source without offset, "Tianditu Map Source". The projection method of these images uses the longitude and latitude projection of the WGS84 coordinate system. In this way, it is ensured that the obtained images are consistent with the actual geographical locations, providing accurate basic data for subsequent processing and analysis; The second step, convert the offset longitude and latitude coordinates of the camera deployment location into the longitude and latitude coordinates in the longitude and latitude projection coordinate system of the non-offset WGS84 coordinate system through the projection coordinate system; The third step, load and parse the elevation data image file, mark it in the figure according to the longitude and latitude coordinates of the camera in the longitude and latitude projection coordinate system of the non-offset WGS84 coordinate system, obtain the contour map of the range around the camera deployment location as the center, and observe the degree of elevation change around; The fourth step, obtain the map azimuth angle of the line connecting the camera and the observation target when the camera is aligned with the observation target according to the initial azimuth angle of the camera and the P parameter, and draw the straight line where the line connecting the camera and the observation target is located on the contour map according to the map azimuth angle and the longitude and latitude coordinates of the camera in the longitude and latitude projection coordinate system of the non-offset WGS84 coordinate system; The fifth step, extract the elevation values on the map passed by the straight line according to the relevant information of the straight line, and at the same time convert the geographical longitude and latitude information corresponding to the sampling points on the straight line into the geographical distance information between the sampling points on the straight line and the camera deployment point, and construct the correlation curve of the elevation values corresponding to the sampling points on the straight line with respect to the distance values from the sampling points on the straight line to the camera deployment point; The sixth step, draw the model solution diagram of the triangulation ranging method on the correlation curve of the elevation values corresponding to the sampling points on the straight line with respect to the distance values from the sampling points on the straight line to the camera deployment point, including drawing the camera position point according to the hanging height of the camera deployment point, drawing the position point of the observation target according to the observed target distance value solved by the plane solution model, and drawing the optical path connection line between the camera position point and the observation target position point; The seventh step, use the interpolation method to calculate the intersection point of the straight line where the optical path is located and the elevation curve, and this intersection point is the distance correction value integrating the natural terrain elevation information.

Images