A forest fire point positioning method combining a tianditu map and a mobile phone image

Abstract

The application discloses a forest fire point positioning method combining a map and a mobile phone image, and solves the problem that the existing technology cannot realize accurate positioning of a forest fire point in the case that the collected forest photo is poor in definition. The application comprises the following steps: obtaining multiple map landmark point coordinates based on a smart phone with a map positioning function and a sensor, and shooting a forest fire point image based on the map landmark point coordinates and accurately acquiring spatial multi-pose information of the mobile forest fire point image; and performing mobile progressive fine positioning on the forest fire point in multiple forest fire point images which are shot, to obtain forest fire point positioning coordinates. The application is used for forest fire point positioning.

Description

TECHNICAL FIELD

[0001] The application relates to a forest fire point positioning method combining a satellite map and a mobile phone image, and belongs to the field of forest fire emergency response. BACKGROUND

[0002] Forests are an important part of the terrestrial ecosystem, and play a significant role in maintaining ecological balance, improving the ecological environment and preserving biodiversity. However, in recent years, forest fires have occurred frequently due to factors such as global warming, interplanting of forests and farmlands and frequent forest tourism activities. Forest fires often occur suddenly, are difficult to predict, are highly dangerous and difficult to control. If the fires are not discovered and controlled in time, they can spread rapidly and cause incalculable losses. Therefore, it is an important issue in forest fire prevention and monitoring to effectively, accurately and quickly position a forest fire point and accurately provide the geographic coordinates of the fire point at the first time of a forest fire, so as to assist fire prevention and command departments in making timely and correct fire fighting decisions and reducing personnel casualties and economic losses caused by forest fires.

[0003] At present, forest fire monitoring methods in China can be divided into four levels according to spatial position, i.e. satellite monitoring, aerial monitoring, near-ground observation and ground patrol, and a three-dimensional forest fire monitoring system has been basically formed. Satellite monitoring has the characteristics of wide monitoring range, high frequency and 24-hour all-weather monitoring, but the satellite remote sensing image has low time and spatial resolution, the deviation between the positioned fire point and the actual fire point reaches the kilometer level, and the entire calculation and processing process takes about half an hour, which is poor in real-time performance and can easily delay the best opportunity for forest fire fighting. Aerial monitoring includes airplane patrol and unmanned aerial vehicle patrol, and has the characteristics of wide investigation range, wide field of view and high flexibility, but airplane patrol is expensive, and some forest protection stations do not have the flight conditions, and unmanned aerial vehicles have the problems of weak carrying capacity and flight easily affected by the environment or the fire site. Near-ground observation mainly includes manual observation and forest area monitoring video positioning, but the monitoring and observation tower points are fixed, the theoretical monitoring range is limited (10-15 km), the construction price is high, and the monitoring is limited by the terrain and the topography, and there are monitoring dead angles and blanks, so it is difficult to achieve full coverage of the forest area. The ground patrol personnel have large patrol range and strong mobility, and can flexibly and dynamically enter the hinterland of the forest area to expand the patrol range, but when the forest fire is large or the personnel cannot reach the fire site, the forest protection personnel cannot accurately provide the position of the fire under the condition of long distance.

[0004] With the rapid development of computer communication, multimedia technology and positioning technology, smart phones with built-in GPS and various sensors are rapidly popularized worldwide, and a large number of research and application in the field of surveying and mapping geographic information are developing towards popularization with the help of smart phones. However, in the prior art, the accuracy of the sensor of the smart phone is not high, and the accuracy and quality of the measured parameters and digital images are lower than those of professional measuring equipment. Although the image collected by the smart phone is portable, the clearness of the photo collected by the smart phone is poor, and the precise positioning of the forest fire point cannot be realized, so the smart phone is not used to realize the positioning of the forest fire point in the prior art.

[0005] Therefore, the prior art has the following technical problems:

[0006] 1. In the case that the clearness of the collected forest photo is poor, the precise positioning of the forest fire point cannot be realized.

[0007] 2. The mobility and portability of the measuring equipment are poor. SUMMARY

[0008] The purpose of the present application is to provide a forest fire point positioning method combining Tianditu and mobile phone images, which solves the problem that the precise positioning of the forest fire point cannot be realized in the case that the clearness of the collected forest photo is poor in the prior art.

[0009] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0010] A forest fire point positioning method combining Tianditu and mobile phone images, comprising the following steps:

[0011] Step S1. Obtain a plurality of Tianditu landmark point coordinates based on a smart phone with a Tianditu positioning function and a sensor, and shoot forest fire point images based on the Tianditu landmark point coordinates, while accurately obtaining spatial multi-pose information of the mobile shooting of the forest fire point images;

[0012] Step S2. Based on the accurately obtained spatial multi-pose information, the forest fire point in the shot multiple forest fire point images is subjected to mobile progressive refinement positioning, and the forest fire point positioning coordinates are obtained.

[0013] Further, the specific steps of step S1 are as follows:

[0014] Step S1.1. Based on spatial semantic constraints, a plurality of camera station positions are screened in combination with Tianditu and real environment, and Tianditu landmark point coordinates are obtained, and forest fire point images are shot on the Tianditu landmark point coordinates in turn based on a multi-pose shooting model, the overlap degree of the shot forest fire point images is 60-80%, and one point in the forest fire area of the forest fire point image is selected as an important feature point based on the first shot forest fire point image.

[0015] Step S1.2. Obtain the mobile shooting parameters of the smartphone when shooting the forest fire point image;

[0016] Step S1.3. Correct the positioning result of the smartphone based on the mobile shooting parameters and the coordinates of the landmark points on the Tianditu map, that is, optimize the attitude information of the smartphone through multi-pose shooting to achieve accurate acquisition of the pose information of the mobile shooting forest fire point image.

[0017] Further, the specific steps of step S3.1 are:

[0018] Step S1.11. First, locate the real-time position of the forest ranger on the Tianditu map;

[0019] Step S1.12. Based on the real-time position of the forest ranger located on the Tianditu map, the spatial semantic constraints, and the real environment, select multiple landmark features as shooting points to obtain the screening of multiple camera station positions, that is, obtain multiple landmark points, wherein the spatial semantic constraints refer to the heterogeneity features between the landmark features in the Tianditu image and the features outside the landmark features, and the heterogeneity features include regional, visibility, and landmark relationships;

[0020] Step S1.13. Obtain the coordinates of the multiple landmark points on the Tianditu map based on the screened multiple landmark points, and obtain the Tianditu landmark point coordinates, and based on the multi-pose shooting model, sequentially shoot the forest fire point image at the Tianditu landmark point coordinates, wherein the multi-pose shooting model refers to a combination of vertical and horizontal shooting for the same fire area.

[0021] Further, the specific steps of step S1.2 are:

[0022] The mobile shooting parameters of the smartphone include internal orientation elements and external orientation elements;

[0023] Internal orientation elements:

[0024] The internal orientation elements are determined by the smartphone and are parameters that describe the relevant positions between the shooting center and the forest fire point image, including three parameters: the vertical distance f of the shooting center S to the image, and the coordinates (x0, y0) of the image principal point o in the frame coordinate system, wherein the vertical distance refers to the principal distance;

[0025] External orientation elements:

[0026] The external orientation elements of the image refer to the spatial position and attitude parameters of the forest fire point image during shooting, and each forest fire point image has six external orientation elements, which are three line elements, that is, the coordinates X S , Y S , Z S of the shooting center S in the object space rectangular coordinate system, and three angle elements that describe the attitude information of the image during shooting, that is, the heading angle The pitch angle ω, the roll angle κ, and the exterior orientation elements of the image are provided by the smart phone, the line elements of the image are obtained using a location service, and the angle elements of the photo are obtained by combining the return values of the acceleration sensor, the magnetic field sensor, and the direction sensor;

[0027] In the Android phone, the output results of the sensors are all referenced to the local coordinate system of the smart phone, and the smart phone coordinate system is a relative coordinate system defined with the phone screen as the reference. The origin of the inertial coordinate system coincides with the origin of the smart phone coordinate system, and the coordinate axes of the inertial coordinate system are parallel to the coordinate axes of the world coordinate system, that is, the inertial coordinate system is regarded as an intermediate state between the smart phone coordinate system and the world coordinate system. Therefore, the inertial coordinate system is needed to complete the conversion of the smart phone from the local coordinate system to the world coordinate system. The conversion formula of the smart phone coordinate system to the world coordinate system is as follows:

[0028] Rotation around the z-axis, with a rotation angle of The resulting rotation matrix is:

[0029]

[0030] Rotation around the x-axis, with a rotation angle of ω, the resulting rotation matrix is:

[0031]

[0032] Rotation around the y-axis, with a rotation angle of κ, the resulting rotation matrix is:

[0033]

[0034] The three basic rotation sequences are combined in different rotation orders to obtain the rotation matrix between the two coordinate systems. The rotation orders include any one of z-x-y, z-y-x, x-z-y, x-y-z, y-z-x, and y-x-z. When the rotation sequence is z-x-y, the rotation matrix is:

[0035]

[0036] Therefore, the rotation relationship of the smart phone coordinate system to the world coordinate system is:

[0037]

[0038] where (x', y', z') is the three-dimensional coordinate of a point in the world coordinate system, (x, y, z) is the three-dimensional coordinate of a point in the smart phone coordinate system, and T represents transposition.

[0039] Further, the specific steps of the step S1.3 are:

[0040] Step S1.31. Position optimization

[0041] Firstly, define the plane coordinates of multiple shooting positions obtained by the smart phone as S1(lon1, lat1), S2(lon2, lat2), S3(lon3, lat3), …, and the plane coordinates of the shooting positions obtained by manual selection of the map are the map landmark point coordinates S'1(Lon1, Lat1), S'2(Lon2, Lat2), S'3(Lon3, Lat3), …;

[0042] Then, the difference between the smart phone positioning in each shooting position coordinate data and the map landmark point coordinate is calculated as the correction number;

[0043] Finally, the arithmetic mean values of the longitude and latitude correction numbers Δlon and Δlat are calculated as the final correction number of the mobile phone positioning:

[0044]

[0045] Wherein, N * refers to a non-zero natural integer;

[0046] The geodetic coordinates of the corrected corresponding shooting position are obtained by adding the smart phone shooting position coordinate data and the final correction number, that is, the space parameters of the optimized exterior orientation elements, and then the geodetic coordinates are converted into space rectangular coordinates according to the following formula:

[0047]

[0048] Wherein, e1 is the first eccentricity, and N is the curvature radius of the equinoctial circle;

[0049] Step S1.32. Attitude optimization

[0050] Obtain vertical and horizontal shooting forest fire point images of the same shooting position and the same fire area, remove the theoretical angle difference of two shooting, and take the average value as the final attitude angle to realize the optimization of the attitude parameter, that is, when the vertical shooting is converted into horizontal shooting, the roll angle and the pitch angle should be different by 90 degrees, so after removing the 90 degree difference, the average value of the two is obtained. Get the optimized attitude parameter;

[0051] Step S1.33. Precise acquisition of the pose information of the mobile shooting forest fire point image based on positioning optimization and attitude optimization.

[0052] Further, the specific steps of the step S2 are:

[0053] Step S2.1. Based on the SI FT algorithm and the RANSAC algorithm, the same forest fire point position in the forest fire point image shot at multiple shooting positions is identified, and the forest fire point coordinates are obtained;

[0054] Step S2.2. Obtain the coordinates of the object points corresponding to the homonymous image points of the forest fire points in the two forest fire point images based on the forest fire point coordinates and the forest fire locating algorithm of the double images.

[0055] Step 2.3. Obtain the forest fire point coordinates locating by moving and gradually refining the coordinates of the object points corresponding to the homonymous image points of the forest fire points in the multiple sets of forest fire point images based on the multiple forest fire point image combination.

[0056] Further, the specific steps of the step S2.1 are:

[0057] Step S2.11. Preprocess each forest fire point image, that is, correct the distortion caused by the wide-angle lens when the smartphone captures the forest fire point image;

[0058] Step S2.12. Identify the feature points in each forest fire point image obtained after preprocessing by using the SIFT algorithm, and perform coarse matching on the two forest fire point images adjacent in shooting time, wherein the forest fire point image captured earlier is taken as the reference image, and the forest fire point image captured later is taken as the image to be matched;

[0059] Step S2.13. Perform field voting denoising on the feature points after coarse matching, and obtain the initial inlier set after denoising;

[0060] Step S2.14. Screen the initial inlier set obtained by coarse matching based on the improved RANSAC algorithm, obtain the accurate and error-free matching points of 6 pairs or more, that is, obtain the corresponding relationship of the feature points of the first forest fire point image as the reference image and the second forest fire point image as the image to be matched, transfer the coordinates of the feature points of the first forest fire point image to the second forest fire point image, and then transfer the second forest fire point image as the reference image to the third forest fire point image, and so on. If the important feature points are contained in the matching points of 6 pairs or more, go to step S2.15, otherwise, based on the important feature points, establish the corresponding relationship of the feature points of each forest fire point image, and then go to step S2.15;

[0061] Step S2.15. Based on the corresponding relationship of the feature points of each forest fire point image, calculate the forest fire point coordinates on the forest fire point images captured subsequently according to the forest fire point manually selected when the first forest fire point image is captured.

[0062] Further, the specific steps of the step S2.12 are:

[0063] Firstly, remove the noise in each forest fire image by Gaussian blur, create multi-scale images, and create multi-scale space based on scale images and Gaussian difference enhanced image features, that is, based on multi-scale images to form Gaussian pyramid of each forest fire image, subtract the pixel points of the same group of adjacent two layers in the Gaussian pyramid to obtain the Gaussian difference pyramid, that is, the multi-scale space:

[0064] L(x,y,σ)=G(x,y,σ)×I(x,y) (8)

[0065]

[0066] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))×I(x,y) (10)

[0067] Wherein, I(x,y) is the two-dimensional image of each forest fire image to be detected; L(x,y,σ) is the Gaussian scale space or Gaussian pyramid or Gaussian image of the forest fire image; G(x,y,σ) is the Gaussian function; σ is the scale space factor, k is the multiple of adjacent scale space, D(x,y,σ) refers to the Gaussian difference pyramid, that is, the multi-scale space;

[0068] Secondly, compare each pixel value of the Gaussian image in the multi-scale space with the surrounding 26 pixel values, if the pixel is the highest or lowest pixel among the adjacent pixels, it is considered that the point is a candidate feature point of each forest fire image in the scale space;

[0069] Thirdly, calculate the second-order Taylor expansion of each candidate feature point in the scale space, and consider that the point belongs to a low-contrast feature point if the result is less than a threshold value, and remove it to obtain a feature point;

[0070] Then, use the gradient direction distribution characteristics of the neighborhood pixels of the removed feature points to specify the direction parameter for each feature point, and count the gradient direction and its gradient amplitude of all pixels in the circle with the feature point as the center and 1.5 times the scale of the Gaussian image in the multi-scale space where the feature point is located as the radius, to create a gradient histogram, wherein the peak value of the histogram represents the main direction of the neighborhood gradient at the key point, that is, the direction of the feature point, wherein the direction reaching 80% of the maximum value is taken as the auxiliary direction;

[0071] Finally, generate a unique fingerprint for the feature point through the main direction, auxiliary direction and size of the adjacent pixels, called feature point descriptor, calculate the distance between each feature point descriptor in the to-be-matched image and each feature point descriptor in the reference image, sort all the results corresponding to each feature point descriptor, and take the nearest distance as the matching point, that is, obtain the result after rough matching; The specific steps of step S2.13 are:

[0072] After the coarse matching, the distance d and the difference of the main direction angle Δθ of any two feature points in each forest fire point image are calculated respectively, and then the distance d and the difference Δθ are normalized according to the row vector. After the normalization, the distance inner product value d ot1 and the main direction angle inner product value d ot2 of each pair of matching points between the reference image and the image to be matched are calculated. Finally, whether the distance inner product value and the main direction angle inner product value of a pair of matching points are less than the threshold value set in advance is compared. If yes, the matching points are put into the inlier set, that is, the initial inlier set is obtained. Otherwise, the matching points are put into the outlier set.

[0073]

[0074] Δθ = θ i - θ j (12)

[0075] d ot1 = d ot (im1(x U , y U ), im2(x u , y u ))

[0076] d ot2 = d ot (im1(θ U ), im2(θ u ))

[0077] Wherein, i, j are any pair of initial matching points in the same forest fire point image, (x U , y U , θ U ) and (x u , y u , θ u ) represent the pixel coordinates and the main direction of the corresponding matching points U and u of the reference image im1 and the image to be matched im2, d ot is the inner product of the reference image im1 and the image to be matched im2, θ i , θ j respectively represent the main direction of any two feature points i and j in each forest fire point image; the specific steps of the step S2.14 are:

[0078] First, four non-collinear sample data are randomly extracted from the feature point set obtained based on the SIFT algorithm, and a 3×3 transformation matrix H is calculated, denoted as model M. Then, the projection error between all data and model M is calculated. If it is less than the threshold, it is added to the initial inlier set. When the number of elements in the initial inlier set Q is greater than the optimal inlier set Q-best, Q-best is updated to Q. At the same time, it is determined whether the number of iterations is greater than the number of iterations K. If it is, the process is terminated. Otherwise, the number of iterations is incremented by 1, and the above operation is repeated until the iteration ends, so as to remove abnormal data and obtain accurate matching points, that is, to obtain the correspondence between the feature points of each reference image and the corresponding image to be matched.

[0079]

[0080] Where (x, y) refers to the position of the feature point in the image to be matched; 's' represents the location of feature points in the reference image; 's' is the scale parameter. H is the transformation matrix;

[0081] The correspondence between the feature points of each reference image and the corresponding image to be matched is obtained, that is, the location of the same forest fire point in each forest fire point image is identified, and the coordinates of the forest fire point in each forest fire point image are obtained.

[0082] Furthermore, the specific steps of step S2.2 are as follows:

[0083] Based on points S1 and S2, two images (left and right) with an overlap of over 60% were captured from the same forest area. S1 and S2 were used as the shooting baselines, with S1o1 as the principal optical axis of the left shooting station and S2o2 as the principal optical axis of the right shooting station. The forest fire point P was visualized as p1 and p2 in the left and right images, respectively. A self-developed 3D electronic compass was used to measure the heading angles of the two smartphones at the moment of image capture in real time. Pitch angle ω, roll angle κ, and simultaneously accurately determine the coordinates (X) of the two shooting centers S1 and S2. S Y S Z S ), where the heading angle This refers to the angle between the principal optical axis and the due north direction. For imaging p1 and p2, the angles between the principal optical axis and the due north direction are respectively... The pitch angle ω refers to the angle between the axis and the vertical plane. When configuring p1 and p2, the angles between the axis and the vertical plane are ω1 and ω2, respectively. The roll angle κ refers to the angle between the axis and the horizontal plane. When configuring p1 and p2, the angles between the axis and the horizontal plane are κ1 and κ2, respectively.

[0084] The shooting center, the image point, and the object point, i.e. the forest fire point, satisfy the collineation equation, and form a light beam. The collineation equation is obtained based on the rotation relationship from the smart phone coordinate system to the world coordinate system, and the double image analysis is performed by using the light beam method. The three-dimensional attitude angle at the shooting moment is obtained by the three-dimensional electronic compass as the initial value for the optimization of the spatial position and the attitude parameter, i.e. the accurate acquisition of the position and posture information of the mobile shooting forest fire point image. After the optimization, all the homonymous image points and the corresponding object points, i.e. the forest fire point coordinates, of the left and right forest fire point images are listed in the error equation according to the collineation equation:

[0085]

[0086] In the formula, (x L , y L ), (x R , y R ) are the image plane coordinates of the homonymous image points p1 and p2 of the forest fire point P on the left and right two images, which are directly obtained from the left and right forest fire point images after the forest fire point image matching, (x L0 , y L0 , f L ), (x R0 , y R0 , f R ) are the internal orientation elements of the left and right cameras; are the coordinates of the shooting center points S1 and S2 of the left and right camera stations in the shooting measurement space rectangular coordinate system and , which are obtained by the mobile terminal GPS positioning; (a L1 , b L1 , c L1 ), (a L2 , b L2 , c L2 ), (a L3 , b L3 , c L3 ) represent the parameters corresponding to the first row, the second row, and the third row of the rotation matrix R1 after the optimization of the spatial position and the attitude parameter of the left image, (a R1 , b R1 , c R1 ), (a R2 , b R2 , c R2 ), (a R3 , b R3 , c R3 ) represent the parameters corresponding to the first row, the second row, and the third row of the rotation matrix R2 after the optimization of the spatial position and the attitude parameter of the right image, and the rotation matrix R1 and the rotation matrix R2 are obtained based on the rotation matrix R; and The coordinates of the corresponding points in the photographing measurement coordinate system for the left and right homonymous image points The homonymous image points are measured, and the three-dimensional attitude angle of the left and right forest fire point images and the coordinates of the corresponding points are solved according to the least square adjustment principle, wherein the coordinates (X, Y, Z) of the corresponding points of the homonymous image points of the forest fire point are the solution.

[0087] Further, the specific steps of the step S2.3 are as follows:

[0088] A set of coordinates (X, Y, Z) of the corresponding points of the homonymous image points of the forest fire point is obtained based on the beam method and the analysis of the two images, and then m sets of positioning coordinates of the forest fire point are obtained from n forest fire point images, wherein n is greater than or equal to 3:

[0089]

[0090] The centroid of each set of the m sets of forest fire points is solved as an estimated value of the coordinates of the forest fire point P, and the coordinates of the ith point are assumed to be (X i , Y i , Z i ), and the coordinates of the centroid are:

[0091]

[0092] The finally solved (X P , Y P , Z P ) is the positioning coordinates of the forest fire point.

[0093] Compared with the prior art, the advantages of the present application are as follows:

[0094] 1. The present application improves the mobility and portability of the measuring equipment, reduces the use cost and complexity, and compared with the prior art, the measurement and parameter acquisition of the image are mostly carried out by professional measuring equipment, while the present application is developed based on the Android system and can be directly used by forest rangers based on mobile smart phones.

[0095] 2. The present application makes full use of the built-in Tianditu platform of the terminal carried by the forest ranger, provides a spatial position reference and data base map for the forest fire point positioning work, improves the single-point positioning accuracy based on the mobile phone, and the interactive correction method can further improve the forest fire point positioning accuracy.

[0096] 3. The present application proposes a progressive refinement positioning method of the forest fire point combined with multiple images to improve the positioning accuracy of the forest fire point.

[0097] ​Fourth, the application makes full use of mobile phone positioning service and built-in sensor to provide shooting pose information, combines with the spatial position reference and data base map of map world, and constructs a forest fire point mobile positioning method combined with map world and smart phone image, to provide relatively accurate information support for emergency rescue, that is, using the mobile phone built-in national geographic information public service platform map world equipped by the ground patrol member, each parameter of mobile phone positioning can be corrected, and multiple photos are combined to make up for the gap of shooting quality, in the state of emergency response, the coordinates of the fire point are quickly and accurately obtained by using only the mobile phone, the purpose of quickly and accurately positioning the forest fire point when the convenience and shooting clarity are poor is met, at present, the positioning accuracy of satellite remote sensing forest fire is low, and the deviation reaches kilometers, the mobile phone shooting is used for long distance positioning in the case, and the deviation should be in the order of hundreds of meters. BRIEF DESCRIPTION OF DRAWINGS

[0098] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0099] Figure 1 It is a schematic diagram of the framework of the application;

[0100] Figure 2 It is a schematic diagram of the forest fire point positioning based on mobile shooting in the application;

[0101] Figure 3 It is a schematic diagram of the accurate framework of the mobile shooting forest fire point image space pose information based on map world in the application, wherein, from left to right, the points near the curved route in the space semantic constraint map world mark point positioning are shooting point 1, shooting point 2 and shooting point 3, and the points in the three images of the smart phone based mark point positioning are shooting point 1, shooting point 2 and shooting point 3 from left to right;

[0102] Figure 4 It is a schematic diagram of the shooting position screening combined with map world and real environment in the application;

[0103] Figure 5 It is Figure 3 a schematic diagram of multi-pose shooting in the application;

[0104] Figure 6 It is a schematic diagram of the forest fire point mobile progressive refinement positioning based on the light beam method double image analysis in the application, wherein, the point in the picture in the forest fire point annotation interactive correction is the forest fire point position;

[0105] Figure 7The schematic diagram for automatic matching of forest fire point image feature points in the present application;

[0106] Figure 8 For Figure 1 The schematic diagram for coordinate calculation of forest fire points by means of light beam method double image analysis;

[0107] Figure 9 For Figure 3 The schematic diagram for obtaining attitude angle information at the moment of photography (shooting moment);

[0108] Figure 10 For Figure 6 The schematic diagram for coordinate calculation. DETAILED DESCRIPTION

[0109] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the scope of protection of the present application.

[0110] First, the landmark feature points suitable for positioning and rectification are screened in combination with the map image and the real scene; then the mobile phone positioning result is corrected in combination with the existing positioning information of the map, the mobile phone attitude information is optimized through multi-attitude shooting, and the accurate acquisition of mobile shooting attitude information is realized; finally, the fire point positions on different images are rectified through multiple mobile shooting forest fire images (forest fire point images), the forest fire point positioning is carried out based on the light beam method double image analysis in combination with the multiple forest fire point images, and the purpose of improving the mobile shooting positioning accuracy of forest fire points is achieved.

[0111] The accurate acquisition of mobile shooting image space attitude information based on the map, that is, obtaining multiple map landmark point coordinates based on the intelligent mobile phone with map positioning function and sensor, and shooting forest fire point images based on the map landmark point coordinates, while accurately acquiring the space multi-attitude information of the mobile shooting forest fire point images:

[0112] The positioning accuracy of a mobile phone navigation module is not high, and generally only road navigation and relevant information query can be provided, and accurate positioning cannot be realized. The spatial attitude information accuracy of a mobile shooting target positioning method directly based on mobile phone positioning service and sensor parameters is low. Therefore, the present application considers regional, visibility and landmark spatial semantic constraints to screen landmark points, corrects the mobile phone positioning result in combination with the existing positioning information of the map world, and optimizes the mobile phone attitude information through multi-attitude shooting, so as to realize accurate acquisition of the mobile shooting image position information. The method comprises three parts. First, the shooting station position screening in combination with the map world and the real environment, that is, based on spatial semantic constraints, the multiple shooting station positions are screened in combination with the map world and the real environment, and the map world landmark point coordinates are obtained. The forest fire point image is shot on the map world landmark point coordinates in sequence based on the multi-attitude shooting model, and the overlap of the shot forest fire point image is 60-80%. Second, the mobile phone shooting image basic parameter acquisition, that is, the mobile shooting parameters of the intelligent mobile phone when shooting the forest fire point image are acquired. Third, the image position parameter optimization based on the map world, that is, the intelligent mobile phone positioning result is corrected based on the mobile shooting parameters and the map world landmark point coordinates, that is, the mobile phone attitude information is optimized through multi-attitude shooting, so as to realize accurate acquisition of the mobile shooting forest fire point image position information.

[0113] Screening of the shooting station position in combination with the map world and the real environment:

[0114] Before screening the shooting station position, the heterogeneity characteristics between the landmark features and other features in the map world image need to be analyzed in depth, the regional, visibility and landmark spatial semantic constraints are considered, and the shooting station position is screened in combination with the map world and the real environment. The regionality means that the shooting station should be located in the vicinity of the position of the forest ranger, and the distance between adjacent landmark points is preferably 30-200 m. The visibility means that the selected shooting station position should be able to shoot the fire scene. The landmark means that the feature has strong distinguishing characteristics in the map world and the real environment, such as road corner.

[0115] Firstly, the real-time position of the forest ranger is acquired through the mobile phone positioning service and is positioned on the map world, and then more than three landmark features are manually selected as shooting points in combination with the feature characteristics of the real environment features and the image characteristics of the region in the map world (that is, the spatial semantic constraint characteristics), such as the road corner shown in Figure 4 .

[0116] After screening 3 or more than 3 mark points, the coordinates of the mark points on the map are obtained through the MarkTool provided by the map API, and the multi-pose shooting model is used to shoot the forest fire point image at the mark points in turn. The photograph overlap degree is required to be between 60-80%, and after shooting the first forest fire point image, a point in the forest fire area of the forest fire point image is manually selected as an important feature point. The multi-pose shooting model refers to a combination of vertical and horizontal shooting of the same fire area.

[0117] Mobile phone shooting photograph basic parameter acquisition:

[0118] The mobile shooting photograph parameters based on the smart phone mainly include two parts: internal orientation elements and external orientation elements.

[0119] 1) Internal orientation elements

[0120] The internal orientation elements are determined by the smart phone, which are parameters describing the relative positions between the shooting center and the forest fire point image, including three parameters: the vertical distance f of the shooting center S to the image, and the coordinates (x0, y0) of the image principal point o in the frame coordinate system. The vertical distance refers to the principal distance.

[0121] Due to the manufacturing precision and assembly process of the camera lens of the mobile phone, the image shot by the mobile phone has distortion. Therefore, the camera of the mobile phone needs to be calibrated according to a group of points with known spatial positions and their corresponding points on the image before the image is acquired, and the internal parameter matrix of the camera is obtained through the conversion of the coordinate system. First, more than 10 chessboard photos shot at different angles are collected and preprocessed; then, the chessboard information is found out and the sub-pixel corner point information is further extracted through the findChessboardCorners function and the cornerSubPix function of openCV; finally, the camera internal parameter matrix and distortion coefficient are calculated through the calibrateCamera function to calibrate the image. Through calibration, the distortion can be improved, and the relationship between the natural unit of the camera and the actual unit is determined, so that after calibration, the size of the object in the image is known.

[0122] 2) External orientation elements

[0123] The external orientation elements of the image refer to the spatial position and pose parameters of the forest fire point image when shooting. Each forest fire point image has six external orientation elements, which are three line elements, i.e. the coordinates X S , Y S , Z S of the shooting center S in the object space rectangular coordinate system, and three angle elements describing the pose information of the image when shooting, i.e. the heading angle The pitch angle ω, the roll angle κ, and the exterior orientation elements of the image are provided by the smart phone, the line elements of the image are obtained using a location service, and the angle elements of the photo are obtained by combining the return values of the acceleration sensor, the magnetic field sensor, and the direction sensor;

[0124] In the Android phone, the output results of the sensors are all referenced to the local coordinate system of the smart phone, and the smart phone coordinate system is a relative coordinate system defined with the screen of the phone as the reference. The origin of the inertial coordinate system coincides with the origin of the smart phone coordinate system, and the coordinate axes of the inertial coordinate system are parallel to the coordinate axes of the world coordinate system, that is, regarded as an intermediate state between the smart phone coordinate system and the world coordinate system. Therefore, the inertial coordinate system is needed to complete the conversion of the smart phone from the local coordinate system to the world coordinate system. The conversion formula of the smart phone coordinate system to the world coordinate system is as follows:

[0125] The rotation matrix obtained by rotating around the z-axis (the rotation angle is ω): ) is:

[0126]

[0127] The rotation matrix obtained by rotating around the x-axis (the rotation angle is ω):

[0128]

[0129] The rotation matrix obtained by rotating around the y-axis (the rotation angle is κ):

[0130]

[0131] The three basic rotation sequences are combined in different rotation orders to obtain the rotation matrix between the two coordinate systems. The rotation order includes any one of z-x-y, z-y-x, x-z-y, x-y-z, y-z-x, and y-x-z. When the rotation sequence is z-x-y, the rotation matrix is:

[0132]

[0133] Therefore, the rotation relationship of the smart phone coordinate system to the world coordinate system is:

[0134]

[0135] Where (x', y', z') is the three-dimensional coordinates of the point in the world coordinate system, (x, y, z) is the three-dimensional coordinates of the point in the smart phone coordinate system, and T represents transposition.

[0136] Optimization of image pose parameters based on Map World:

[0137] 1) Positioning optimization

[0138] First, the planar coordinates of multiple shooting locations obtained through a smartphone are defined as S1(lon1, lat1), S2(lon2, lat2), S3(lon3, lat3), ..., and the planar coordinates of the shooting locations are manually selected through Tianditu, i.e., the coordinates of the Tianditu marker points are S′1(Lon1, Lat1), S′2(Lon2, Lat2), S′3(Lon3, Lat3), ...;

[0139] Then, the difference between the smartphone location and the coordinates of the Tianditu marker point in the coordinate data of each shooting location is calculated as the correction value;

[0140] Finally, the arithmetic mean of the longitude and latitude corrections, Δlon and Δlat, are calculated separately as the final correction values ​​for mobile phone positioning:

[0141]

[0142] Where, N * It refers to a non-zero natural integer;

[0143] The corrected geodetic coordinates of the corresponding shooting location are obtained by adding the smartphone's shooting location coordinates to the final correction value. This gives the spatial parameters of the optimized exterior orientation element. The geodetic coordinates can then be converted to spatial rectangular coordinates using the following formula:

[0144]

[0145] Where e1 is the first eccentricity and N is the radius of curvature of the zonal loop;

[0146] 2) Attitude optimization

[0147] Acquire vertical and horizontal images of the same fire area from the same shooting location (e.g., Figure 5 (As shown) Forest fire image, after removing the theoretical angle difference between the two shots, the average value is taken as the final attitude angle to optimize the attitude parameters. That is, when switching from vertical to horizontal shooting, the roll angle and pitch angle should differ by 90 degrees. Therefore, after removing this 90-degree difference, the average value of the two is taken to obtain the optimized attitude parameters.

[0148] Based on positioning optimization and attitude optimization, the pose information of forest fire images captured by mobile cameras is accurately obtained.

[0149] Forest fire hotspot location: Progressive refinement positioning

[0150] The same forest fire point target has different features in different images, and the positioning accuracy of forest fire positioning based on single-point or double-point mobile shooting is low, therefore, in order to further improve the positioning accuracy of the spatial position of the forest fire point, the present application provides a kind of forest fire point mobile progressive refinement positioning based on light beam method double image analysis. The method is divided into three parts, first, the image position recognition of the same fire point in multiple images, that is, the same forest fire point position in the forest fire point image photographed in multiple shooting positions is recognized based on the SIFT algorithm and the RANSAC algorithm, and the forest fire point coordinates are obtained;Second, the forest fire positioning algorithm based on light beam method double image analysis, that is, the corresponding object point coordinates of the same name image points of the forest fire point in two forest fire point images are obtained based on the forest fire point coordinates and the forest fire positioning algorithm of double images;Third, the forest fire point mobile progressive refinement positioning of multiple images, that is, the corresponding object point coordinates of multiple groups of the same name image points of the forest fire point are mobile progressive refinement positioned based on the multiple forest fire point images, and the forest fire point coordinate positioning is obtained.

[0151] The image position recognition of the same fire point in multiple images:

[0152] The SIFT algorithm has the characteristics of scale invariance, strong anti-interference ability and good robustness. The core of the RANSAC algorithm is to calculate other data from a set of observation data by iteration, and finally filter out incorrect data. However, the classic RANSAC algorithm needs to be iterated constantly, which will consume a lot of time, and the monitoring of the fire cannot achieve real-time. The present application combines the SIFT algorithm with the improved RANSAC algorithm, first pre-processes the picture, then uses the SIFT algorithm to identify the feature points for rough matching, and then uses the improved RANSAC algorithm to screen the feature points, to obtain accurate matching points.

[0153] The implementation of the SIFT algorithm specifically includes four steps: creating a scale space, candidate feature point detection and accurate positioning, direction assignment, and constructing a feature point descriptor. First, remove noise from the image by Gaussian blur, then create multiple scale images, and enhance image features based on scale images and Gaussian difference to create a multi-scale space (equations 8, 9, 10); second, compare each pixel value in the Gaussian image in the multi-scale space with the values of the surrounding 26 pixels. If the pixel is the highest or lowest pixel among the adjacent pixels, it is considered a candidate feature point of the image at that scale. Then, calculate the second-order Taylor expansion of the scale space for each candidate feature point. If the result is less than the threshold, it is considered a low-contrast feature point and is removed. After removal, the feature points are obtained. Then, use the gradient direction distribution characteristics of the feature point neighborhood pixels to assign a direction parameter to each key point. Count all the gradient directions and their gradient amplitudes within a circle centered at the feature point and with a radius of 1.5 times the scale of the Gaussian image where the feature point is located to create a gradient histogram. The peak value of the histogram represents the main direction of the neighborhood gradient at the feature point, which is the direction of the feature point. Other directions that reach 80% of the maximum value can be used as auxiliary directions. Finally, a unique fingerprint, called "feature point descriptor," is generated for each key point by using the gradient directions of the adjacent pixels (including the main direction and the auxiliary direction) and their sizes. By calculating the distances between the feature point descriptors in the to-be-matched image and the feature point descriptors in the reference image, and sorting all the results obtained by each feature point descriptor, the closest distance is taken as the matching point, i.e., the result after coarse matching is obtained.

[0154] L(x,y,σ) = G(x,y,σ) x I(x,y) (8)

[0155]

[0156] D(x,y,σ) = (G(x,y,kσ) - G(x,y,σ)) x I(x,y) (10)

[0157] where I(x,y) is the two-dimensional image of each forest fire point image to be detected; L(x,y,σ) is the Gaussian scale space or Gaussian pyramid or Gaussian image of the forest fire point image; G(x,y,σ) is the Gaussian function; σ is the scale space factor, k is the multiple of the adjacent scale space, D(x,y,σ) refers to the Gaussian difference pyramid, i.e., the multi-scale space, and exp represents the exponential function with the natural number e as the base;

[0158] After obtaining the initial features using the SIFT algorithm, the feature points obtained after coarse matching are used for domain voting to filter out some incorrect matching points, and the final sample point set is obtained to reduce the number of iterations of the RANSAC algorithm.

[0159] The field voting first needs to calculate the distance d (formula 11) and the difference of the main direction angle Δθ (formula 12) of any two feature points in each forest fire point image, and then normalizes them according to the row vector. After the normalization, the distance inner product d ot1 and the main direction angle inner product value d ot2 are calculated. Finally, whether the distance inner product value and the main direction angle inner product value of a pair of matching points are less than the threshold value is compared through the threshold value set in advance (a large number of experiments have proved that when the distance threshold value td and the direction threshold value tθ are set to 0.4 and 0.5 respectively, a relatively large number of inner point sets can be obtained). If yes, it is put into the inner point set, otherwise it is put into the outer point set.

[0160]

[0161] Δθ = θ i - θ j (12)

[0162] d ot1 = d ot (im1(x U , y U ), im2(x u , y u ))

[0163] d ot2 = d ot (im1(θ U ), im2(θ u ))

[0164] Wherein, i, j are any pair of initial matching points in the same forest fire point image, (x U , y U , θ U ) and (x u , y u , θ u ) represent the pixel coordinates and the main direction of the corresponding matching points U and u of the reference image im1 and the matching image im2, d ot is the inner product of the reference image im1 and the matching image im2, θ i , θ j respectively represent the main direction of any two feature points i and j in each forest fire point image.

[0165] The final sample point set is obtained by screening out some error matching points through field voting, so as to reduce the iteration number of the RANSAC algorithm. The implementation of the RANSAC algorithm specifically includes three steps, i.e. calculation of a transformation matrix, calculation of a projection error and an iteration process. Firstly, 4 sample data not in a same line are randomly extracted from a known data set (a feature point set obtained based on the SIFT algorithm), a 3*3 transformation matrix H (formula 13) is calculated, denoted as a model M, then a projection error of all data and the model M is calculated, if the projection error is less than a threshold, the data is added to an inner point set, when the number of elements in the point set Q is greater than an optimal inner point set Q-best, the Q-best is updated as Q, and the iteration number K is updated, finally if the iteration number is greater than K, the iteration is exited, otherwise the iteration number is increased by 1, and the above operation is repeated until the iteration is ended, so as to remove the abnormal data and obtain accurate matching points, i.e. to obtain the corresponding relationship between the feature points of each reference image and the corresponding to-be-matched image.

[0166]

[0167] Wherein, (x, y) is the position of the feature point of the to-be-matched image; is the position of the feature point of the reference image; s is a scale parameter, is a transformation matrix H;

[0168] The initial inner point set obtained through the rough matching is screened based on the improved RANSAC algorithm, so as to obtain accurate matching points of 6 pairs or more, i.e. to obtain the corresponding relationship between the feature points of the first forest fire point image as a reference image and the second forest fire point image as a to-be-matched image, the coordinates of the feature points of the first forest fire point image are transmitted to the second forest fire point image, then the second forest fire point image is transmitted to the third forest fire point image as a reference image, and so on. If the 6 pairs or more of matching points contain important feature points, the next step is performed, otherwise, the corresponding relationship between the feature points of each forest fire point image is established based on the important feature points (the epipolar line of the important feature point on the to-be-matched image can be obtained based on the fundamental matrix between the reference image and the to-be-matched image according to the matching points, and the specific position of the forest fire point on the to-be-matched image can be searched by performing block matching along the epipolar line), and then the next step is performed.

[0169] After the corresponding relationship between the feature points of the reference image and the to-be-matched image is obtained, the forest fire point image coordinates on the other to-be-matched images can be calculated according to the forest fire points manually selected when the first image is shot.

[0170] Forest fire positioning algorithm based on double images:

[0171] As Figure 8As shown, based on S1, S2 points, the same forest area is photographed to obtain two images with an overlap of more than 60%, left and right, with S1S2 as the shooting baseline, S1o1 as the main optical axis of the left shooting station, S2o2 as the main optical axis of the right shooting station, and the forest fire point P in the left and right images as p1 and p2, respectively. The heading angle of the two intelligent mobile phones at the shooting moment is determined in real time by the self-developed three-dimensional electronic compass when the image is shot The pitch angle ω and the roll angle κ are simultaneously determined, and the coordinates (X S , Y S , Z S ) of the two shooting centers S1 and S2 are accurately determined. The heading angle is the angle between the main optical axis and the north direction, and the angles between the main optical axes and the north direction in the images p1 and p2 are ω1 and ω2, respectively. The pitch angle ω is the angle between the axis and the vertical plane, and the angles between the axes and the vertical plane in the images p1 and p2 are ω1 and ω2, respectively. The roll angle k is the angle between the axis and the horizontal plane, and the angles between the axes and the horizontal plane in the images p1 and p2 are k1 and k2, respectively.

[0172] The shooting center, the image point, and the object point, i.e. the forest fire point, satisfy the collinearity equation and form a light beam. The collinearity equation is obtained based on the rotation relationship from the intelligent mobile phone coordinate system to the world coordinate system, and the light beam method is used for double image analysis. The three-dimensional attitude angle at the shooting moment is obtained by the three-dimensional electronic compass as the initial value for optimization of the spatial position and attitude parameters, i.e. accurate acquisition of the pose information of the mobile shooting forest fire point image. After optimization, all the homonymous image points and the corresponding object points, i.e. the forest fire point coordinates, of the left and right forest fire point images satisfy the collinearity condition equation, and the error equation is listed as follows:

[0173]

[0174] In the formula, (x L , y L ), (x R , y R ) are the image plane coordinates of the homonymous image points p1 and p2 of the forest fire point P in the left and right two images, which are directly obtained from the left and right forest fire point images after image matching, (x L0 , y L0 , f L ), (x R0 , y R0 , f R ) are the internal orientation elements of the left and right cameras, respectively. S1 and S2 are the coordinates of the shooting centers S1 and S2 in the shooting measurement space rectangular coordinate system and , respectively, which are obtained by mobile GPS positioning. (aL1 , b L1 , c L1 ), (a L2 , b L2 , c L2 ), (a L3 , b L3 , c L3 ) respectively represent the parameters corresponding to the first row, the second row and the third row of the rotation matrix R1 after the optimization of the spatial position and the attitude parameters of the left image, (a R1 , b R1 , c R1 ), (a R2 , b R2 , c R2 ), (a R3 , b R3 , c R3 ) respectively represent the parameters corresponding to the first row, the second row and the third row of the rotation matrix R2 after the optimization of the spatial position and the attitude parameters of the right image, and the rotation matrix R1 and the rotation matrix R2 are obtained based on the rotation matrix R; and are the coordinates of the corresponding points of the left and right homonymous image points in the measuring coordinate system and , the three-dimensional attitude angles of the left and right forest fire point images and the coordinates of the corresponding points of the homonymous image points are solved according to the least square adjustment principle, wherein the coordinates (X, Y, Z) of the corresponding points of the homonymous image points of the forest fire point are the solution.

[0175] Progressive positioning of the forest fire point based on multi-image combination:

[0176] A set of coordinates (X, Y, Z) of the corresponding points of the homonymous image points of the forest fire point are obtained based on the bundle adjustment of each set of reference images and the to-be-matched images, and then m sets of positioning coordinates of the forest fire point are obtained based on n images (n≥3):

[0177]

[0178] For a real fire point P, the m sets of positioning coordinates P1, P2, P3, etc. of the forest fire point calculated by the method should be adjacent to the real fire point P, and thus the centroid of the m points can be solved as an estimated value of the coordinates of the P point.

[0179] The centroid refers to a point whose horizontal coordinate, vertical coordinate and Z coordinate are the average values of the horizontal coordinates, vertical coordinates and Z coordinates of the m points, respectively. That is, assuming that the coordinates of the i-th point are (X i , Y i , Z i ), the coordinates of the centroid are:

[0180]

[0181] Then the final solution (X P , Y P , Z P ) is the coordinate of the forest fire point.

Claims

1. A forest fire point positioning method combining satellite images and mobile phone images, characterized in that, The method comprises the following steps: Step S1. Obtain a plurality of coordinates of landmark points on a map based on a smart phone with a map positioning function and sensors, and take images of forest fire points based on the coordinates of the landmark points on the map, while accurately obtaining spatial multi-pose information of the mobile shooting of the images of the forest fire points; Step S2. Perform mobile progressive refinement positioning on the forest fire points in the plurality of taken images of the forest fire points based on the accurately obtained spatial multi-pose information, to obtain coordinates of the forest fire points; The specific steps of the step S1 are: Step S1.

1. Based on spatial semantic constraints, select a plurality of shooting positions in combination with the map and the real environment, to obtain coordinates of landmark points on the map, and sequentially take images of forest fire points on the coordinates of the landmark points on the map based on a multi-pose shooting model, wherein the overlap degree of the taken images of the forest fire points is 60-80%, and manually select a point in the forest fire area in the taken image of the forest fire points as an important feature point based on the first taken image of the forest fire points; Step S1.

2. Obtain mobile shooting parameters of the smart phone when taking the images of the forest fire points; Step S1.

3. Correct the positioning result of the smart phone based on the mobile shooting parameters and the coordinates of the landmark points on the map, that is, optimize the pose information of the smart phone through multi-pose shooting, to realize accurate acquisition of the pose information of the mobile shooting of the images of the forest fire points; The specific steps of the step S2 are: Step S2.

1. Identify the same forest fire point position in the images of the forest fire points taken at a plurality of shooting positions based on SIFT algorithm and RANSAC algorithm, to obtain coordinates of the forest fire points; Step S2.

2. Obtain coordinates of object points corresponding to the same-named image points of the forest fire points in the two images of the forest fire points based on the coordinates of the forest fire points and a double-image forest fire positioning algorithm; Step 2.

3. Perform mobile progressive refinement positioning based on the coordinates of the object points corresponding to the same-named image points of a plurality of forest fire points in a plurality of groups of forest fire point images, to obtain coordinates of the forest fire points; The specific steps of the step S2.3 are: The reference image and the image to be matched in each group are based on the light beam method to obtain a set of corresponding object point coordinates (X, Y, Z) of the same-named image points of the forest fire points, and then The forest fire point image is obtained The positioning coordinates of the group of fire points are obtained, wherein : By solving this equation, the coordinates of the centroid of each group of points are obtained as an estimate of the coordinates of the forest fire point P. Assuming that the coordinates of the first point are (x1, y1), then the coordinates of the centroid are: , , (x, y) = ((x1+ x2+ x3+ x4+ x5+ x6+ x7+ x8+ x9+ x10+ x11+ x12+ x13+ x14+ x15+ x16+ x17+ x18+ x19+ x20+ The final solution of (x, y) is the coordinate of the forest fire point. , , ) is the coordinate of the forest fire point.

2. The method according to claim 1, wherein, The specific steps of the step S1.1 are: Step S1.

11. First, locate the real-time position of the forest ranger on the map; Step S1.

12. Select a plurality of landmark features as shooting points based on the real-time position of the forest ranger located on the map, spatial semantic constraints and the real environment, to select a plurality of shooting positions, that is, to obtain a plurality of landmark points, wherein the spatial semantic constraints refer to the heterogeneity features between the landmark features in the map image and the features outside the landmark features, and the heterogeneity features include regional, visual and landmark relationships; Step S1.

13. Obtain the coordinates of the plurality of landmark points on the map based on the selected plurality of landmark points, to obtain coordinates of landmark points on the map, and sequentially take images of forest fire points on the coordinates of the landmark points on the map based on a multi-pose shooting model, wherein the multi-pose shooting model refers to a combination of vertical and horizontal shooting for the same fire area.

3. The method of claim 2, wherein the method further comprises: The specific steps of the step S1.2 are: The mobile shooting parameters of the smart phone include internal orientation elements and external orientation elements; The internal orientation elements are: The inner orientation elements are determined by the smartphone and are parameters describing the relative position between the shooting center and the forest fire point image, including three parameters: the vertical distance f of the shooting center S to the image, and the coordinates of the image principal point o in the frame coordinate system ), wherein the vertical distance refers to the principal distance; The external orientation elements are: The exterior orientation elements of the image refer to the spatial position and attitude parameters of the forest fire point image at the time of shooting, and each forest fire point image has six exterior orientation elements, which are three line elements, i.e. the coordinates X S , Y S , Z S of the shooting center S in the object space rectangular coordinate system and three angle elements describing the attitude information of the image at the time of shooting, i.e. the heading angle , the pitch angle , and the roll angle . The exterior orientation elements of the image are provided by the smart phone, the line elements of the image are obtained by using the location service, and the angle elements of the photo are obtained by jointly solving the return values of the acceleration sensor, the magnetic field sensor and the direction sensor. In the Android mobile phone, the output of the sensor is taken as the reference of the local coordinate system of the smart phone, and the smart phone coordinate system is a relative coordinate system defined with the screen of the mobile phone as the reference. The origin of the inertial coordinate system coincides with the origin of the smart phone coordinate system, and the coordinate axes of the inertial coordinate system are parallel to the coordinate axes of the world coordinate system, that is, the inertial coordinate system is regarded as the intermediate state between the smart phone coordinate system and the world coordinate system. Therefore, the inertial coordinate system is used to convert the smart phone coordinate system into the world coordinate system, and the conversion formula of the smart phone coordinate system into the world coordinate system is as follows: Rotation around the z-axis with rotation angle The resulting rotation matrix is: Rotation around the x-axis with rotation angle The resulting rotation matrix: Rotation around the y-axis with rotation angle The resulting rotation matrix: The three basic rotation sequences are combined in different rotation sequences to obtain a rotation matrix between two coordinate systems, and the rotation sequence includes any one of z-x-y, z-y-x, x-z-y, x-y-z, y-z-x and y-x-z. If the z-x-y sequence is rotated, the rotation matrix is: Therefore, the rotation relationship of the smart phone coordinate system to the world coordinate system is: wherein (x, y, z) are the three-dimensional coordinates of the point in the world coordinate system, , , ) are the three-dimensional coordinates of the point in the world coordinate system, , , ) are the three-dimensional coordinates of the point in the smartphone coordinate system, denotes the transpose.

4. The method according to claim 3, wherein, The specific steps of the step S1.3 are: Step S1.

31. Position optimization First, define the plane coordinates of the multiple shooting positions obtained by the smart phone as (lon1, lat1), (lon2, lat2), (lon3, lat3), …, and the plane coordinates of the shooting positions obtained by manual selection of the Map World are respectively (Lon1, Lat1), (Lon2, Lat2), (Lon3, Lat3), …; Then, the difference between the smart phone positioning in each shooting position coordinate data and the coordinate of the map landmark is calculated as the correction number; Finally, the arithmetic mean of the longitude and latitude corrections is calculated , As the final correction for the mobile phone positioning: wherein means a non-zero natural integer; Based on the smart phone shooting position coordinate data and the final correction number, the corrected geodetic coordinates of the corresponding shooting position are obtained , , ), that is, the space parameters of the optimized exterior orientation elements are obtained, and then the geodetic coordinates are converted into space rectangular coordinates according to the following formula: wherein is the first eccentricity, is the radius of curvature of the quadrant of the circle; Step S1.

32. Attitude optimization Get the vertical and horizontal shooting forest fire point images of the same shooting position and the same fire area, remove the theoretical angle difference of the two shootings, and take the average value as the final attitude angle to realize the optimization of the attitude parameter, that is, when the vertical shooting is converted into horizontal shooting, the roll angle and the pitch angle should differ by 90 degrees. Therefore, after removing the 90 degree difference, the average value of the two is obtained, and the optimized attitude parameter is obtained; Step S1.

33. Precise acquisition of the pose information of the mobile shooting forest fire point image based on the position optimization and the attitude optimization.

5. The method of claim 4, wherein the method further comprises: The specific steps of the step S2.1 are: Step S2.

11. Preprocessing of each forest fire point image, that is, distortion of the smart phone when shooting the forest fire point image, and the distortion of the wide-angle lens needs to be corrected; Step S2.

12. The SIFT algorithm is used to identify the feature points of each forest fire point image obtained after preprocessing, and the two adjacent forest fire point images in shooting time are coarsely matched, wherein the forest fire point image shot in advance is taken as the reference image, and the forest fire point image shot later is taken as the matching image; Step S2.

13. Field voting denoising is performed on the coarse matched feature points, and the initial inlier set is obtained after denoising; Step S2.

14. Screening the initial inlier set obtained by coarse matching based on the improved RANSAC algorithm to obtain accurate and correct matching points of more than 6 pairs, that is, obtaining the corresponding relationship of the feature points of the first forest fire point image as the reference image and the second forest fire point image as the matching image, and transferring the coordinates of the feature points of the first forest fire point image to the second forest fire point image, and then transferring the second forest fire point image as the reference image to the third forest fire point image, and so on. If the important feature points are contained in the matching points of more than 6 pairs, go to step S2.15, otherwise, based on the important feature points, the corresponding relationship of the feature points of each forest fire point image is established, and then go to step S2.

15. Step S2.

15. Based on the corresponding relationship of the feature points of each forest fire point image, the forest fire point coordinates on the subsequent forest fire point images are calculated according to the manually selected forest fire points when the first forest fire point image is shot.

6. The method of claim 5, wherein the method further comprises: The specific steps of step S2.12 are as follows: Firstly, remove the noise points in each forest fire point image by Gaussian blur, create a multi-scale image, and create a multi-scale space based on the scale image and the Gaussian difference enhanced image features, that is, based on the multi-scale image, a Gaussian pyramid of each forest fire point image is constructed, and the pixel points of the same group of adjacent two layers in the Gaussian pyramid are subtracted to obtain a Gaussian difference pyramid, that is, a multi-scale space: wherein, is a two-dimensional image of each forest fire point image to be detected; is a Gaussian scale space or a Gaussian pyramid or a Gaussian image of the forest fire point image; is a Gaussian function; is a scale space factor, is a multiple of the adjacent scale space, refers to a Gaussian difference pyramid, i.e. a multi-scale space; Secondly, compare each pixel value of the Gaussian image in the multi-scale space with the surrounding 26 pixel values. If the pixel is the highest or lowest pixel among the adjacent pixels, it is considered that the point is a candidate feature point of each forest fire point image in the scale space. Then, the gradient direction distribution characteristics of the neighborhood pixels of the removed feature points are used to specify the direction parameters for each feature point, and the gradient direction and its gradient amplitude of all pixels within a circle with the feature point as the center and 1.5 times the scale of the Gaussian image in the multi-scale space where the feature point is located as the radius are counted to create a gradient histogram, wherein the peak value of the histogram represents the main direction of the neighborhood gradient at the key point, that is, the direction of the feature point, and the direction reaching 80% of the maximum value is taken as the auxiliary direction. Finally, a unique fingerprint, called feature point descriptor, is generated for the feature point by the gradient main direction, auxiliary direction and size of the adjacent pixels. The distances between the feature point descriptors in the matching image and the feature point descriptors in the reference image are calculated, and all the results corresponding to the feature point descriptors are sorted, and the nearest distance is taken as the matching point, that is, the result after coarse matching is obtained. The specific steps of step S2.13 are as follows: After the coarse matching, the distance d and the difference of the main direction angle Δθ of any two feature points in each forest fire point image are calculated respectively, and then the distance d and the difference Δθ are normalized according to the row vector. After the normalization, the distance inner product value of each pair of matching points between the reference image and the to-be-matched image is calculated and the main direction angle inner product value Finally, through the threshold value set in advance, whether the distance inner product value and the main direction angle inner product value of a pair of matching points are less than the threshold value is compared. If yes, the matching point is put into the inlier set, that is, the initial inlier set is obtained, otherwise, the matching point is put into the outlier set. in, , For any pair of initial matching points on the same forest fire image, ( , , )and( , , () is represented as a reference image and the image to be matched Corresponding matching point and The pixel coordinates and main direction, Reference image and the image to be matched The inner product, These represent any two feature points in each forest fire image. and The main direction; the specific steps of step S2.14 are as follows: First, randomly extract four sample data which are not collinear from the feature point set based on SIFT algorithm, calculate the 3x3 transformation matrix H, denoted as model M, then calculate the projection error of all data and model M, if less than the threshold, add to the initial inlier set, when the number of elements in the initial inlier set Q is greater than the optimal inlier set Q-best, update Q-best=Q, at the same time, judge whether the iteration number is greater than the number K, if yes, exit, otherwise, the iteration number is increased by 1, and the above operation is repeated until the iteration is finished, so as to realize the elimination of abnormal data, obtain accurate matching points, that is, obtain the corresponding relationship of feature points between each reference image and the corresponding matching image; in,( , () refers to the location of the feature points in the image to be matched; , ) represents the location of feature points in the reference image; s is the scale parameter. H is the transformation matrix; The corresponding relationship of feature points between each reference image and the corresponding matching image is obtained, that is, the same forest fire point position in each forest fire point image is recognized, and the forest fire point coordinates in each forest fire point image are obtained.

7. The method of claim 6, wherein the method further comprises: The specific steps of the step S2.2 are: Based on , point, the same forest area is photographed to obtain two images with an overlap of more than 60%, left and right, with as the shooting baseline, as the main optical axis of the left shooting station, as the main optical axis of the right shooting station, and the forest fire point The composition of the left and right images is , The heading angle pitch angle and roll angle of the two shooting centers are determined at the shooting moment by the self-developed three-dimensional electronic compass, and the coordinates of the two shooting centers are accurately determined , , , , , wherein the heading angle is the angle between the main optical axis and the north direction, and when the composition is , , the angle between the main optical axis and the north direction is , , the pitch angle is the angle between the axis and the vertical plane, and when the composition is , , the angle between the axis and the vertical plane is , , and the roll angle is the angle between the axis and the horizontal plane, and when the composition is , , the angle between the axis and the horizontal plane is , . The shooting center, the image point, the object point, that is, the forest fire point satisfies the collinear equation, constitutes a light beam, the collinear equation is obtained based on the rotation relationship from the intelligent mobile phone coordinate system to the world coordinate system, the double image analysis is carried out by using the light beam method, the light beam is taken as the adjustment unit, the three-dimensional attitude angle at the shooting moment is obtained by the three-dimensional electronic compass and is taken as the initial value to optimize the space position and attitude parameter, that is, the pose information of the mobile shooting forest fire point image is accurately obtained, after optimization, all the homonymous image points and the corresponding object points, that is, the forest fire point coordinates of the left and right forest fire point images satisfy the error equation formula of the collinear equation: In the formula, , ), ( , ) are the forest fire points The image plane coordinates of the same named image points , on the left and right two images are directly obtained from the left and right forest fire point images after image matching, , , ), ( , , ) are the internal orientation elements of the left and right cameras; , , ), ( , , ) are the coordinates of the left and right camera centers , in the right-angle coordinate system of the surveying and mapping space and , which are obtained by mobile terminal GPS positioning; , ), ( , ), ( , ) represent the parameters corresponding to the first row, the second row and the third row of the rotation matrix R1 after the optimization of the spatial position and attitude parameters of the left image, , ), ( , ), ( , ) represent the parameters corresponding to the first row, the second row and the third row of the rotation matrix R2 after the optimization of the spatial position and attitude parameters of the right image, and the rotation matrix R1 and the rotation matrix R2 are obtained based on the rotation matrix R; , ) and ( , ) are the coordinates of the corresponding points of the left and right same named image points in the surveying and mapping coordinate system and , and the three-dimensional attitude angles of the left and right forest fire point images and the coordinates of the corresponding points of the same named image points are solved according to the least square adjustment principle by measuring 6 same named image points, wherein the coordinates of the corresponding points of the forest fire point same named image points , ) are the solutions.

Images