Fire point detection position and area estimation method based on ortho-angle of view of unmanned aerial vehicle

Abstract

The invention provides a fire point detection position and area estimation method based on an ortho-angle of view of an unmanned aerial vehicle. The method comprises the steps of 1, planning a flight route of an unmanned aerial vehicle, and setting flight height and an orthographic view angle; step 2, cascading real-time video streams collected by an unmanned aerial vehicle camera through a GB / T28181 protocol; 3, fire point area detection is carried out on the collected video, and the pixel point position of a fire point is identified; and 4, estimating the area of the fire point according to the internal reference and the flight height of the unmanned aerial vehicle camera, and determining the area by the number of fire point pixels. According to the method for estimating the flame detection area at the orthographic view angle of the unmanned aerial vehicle, the real position and area estimation of flame pixel points in a picture inferred by deep learning can be quickly calculated according to the flight height and the internal reference of the camera. According to the method, the estimated position and area are calculated according to different camera loads mounted on the unmanned aerial vehicle, the accuracy is high, and the practicability is high.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision technology, and specifically is a method for detecting the position and area of ​​a fire point based on the orthographic viewing angle of an unmanned aerial vehicle. Background Art

[0002] Due to the rise of drone applications and their increasingly mature technological advantages, such as high mobility and flexibility: drones can hover, turn, ascend, descend, dive, and climb in various environments, and can achieve various heading changes as needed; the ability to reach difficult or dangerous areas: drones can fly into places that are difficult for humans to reach. Drone technology can help humans complete some high-risk or even impossible tasks, such as rescue, exploration, and surveying; reducing the risk to human life: drone technology can replace humans to complete some dangerous tasks, thereby reducing the risk to human life, which brings people more safe working environments and life guarantees.

[0003] In the important application scenario of forest and grass fire prevention, unmanned aircraft can fly according to the predetermined design plan, collect video data or take photos in real time during the flight, and analyze and infer the relevant view data through the return protocol to calculate the location of the fire point in the view data. In addition, since the flight attitude information of the drone is also synchronized with the view information and returned, the area and location of the fire point can be estimated based on the camera pinhole model. Summary of the invention

[0004] This method proposes a method for detecting the position and surface estimation of fire points based on the orthographic view of unmanned aerial vehicles. The flame detection model is used to infer the position and pixel clusters of the fire points in the image, and the intrinsic parameter information of the camera and the height information of the flight relative to the ground during the flight are combined with the cluster information of the fire point inference results in the image to calculate the estimated flame area and position. This position is generally the center point information of the cluster.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a method for detecting the position and area of ​​a fire point based on an orthographic view of a drone, comprising the following steps:

[0006] S1. Plan the area that the drone needs to inspect, set the flight height relative to the ground and the front view shooting angle;

[0007] S2. The video stream collected by the drone payload device is transmitted to the streaming media forwarding device, which forwards the video stream to the analysis system through the GB / T28181 video cascade protocol;

[0008] S3. Use a decoder to decode the video stream information and obtain each frame of the image;

[0009] S4. Use the trained image fire point detection algorithm model to infer the fire point information in the image;

[0010] S5. Image morphological algorithm calculates the pixel clusters of the fire point;

[0011] S6. Read the height information of the fire point reasoning frame;

[0012] S7. Calculate the actual position and area of ​​the mass using the pinhole model;

[0013] S8. Fire location and area estimation is completed.

[0014] Beneficial effect: The method for detecting the position and estimating the area of ​​a fire point in the orthographic perspective of a UAV described in the present invention utilizes the high maneuverability of UAVs to reach locations that are difficult for other tools to reach to complete some inspection tasks. In inspection tasks, especially in forest and grass fire prevention scenarios, the task can be completed quickly to discover fire points early and estimate the position and area of ​​the fire points. The alarm data can be used for immediate early warning and disposal, which can effectively discover fire points in the early stages and promptly handle and extinguish fires, thereby protecting the safety of life and property. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method steps of the present invention;

[0016] Figure 2 The camera imaging pinhole model used in the present invention; DETAILED DESCRIPTION

[0017] In order to more clearly describe the technical solution in the example of the present invention, the present invention will be described in detail below in conjunction with the drawings in the embodiment. The specific embodiment described in the present invention, as well as other embodiments based on the present invention, all belong to the protection scope of the present invention.

[0018] The purpose of the present invention is to provide a method for estimating the position and area of ​​fire point detection based on the orthographic view of unmanned aerial vehicles. The method mainly includes 7 modules, namely, unmanned aerial vehicle flight industry planning, cascade video stream after payload return view image, video decoding to obtain image frame, fire point pixel cluster reasoning in the image, configuration of camera pinhole model, horizontal and vertical coordinates of the flight attitude of the analyzed image frame and relative ground height acquisition, and calculation of the real fire point position and area corresponding to the cluster in the image.

[0019] like Figure 1 The figure is a flow chart of the steps of a method for detecting the position and estimating the area of ​​a fire point based on the orthographic perspective of an unmanned aerial vehicle according to an example of the present invention.

[0020] The embodiment of the present invention provides a method for detecting the position and area of ​​a fire point based on an orthographic view of an unmanned aerial vehicle, comprising the following steps:

[0021] Step S1. Plan the area that the drone needs to inspect, set the flight height relative to the ground and the front view shooting angle;

[0022] Step S2. The video stream collected by the drone load device is transmitted to the streaming media forwarding device, and the streaming media forwarding device forwards the video stream to the analysis system through the GB / T28181 video cascade protocol;

[0023] Step S3. Use a decoder to decode the video stream information and obtain each frame of image;

[0024] Step S4. Use the trained image fire point detection algorithm model to infer the fire point information in the image;

[0025] Step S5. Calculate the pixel clusters of the fire point using the image morphological algorithm;

[0026] Step S6. Read the height information of the fire point reasoning frame;

[0027] Step S7. Calculate the real area of ​​the mass using the pinhole model. The specific diagram is as follows: Figure 2 As shown;

[0028] S71. The pinhole model of camera imaging includes the intrinsic and extrinsic parameters of the camera. The intrinsic and extrinsic parameters jointly determine the projection of real-world objects to specific pixel positions and sizes.

[0029] S72. The specific calculation method is as follows:

[0030]

[0031] Among them, Z c is the depth of the three-dimensional point in space from the center of the camera, u and v are the imaging positions of the three-dimensional point on the X and Y axes in the image, and the center point of the image is the coordinate origin position, X w , Y w and Z w is the coordinate of a point in three-dimensional space in the real world. K is the camera internal parameter, [R t] is the camera external parameter, R is the rotation matrix between the space coordinate system and the camera coordinate system in three-dimensional space, and t is the translation vector.

[0032] S73. The matrix of the camera internal parameter K is expressed as:

[0033]

[0034] S74. The matrix representation of the camera external parameters [R t] is:

[0035]

[0036] Among them, α, β, and γ are the angles between the real space coordinate system and the camera coordinate system in the three directions of X, Y, and Z. x ,t y ,tz is the X, Y, Z distance between the origins of the two coordinate systems. Then the two coordinate systems can be moved to the same origin and the directions of the three coordinate systems X, Y, Z can be consistent through the rotation matrix and translation vector;

[0037] S75. Combining formulas (1), (2), (3) and (4), the expanded calculation method is:

[0038]

[0039] S76. Assume that the 3D space coordinate system and the camera coordinate system are the same coordinate system, then: the angles between the three coordinate systems α, β, and γ are all 0°, and the angle between the origins of the other two coordinate systems is t x ,t y ,t z The translation size is also 0.

[0040]

[0041] S77. In addition, set the camera distortion s to zero. Since the flight is at a relative height to the ground, the imaged object can be basically determined on a plane Z w Can be set to 1. Then formula (5) is:

[0042]

[0043] S78. Since the flight is at a relative height to the ground, the imaged object can be basically determined on a plane Z w On the plane,

[0044]

[0045] So,

[0046]

[0047] S79. In the case of high spatial consistency, the real area of ​​the fire point is calculated by the number of pixels in the horizontal and vertical directions of the cluster, where the real area of ​​each pixel is S = X w Y w , then the size of the actual fire point of N pixels is NS. The position is the coordinate position of the center point of the cluster (X c Y c Z AGL ), where Z AGL =Z w -Z S , Z S Set the altitude for the flight.

[0048] S710.f x and f yThe internal parameters of the camera are mainly fixed by the lens manufacturer. The conversion formula related to the focal length, imaging photosensitive material size and resolution is as follows:

[0049]

[0050] Where f is the focal length in mm, 1 / dx and 1 / dy are the number of pixels per mm in the x and y directions of the image, respectively. Query the size of the camera CMOS or CCD. If the x and y pixel resolutions are set, and the image size is L by W, then 1 / dx = L / x. Similarly, dy can be calculated. Substitute the relevant values ​​into formula (9) to complete the calculation.

[0051] Step S8. Fire point location and area estimation is completed.

Claims

1. A video concentration method for multi-target tracking, comprising the following steps: (1) Plan the area that the drone needs to inspect, set the flight height relative to the ground and the front view shooting angle; (2) The video stream collected by the drone payload device is transmitted to the streaming media forwarding device, which forwards the video stream to the analysis system through the GB / T28181 video cascade protocol; (3) Use a decoder to decode the video stream information and obtain each frame of the image; (4) Use the trained image fire point detection algorithm model to infer the fire point information in the image; (5) Image morphological algorithm calculates the pixel clusters of the fire point; (6) Read the internal parameters of the camera through the payload information mounted on the drone and configure the matrix of the pinhole model; (7) Read the height information of the fire point reasoning frame; (8) Calculate the actual position and area of ​​the mass using the pinhole model; (9) Fire location and area estimation completed.

Images