Method and device for fast locating a fire source

Abstract

The application provides a method and device for quickly locating a fire source, and relates to the technical field of image processing, and comprises the following steps: acquiring pose information of a UAV carrying a camera in a navigation coordinate system, and determining a first conversion relationship between the navigation coordinate system and a line-of-sight coordinate system according to the pose information, wherein the line-of-sight coordinate system takes the optical center of the camera as the origin and the optical axis as the Z-axis; acquiring a to-be-processed image collected by the camera, a first distance measured along the Z-axis direction of the line-of-sight coordinate system, and a fire source region in the to-be-processed image, and obtaining a fire source region centroid coordinate; determining a second conversion relationship between the line-of-sight coordinate system and a fire source coordinate system according to the fire source region centroid coordinate, wherein the fire source coordinate system takes the optical center of the camera as the origin and a line connecting the optical center and the fire source region as the Z-axis; and determining the absolute three-dimensional coordinate of the fire source region according to the first distance, the first conversion relationship and the second conversion relationship.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for rapid fire source location. Background Technology

[0002] Forest fire prevention is crucial for protecting forest resources and the ecological environment. Timely detection and accurate location of fire sources are the core elements of fire emergency response. Unmanned Aerial Vehicles (UAVs) are widely used in forest fire patrols due to their mobility, flexibility, and rapid deployment. Equipped with visual sensors, UAVs can conduct large-scale and efficient fire reconnaissance.

[0003] One existing method for locating fire sources primarily relies on a gimbal camera system equipped with a laser rangefinder. This method first captures the fire source target using a visual sensor, then controls the gimbal to rotate, ensuring the camera always locks the fire source target in the center of the field of view. When the target is in the center of the field of view, the laser rangefinder is activated to measure the straight-line distance between the drone and the fire source. Finally, combining the drone's own GPS position and attitude information, the geographical location of the fire source is calculated through coordinate transformation.

[0004] However, the aforementioned existing technical solutions have significant drawbacks: First, these solutions heavily rely on gimbal systems, requiring the gimbal camera to constantly lock the target in the center of the field of view, which increases system complexity and hardware costs. Second, in remote monitoring applications, there is an unavoidable delay in the transmission of video streams from the drone to the cloud platform. This makes it difficult for ground operators to control the gimbal in real time and accurately keep the moving or changing fire source in the center of the field of view, resulting in significant positioning delays and decreased accuracy. Finally, some solutions, in order to simplify calculations, assume that the drone's takeoff altitude is the same as the ground altitude of the fire source. This is generally not valid in mountainous forest fire prevention scenarios with complex and varied terrain, limiting the applicability of the technology. Summary of the Invention

[0005] This invention provides a method and apparatus for rapid fire source location, which solves the problem of how to more conveniently and rapidly locate fire sources in the prior art.

[0006] This invention provides a method for rapid fire source location, comprising the following steps:

[0007] The pose information of the drone equipped with a camera in the navigation coordinate system is obtained, and a first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system is determined based on the pose information, wherein the line-of-sight coordinate system takes the optical center of the camera as the origin and the optical axis as the Z-axis.

[0008] The image to be processed is acquired by the camera, and the first distance is measured along the Z-axis of the line-of-sight coordinate system. The fire source region in the image to be processed is identified, and the centroid coordinates of the fire source region are obtained.

[0009] Based on the centroid coordinates of the fire source area, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis;

[0010] The absolute three-dimensional coordinates of the fire source region are determined based on the first distance, the first transformation relationship, and the second transformation relationship.

[0011] According to a method for rapid fire source location provided by the present invention, the absolute three-dimensional coordinates of the fire source area are determined based on a first distance, a first transformation relationship, and a second transformation relationship, including:

[0012] Based on the first distance and the first conversion relationship, the fire source height of the fire source area is determined;

[0013] Based on the height of the fire source, and the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated;

[0014] Based on the second distance and the location information of the UAV, the absolute three-dimensional coordinates of the fire source area are determined.

[0015] According to a method for rapid fire source location provided by the present invention, the absolute three-dimensional coordinates of the fire source area are determined based on a second distance and the position information of the UAV, including:

[0016] By combining the second distance with the second transformation relationship, the relative position vector of the fire source area in the line-of-sight coordinate system is calculated;

[0017] Using the first transformation relationship, the relative position vector is transformed to the navigation coordinate system;

[0018] The converted relative position vector is vector-added with the absolute position vector of the UAV in the navigation coordinate system to obtain the absolute three-dimensional coordinates of the fire source area.

[0019] According to a method for rapid fire source localization provided by the present invention, the first transformation relationship is used to represent a first rotation matrix of the spatial attitude of the line-of-sight coordinate system relative to the navigation coordinate system;

[0020] The second transformation relationship is used to represent the second rotation matrix that deviates the fire source coordinate system from the line-of-sight coordinate system.

[0021] According to a method for rapid fire source location provided by the present invention, the method for determining the second rotation matrix includes:

[0022] Based on the centroid coordinates of the fire source region, the image center coordinates of the image to be processed, and the camera intrinsic parameters, the fire source tilt angle and fire source deflection angle of the fire source region relative to the optical axis of the line-of-sight coordinate system are calculated.

[0023] The second rotation matrix is ​​generated based on the fire source tilt angle and fire source deflection angle.

[0024] According to the present invention, a method for rapid fire source localization is provided, which identifies the fire source region in the image to be processed and obtains the centroid coordinates of the fire source region, including:

[0025] An object detection network is used to analyze the image to be processed, identify the fire source region, and obtain the bounding box of the fire source region;

[0026] Obtain the centroid coordinates of the boundary box of the fire source region to obtain the centroid coordinates of the fire source region.

[0027] The present invention also provides a device for rapid fire source location, comprising:

[0028] The acquisition module is used to acquire the pose information of the drone equipped with a camera in the navigation coordinate system, and determine the first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information.

[0029] The recognition module is used to acquire the image to be processed captured by the camera, as well as the first distance measured along the Z-axis of the line-of-sight coordinate system, and to identify the fire source area in the image to be processed, and obtain the centroid coordinates of the fire source area.

[0030] The determination module is used to determine a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system based on the centroid coordinates of the fire source area;

[0031] The calculation module is used to determine the absolute three-dimensional coordinates of the fire source area based on the first distance, the first transformation relationship, and the second transformation relationship.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for rapid fire source location as described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for rapid fire source location as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for rapid fire source location as described above.

[0035] The present invention provides a method and apparatus for rapid fire source localization. By fixing a laser rangefinder to a camera, the absolute three-dimensional coordinates of the fire source can be directly calculated using the pixel position of the fire source in a single frame image, combined with the UAV pose and a single laser rangefinder measurement, through coordinate system transformation and mathematical model solving. This method does not require the fire source to be located at the center of the image, thus avoiding the latency problem of remote control gimbals and significantly reducing the system's hardware dependence and cost. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of a fire source observation scenario provided in this invention;

[0038] Figure 2 This is a flowchart illustrating the method for rapid fire source location provided by the present invention;

[0039] Figure 3 A schematic diagram of the device for rapid fire source positioning provided by the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0042] Figure 1 This is a schematic diagram of a fire source observation scenario provided in this invention, such as... Figure 1As shown, the camera is mounted below the drone, and the camera's height above the ground is not equal to the height of the camera and the fire area. The laser optical axis is always centered on the camera and can be directly obtained. However, the fire source detection optical axis does not coincide with the laser optical axis. Therefore, the most common algorithm is to first move the camera so that the fire source optical axis is always centered on the camera, i.e., coincident with the laser optical axis. At this point, the distance to the fire source optical axis can be directly obtained.

[0043] However, this solution requires the design of a control system to adjust the position of the PTZ, which introduces two problems: 1) It is not suitable for deployment on monitoring platforms such as cloud platforms because the latency of video transmission to the monitoring platform is large, which makes the design of the PTZ control system more difficult; 2) The PTZ must be used, which makes the overall system cost higher.

[0044] Therefore, in this invention, the absolute position of the fire source is adaptively calculated directly using laser ranging information and pixel information.

[0045] Figure 2 This is a flowchart illustrating the method for rapid fire source location provided by the present invention, as shown below. Figure 2 As shown, the method includes the following:

[0046] Step 210: Obtain the pose information of the drone equipped with the camera in the navigation coordinate system, and determine the first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information, wherein the line-of-sight coordinate system takes the optical center of the camera as the origin and the optical axis as the Z-axis;

[0047] In this invention, the navigation coordinate system is a global map. Typically, a North-East-Earth (NED) coordinate system is used, which is a fixed reference frame.

[0048] More specifically, the navigation coordinate system is established using true north as the x-axis, true east as the y-axis, and the z-axis as the right-hand rule. The instantaneous position of the camera is the origin of the coordinate system.

[0049] The drone can know its absolute position (latitude, longitude and altitude) and attitude (nose direction, tilt and pitch) on the map in real time through its built-in GPS and inertial measurement unit.

[0050] The line-of-sight coordinate system is the camera's "self-viewpoint". It takes the optical center of the camera lens as its origin, and the Z-axis always points directly in front of the camera as it is shooting, that is, in the direction of the optical axis. This coordinate system moves and rotates with the camera.

[0051] More specifically, the line-of-sight coordinate system is established with the right side of the camera as the x-axis, the front as the z-axis, and the y-axis following the right-hand rule, with the origin at the camera center. The camera pose angle is defined as follows:

[0052] Line-of-sight tilt angle qf: The angle between the camera's optical axis and the plane xoy of the navigation coordinate system;

[0053] Line of sight deflection qh: The angle between the projection of the camera's optical axis onto the xoy plane of the navigation coordinate system and the ox axis;

[0054] In this invention, the first transformation relationship is the first rotation matrix. The first rotation matrix is ​​constructed using the attitude angles of the UAV, which can transform the coordinates in the line-of-sight coordinate system to the navigation coordinate system, achieving a precise association between the two coordinate systems and providing a foundation for subsequent positioning calculations.

[0055] Step 220: Obtain the image to be processed captured by the camera, and the first distance measured along the Z-axis of the line-of-sight coordinate system, and identify the fire source region in the image to be processed to obtain the centroid coordinates of the fire source region.

[0056] In this invention, the camera of the drone acquires images to be processed, typically in a two-dimensional plane. The camera is mounted on the drone and captures images in real time as the drone moves. The image content reflects the environment surrounding the drone and serves as the image basis for subsequent fire source identification and location.

[0057] Simultaneously, the first distance is measured along the Z-axis of the line-of-sight coordinate system. The camera's optical axis is the Z-axis of the line-of-sight coordinate system. The laser rangefinder is integrated into the center of the camera to ensure that the laser beam is emitted along the Z-axis during distance measurement, accurately measuring the straight-line distance from the camera to the fire source and providing distance data for positioning.

[0058] Next, the fire source region in the image to be processed is identified and its centroid coordinates are calculated. An object detection network (such as U-Net based on deep learning) is used to analyze the image, identify the fire source region, and label it with a bounding box. The centroid coordinates of the bounding box are the same as the centroid coordinates of the fire source region, calculated by averaging the coordinates of the top-left and bottom-right corners of the bounding box. These coordinates reflect the position of the fire source in the image and are crucial for subsequently determining the second transformation relationship between the viewpoint coordinate system and the fire source coordinate system.

[0059] Step 230: Determine the second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system based on the centroid coordinates of the fire source area, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis.

[0060] In this invention, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined based on the centroid coordinates of the fire source region. The fire source coordinate system has the optical center of the camera as its origin, and the line connecting the optical center to the fire source region is the Z-axis.

[0061] Specifically, the centroid coordinates of the fire source region are represented as (x, y) in the line-of-sight coordinate system. Based on these centroid coordinates, the fire source tilt angle and fire source deflection angle relative to the optical axis of the line-of-sight coordinate system can be calculated. The fire source tilt angle refers to the angle between the fire source direction and the optical axis of the line-of-sight coordinate system in the vertical plane, while the fire source deflection angle refers to the angle between the projection of the fire source direction onto the horizontal plane and the projection of the optical axis of the line-of-sight coordinate system onto the horizontal plane.

[0062] Using these two angles, a rotation matrix can be generated that transforms the line-of-sight coordinate system to the fire source coordinate system. This rotation matrix is ​​the second transformation relation, which describes the angular deviation of the fire source coordinate system relative to the line-of-sight coordinate system.

[0063] The second transformation relationship can convert coordinates in the line-of-sight coordinate system to the fire source coordinate system. This transformation allows the position of the fire source area in the line-of-sight coordinate system to be converted to the fire source coordinate system, thus providing a basis for subsequently determining the absolute three-dimensional coordinates of the fire source area.

[0064] Step 240: Determine the absolute three-dimensional coordinates of the fire source area based on the first distance, the first transformation relationship, and the second transformation relationship.

[0065] In this invention, based on a first distance—that is, the distance measured by the laser rangefinder at the center of the camera—and the attitude information of the UAV, such as pitch angle, roll angle, and yaw angle, the height of the fire source area is calculated using a first transformation relationship. This height represents the altitude of the fire source in the navigation coordinate system.

[0066] Next, using the centroid coordinates of the fire source region, camera intrinsic parameters, and the first transformation relationship, and with the aid of the second transformation relationship—the rotation matrix between the line-of-sight coordinate system and the fire source coordinate system—the relative position vector of the fire source region in the line-of-sight coordinate system is calculated. This step combines the fire source tilt angle and deflection angle to transform the position of the fire source region from the line-of-sight coordinate system to the fire source coordinate system.

[0067] Then, the relative position vector is transformed to the navigation coordinate system using the first transformation relationship. Finally, the transformed relative position vector is vector-added with the absolute position vector of the UAV in the navigation coordinate system to obtain the absolute three-dimensional coordinates of the fire source area in the navigation coordinate system.

[0068] In this invention, by fixing the laser rangefinder to the camera, the absolute three-dimensional coordinates of the fire source can be directly calculated using the pixel position of the fire source in a single frame image, combined with the UAV pose and a single laser rangefinder measurement, through coordinate system transformation and mathematical model solving, without the need for gimbal rotation and alignment. This method does not require the fire source to be located at the center of the image, thus avoiding the latency problem of remote control gimbals and significantly reducing the system's hardware dependence and cost.

[0069] Optionally, determining the absolute three-dimensional coordinates of the fire source region based on the first distance, the first transformation relationship, and the second transformation relationship includes:

[0070] Based on the first distance and the first conversion relationship, the fire source height of the fire source area is determined;

[0071] Based on the height of the fire source, and the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated;

[0072] Based on the second distance and the location information of the UAV, the absolute three-dimensional coordinates of the fire source area are determined.

[0073] In this invention, the height of the fire source in the fire source area can be determined based on a first distance and a first transformation relationship. The first distance is the distance measured by the laser rangefinder at the center of the camera, and the first transformation relationship is the rotation matrix between the navigation coordinate system and the line-of-sight coordinate system. With the help of the pitch attitude information of the UAV, the height of the fire source in the fire source area can be inferred.

[0074] Next, combining the fire source height with the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated. The second transformation relationship is a rotation matrix between the line-of-sight coordinate system and the fire source coordinate system. Through the second transformation relationship, the position of the fire source area in the line-of-sight coordinate system can be transformed to the fire source coordinate system, and then the second distance can be calculated in combination with the fire source height.

[0075] The second distance represents the straight-line distance from the fire source area to the camera. The second transformation relationship is the rotation matrix from the line-of-sight coordinate system to the fire source coordinate system, reflecting the angular deviation of the fire source area in the line-of-sight coordinate system. Combining the second distance with the second transformation relationship is equivalent to determining the three-dimensional position of the fire source area in the line-of-sight coordinate system. With the camera as the origin, a vector is generated to represent the direction and distance of the fire source, which is the relative position vector.

[0076] By combining the second distance with the second transformation relationship, the relative position vector of the fire source area in the line-of-sight coordinate system is calculated. The second transformation relationship is the transformation relationship between the line-of-sight coordinate system and the fire source coordinate system. It can be understood as a rotation matrix that converts the direction information of the fire source area from the line-of-sight coordinate system to its representation in the fire source coordinate system. Combining the second distance with this transformation relationship is equivalent to determining the specific position of the fire source area in the line-of-sight coordinate system, forming a vector representing the relative position of the fire source area in the line-of-sight coordinate system.

[0077] Next, using the first transformation relationship (i.e., the transformation relationship between the navigation coordinate system and the line-of-sight coordinate system, which is also a rotation matrix), the aforementioned relative position vector is transformed from the line-of-sight coordinate system to the navigation coordinate system. This step transforms the position information of the fire source area from a camera-centric perspective to a perspective of the UAV throughout the entire flight environment.

[0078] Finally, the transformed relative position vector is vector-added with the UAV's absolute position vector in the navigation coordinate system. The UAV's absolute position vector is its specific location in the navigation coordinate system. After vector addition, the absolute three-dimensional coordinates of the fire source area in the navigation coordinate system are obtained, thus accurately determining the specific location of the fire source.

[0079] More specifically, the altitudes of the fire-affected areas can be approximated as equal. This leads to the following relationship:

[0080] ;

[0081] ;

[0082] In the above formula The first distance information obtained by the laser rangefinder The altitude of the fire source. The height of the point where the optical axis of the line of sight points.

[0083] This is the transformation matrix from the line-of-sight coordinate system to the navigation coordinate system, i.e., the first transformation relationship.

[0084] ;

[0085] The second distance information from the camera to the fire source is the variable to be determined. It is the rotation matrix between the line-of-sight coordinate system and the fire source coordinate system, i.e., the second transformation relationship. This is a placeholder.

[0086] Therefore, based on the above equations and It can be solved Solve Therefore, the position of the fire source relative to the camera in the navigation coordinate system is:

[0087] ;

[0088] Since the drone's position is obtained in real time by the navigation coordinate system, the final location of the fire source is:

[0089] ;

[0090] The above This is the final absolute location information of the fire source. Let be the absolute position vector of the UAV.

[0091] Optionally, the first transformation relationship is used to represent a first rotation matrix of the spatial attitude of the line-of-sight coordinate system relative to the navigation coordinate system;

[0092] The second transformation relationship is used to represent the second rotation matrix that deviates the fire source coordinate system from the line-of-sight coordinate system.

[0093] In this invention, the first transformation relationship is a first rotation matrix used to represent the spatial attitude of the line-of-sight coordinate system relative to the navigation coordinate system. Its function is to transform the coordinates in the line-of-sight coordinate system to the navigation coordinate system. Simply put, this rotation matrix can be understood as describing the orientation of the camera's line of sight within the entire flight environment coordinate system (navigation coordinate system) of the UAV.

[0094] For example, when a drone is flying, the line-of-sight (i.e., the line-of-sight coordinate system) of its onboard camera changes with the drone's attitude. The navigation coordinate system is typically a fixed coordinate system with its origin at a specific point on the ground. The first rotation matrix can convert the position information of a point in the camera's line-of-sight coordinate system into its corresponding position in the navigation coordinate system. This allows the scene's position information "seen" by the camera to be mapped to the coordinate system of the entire drone's flight environment, facilitating subsequent position calculations and navigation applications.

[0095] The second transformation relation is a rotation matrix used to represent the deviation of the fire source coordinate system from the line-of-sight coordinate system. Its function is to transform the coordinates in the fire source coordinate system to the line-of-sight coordinate system. This rotation matrix primarily describes the angular offset of the fire source relative to the camera's line-of-sight direction.

[0096] In practical applications, after the camera captures an image, image processing algorithms can determine the position of the fire source in the image (i.e., the centroid coordinates of the fire source region). Based on this positional information, the offset angle of the fire source relative to the camera's line-of-sight center axis (i.e., the Z-axis of the line-of-sight coordinate system) can be calculated, including the fire source tilt angle and the fire source deflection angle. The second rotation matrix is ​​constructed based on these offset angles. It can convert the coordinates in the fire source coordinate system (a coordinate system with the camera's optical center as the origin and the line connecting the optical center to the fire source region as the Z-axis) to coordinates in the line-of-sight coordinate system. This allows the specific position of the fire source in the camera's line-of-sight coordinate system to be determined, thus providing a basis for further calculation of the absolute position of the fire source.

[0097] Optionally, the method for determining the second rotation matrix includes:

[0098] Based on the centroid coordinates of the fire source region, the image center coordinates of the image to be processed, and the camera intrinsic parameters, the fire source tilt angle and fire source deflection angle of the fire source region relative to the optical axis of the line-of-sight coordinate system are calculated.

[0099] The second rotation matrix is ​​generated based on the fire source tilt angle and fire source deflection angle.

[0100] In this invention, the second rotation matrix is ​​generated based on the tilt and deflection angles of the fire source region relative to the optical axis of the line-of-sight coordinate system. These two angles describe the positional deviation of the fire source region in the image, thereby helping to determine the angular relationship between the fire source coordinate system and the line-of-sight coordinate system.

[0101] First, these two angles are calculated using the centroid coordinates of the fire source region, the image center coordinates, and camera intrinsic parameters. The centroid coordinates of the fire source region are obtained from the image processing algorithm and reflect the position of the fire source in the image. By comparing these coordinates with the image center coordinates (i.e., the projection point of the camera's optical axis onto the image plane), the offset of the fire source in the image can be obtained.

[0102] By combining this with camera intrinsic parameters, this offset can be converted into an actual angle value. Camera intrinsic parameters contain information such as the camera's focal length and principal point coordinates, and are used to convert image planar coordinates into actual physical coordinates.

[0103] Using the calculated tilt and deflection angles of the fire source, a rotation matrix is ​​generated that can convert coordinates in the line-of-sight coordinate system to coordinates in the fire source coordinate system.

[0104] Specifically, first, a rotation matrix is ​​constructed to rotate the fire source tilt angle around the y-axis, then a rotation matrix is ​​constructed to rotate the fire source deflection angle around the z-axis. Finally, these two rotation matrices are multiplied to obtain the final second rotation matrix. This matrix can transform points in the line-of-sight coordinate system to the fire source coordinate system, thereby determining the specific angular direction of the fire source region relative to the camera's optical axis.

[0105] In this invention, by calculating the tilt and deflection angles of the fire source and generating a second rotation matrix, the orientation of the fire source region relative to the optical axis of the line-of-sight coordinate system can be accurately described. This helps to more accurately locate the fire source and determine its exact direction in space. Utilizing the centroid coordinates of the fire source region, the image center coordinates, and camera intrinsic parameters to calculate the angle, combined with image processing and geometric information, improves the accuracy of fire source positioning, making the positioning results more accurate and reliable. The second rotation matrix is ​​used to convert coordinates in the line-of-sight coordinate system to coordinates in the fire source coordinate system. An accurate matrix enables precise transformation between coordinate systems, providing a reliable foundation for subsequent positioning and navigation calculations and improving the performance of the entire positioning system.

[0106] Optionally, identifying the fire source region in the image to be processed and obtaining the centroid coordinates of the fire source region includes:

[0107] An object detection network is used to analyze the image to be processed, identify the fire source region, and obtain the bounding box of the fire source region;

[0108] Obtain the centroid coordinates of the boundary box of the fire source region to obtain the centroid coordinates of the fire source region.

[0109] In this invention, the object detection network is a deep learning-based algorithm capable of identifying and locating specific objects in an image. Specifically, we use object detection networks (such as U-Net and YOLO) to analyze images captured by a drone camera and identify fire source regions within them. The output of the object detection network typically includes a bounding box representing the fire source region. This bounding box is a rectangle that identifies the location of the fire source within the image. By using the bounding box, we can clearly pinpoint the exact location of the fire source in the image.

[0110] For example, the U-Net architecture is first constructed, which is an autoencoder structure consisting of an encoder and a decoder. The encoder captures image features, while the decoder reconstructs the segmented image. An appropriate depth and number of channels are chosen based on the complexity of the dataset and computational resources. The U-Net model is then trained using a training set to minimize the loss function for the segmentation task, typically pixel-level cross-entropy loss. A validation set is used to monitor model performance, perform hyperparameter tuning, and implement early stopping strategies. Finally, an appropriate threshold is applied to the segmentation mask to separate fire zones from non-fire zones.

[0111] After identifying the fire source region and obtaining its bounding box, it is necessary to calculate the centroid coordinates of the bounding box. The centroid coordinates are the coordinates of the center point of the bounding box and can be used to represent the center position of the fire source region.

[0112] The bounding box of the fire source region is a rectangle used to identify the location and extent of the fire source in the image. It iterates through each pixel within the region, obtaining its coordinates (x, y) for each pixel. The x and y coordinate values ​​of all pixels are accumulated. The total number of pixels within the region is calculated. The centroid coordinates (x_c, y_c) are obtained by dividing the accumulated x and y coordinate values ​​by the number of pixels, where x_c is the average of the x-coordinates and y_c is the average of the y-coordinates.

[0113] In this invention, by installing a laser rangefinder at the center of the camera, real-time synchronization of ranging and imaging data is achieved, ensuring the accuracy of fire source location. This technological improvement significantly enhances the system's positioning performance under different environmental conditions, making it more reliable in practical applications. It reduces reliance on complex gimbal systems, thereby significantly lowering system implementation and maintenance costs. The design is simple and easy to deploy, enabling the technology to be applied in more scenarios, especially in situations with limited resources.

[0114] Furthermore, this invention does not rely on strong assumptions, such as the consistency between the drone's takeoff altitude and the fire source altitude. This makes it more widely applicable in practical applications, especially in complex environments requiring dynamic monitoring, such as forest fire prevention, where drones often operate with significant variations in takeoff altitude. By avoiding these assumptions, this application can effectively cope with various terrain and environmental conditions, improving the universal applicability of fire source location.

[0115] The apparatus for rapid fire source positioning provided by the present invention will be described below. The apparatus for rapid fire source positioning described below can be referred to in correspondence with the method for rapid fire source positioning described above.

[0116] Figure 3 This is a schematic diagram of the device for rapid fire source positioning provided by the present invention, as shown below. Figure 3 As shown, it includes:

[0117] The acquisition module 310 is used to acquire the pose information of the drone equipped with a camera in the navigation coordinate system, and determine the first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information;

[0118] The recognition module 320 is used to acquire the image to be processed captured by the camera, as well as the first distance measured along the Z-axis of the line-of-sight coordinate system, and to identify the fire source area in the image to be processed, and obtain the centroid coordinates of the fire source area.

[0119] The determining module 330 is used to determine a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system based on the centroid coordinates of the fire source area;

[0120] The calculation module 340 is used to determine the absolute three-dimensional coordinates of the fire source area based on the first distance, the first transformation relationship and the second transformation relationship.

[0121] Optionally, the device is further used for:

[0122] Based on the first distance and the first conversion relationship, the fire source height of the fire source area is determined;

[0123] Based on the height of the fire source, and the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated;

[0124] Based on the second distance and the location information of the UAV, the absolute three-dimensional coordinates of the fire source area are determined.

[0125] Optionally, the device is further used for:

[0126] By combining the second distance with the second transformation relationship, the relative position vector of the fire source area in the line-of-sight coordinate system is calculated;

[0127] Using the first transformation relationship, the relative position vector is transformed to the navigation coordinate system;

[0128] The converted relative position vector is vector-added with the absolute position vector of the UAV in the navigation coordinate system to obtain the absolute three-dimensional coordinates of the fire source area.

[0129] Optionally, the device is further used for:

[0130] Based on the centroid coordinates of the fire source region, the image center coordinates of the image to be processed, and the camera intrinsic parameters, the fire source tilt angle and fire source deflection angle of the fire source region relative to the optical axis of the line-of-sight coordinate system are calculated.

[0131] The second rotation matrix is ​​generated based on the fire source tilt angle and fire source deflection angle.

[0132] Optionally, the device is further used for:

[0133] An object detection network is used to analyze the image to be processed, identify the fire source region, and obtain the bounding box of the fire source region;

[0134] Obtain the centroid coordinates of the boundary box of the fire source region to obtain the centroid coordinates of the fire source region.

[0135] In this invention, by fixing the laser rangefinder to the camera, the absolute three-dimensional coordinates of the fire source can be directly calculated using the pixel position of the fire source in a single frame image, combined with the UAV pose and a single laser rangefinder measurement, through coordinate system transformation and mathematical model solving, without the need for gimbal rotation and alignment. This method does not require the fire source to be located at the center of the image, thus avoiding the latency problem of remote control gimbals and significantly reducing the system's hardware dependence and cost.

[0136] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for rapid fire source localization. This method includes: acquiring the pose information of a camera-equipped UAV in a navigation coordinate system, and determining a first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information, wherein the line-of-sight coordinate system has the optical center of the camera as its origin and the optical axis as the Z-axis.

[0137] The image to be processed is acquired by the camera, and the first distance is measured along the Z-axis of the line-of-sight coordinate system. The fire source region in the image to be processed is identified, and the centroid coordinates of the fire source region are obtained.

[0138] Based on the centroid coordinates of the fire source area, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis;

[0139] The absolute three-dimensional coordinates of the fire source region are determined based on the first distance, the first transformation relationship, and the second transformation relationship.

[0140] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the method for rapid fire source location provided by the above methods, the method including: acquiring the pose information of a drone equipped with a camera in a navigation coordinate system, and determining a first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information, wherein the line-of-sight coordinate system has the optical center of the camera as the origin and the optical axis as the Z-axis;

[0142] The image to be processed is acquired by the camera, and the first distance is measured along the Z-axis of the line-of-sight coordinate system. The fire source region in the image to be processed is identified, and the centroid coordinates of the fire source region are obtained.

[0143] Based on the centroid coordinates of the fire source area, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis;

[0144] The absolute three-dimensional coordinates of the fire source region are determined based on the first distance, the first transformation relationship, and the second transformation relationship.

[0145] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for rapid fire source localization provided by the above methods. The method includes: acquiring pose information of a drone equipped with a camera in a navigation coordinate system, and determining a first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information, wherein the line-of-sight coordinate system has the optical center of the camera as the origin and the optical axis as the Z-axis.

[0146] The image to be processed is acquired by the camera, and the first distance is measured along the Z-axis of the line-of-sight coordinate system. The fire source region in the image to be processed is identified, and the centroid coordinates of the fire source region are obtained.

[0147] Based on the centroid coordinates of the fire source area, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis;

[0148] The absolute three-dimensional coordinates of the fire source region are determined based on the first distance, the first transformation relationship, and the second transformation relationship.

[0149] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0150] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapid fire source location, characterized in that, include: The pose information of the drone equipped with a camera in the navigation coordinate system is obtained, and a first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system is determined based on the pose information, wherein the line-of-sight coordinate system takes the optical center of the camera as the origin and the optical axis as the Z-axis. The image to be processed is acquired by the camera, and the first distance is measured along the Z-axis of the line-of-sight coordinate system. The fire source region in the image to be processed is identified, and the centroid coordinates of the fire source region are obtained. Based on the centroid coordinates of the fire source area, a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system is determined, wherein the fire source coordinate system takes the optical center of the camera as the origin and the line connecting the optical center to the fire source area as the Z-axis; Based on the first distance, the first transformation relationship, and the second transformation relationship, determine the absolute three-dimensional coordinates of the fire source area; The determination of the absolute three-dimensional coordinates of the fire source region based on the first distance, the first transformation relationship, and the second transformation relationship includes: Based on the first distance and the first conversion relationship, the fire source height of the fire source area is determined; Based on the height of the fire source, and the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated; Based on the second distance and the location information of the UAV, the absolute three-dimensional coordinates of the fire source area are determined; The determination of the absolute three-dimensional coordinates of the fire source area based on the second distance and the location information of the drone includes: By combining the second distance with the second transformation relationship, the relative position vector of the fire source area in the line-of-sight coordinate system is calculated; Using the first transformation relationship, the relative position vector is transformed to the navigation coordinate system; The converted relative position vector is vector-added with the absolute position vector of the UAV in the navigation coordinate system to obtain the absolute three-dimensional coordinates of the fire source area; Wherein, the first transformation relationship is used to represent the first rotation matrix of the spatial attitude of the line-of-sight coordinate system relative to the navigation coordinate system; The second transformation relationship is used to represent the second rotation matrix that deviates from the fire source coordinate system relative to the line-of-sight coordinate system; The method for determining the second rotation matrix includes: Based on the centroid coordinates of the fire source region, the image center coordinates of the image to be processed, and the camera intrinsic parameters, the fire source tilt angle and fire source deflection angle of the fire source region relative to the optical axis of the line-of-sight coordinate system are calculated. The second rotation matrix is ​​generated based on the fire source tilt angle and fire source deflection angle.

2. The method for rapid fire source location according to claim 1, characterized in that, Identifying the fire source region in the image to be processed and obtaining the centroid coordinates of the fire source region includes: An object detection network is used to analyze the image to be processed, identify the fire source region, and obtain the bounding box of the fire source region; Obtain the centroid coordinates of the boundary box of the fire source region to obtain the centroid coordinates of the fire source region.

3. A device for rapid fire source location, characterized in that, include: The acquisition module is used to acquire the pose information of the drone equipped with a camera in the navigation coordinate system, and determine the first transformation relationship between the navigation coordinate system and the line-of-sight coordinate system based on the pose information. The recognition module is used to acquire the image to be processed captured by the camera, as well as the first distance measured along the Z-axis of the line-of-sight coordinate system, and to identify the fire source area in the image to be processed, and obtain the centroid coordinates of the fire source area. The determination module is used to determine a second transformation relationship between the line-of-sight coordinate system and the fire source coordinate system based on the centroid coordinates of the fire source area; The calculation module is used to determine the absolute three-dimensional coordinates of the fire source area based on the first distance, the first transformation relationship, and the second transformation relationship; Based on the first distance, the first transformation relationship, and the second transformation relationship, the absolute three-dimensional coordinates of the fire source area are determined, including: Based on the first distance and the first conversion relationship, the fire source height of the fire source area is determined; Based on the height of the fire source, and the first and second transformation relationships, the second distance from the origin of the line-of-sight coordinate system to the fire source area is calculated; Based on the second distance and the location information of the UAV, the absolute three-dimensional coordinates of the fire source area are determined; The determination of the absolute three-dimensional coordinates of the fire source area based on the second distance and the location information of the drone includes: By combining the second distance with the second transformation relationship, the relative position vector of the fire source area in the line-of-sight coordinate system is calculated; Using the first transformation relationship, the relative position vector is transformed to the navigation coordinate system; The converted relative position vector is vector-added with the absolute position vector of the UAV in the navigation coordinate system to obtain the absolute three-dimensional coordinates of the fire source area; Wherein, the first transformation relationship is used to represent the first rotation matrix of the spatial attitude of the line-of-sight coordinate system relative to the navigation coordinate system; The second transformation relationship is used to represent the second rotation matrix that deviates from the fire source coordinate system relative to the line-of-sight coordinate system; The method for determining the second rotation matrix includes: Based on the centroid coordinates of the fire source region, the image center coordinates of the image to be processed, and the camera intrinsic parameters, the fire source tilt angle and fire source deflection angle of the fire source region relative to the optical axis of the line-of-sight coordinate system are calculated. The second rotation matrix is ​​generated based on the fire source tilt angle and fire source deflection angle.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for rapid fire source location as described in any one of claims 1 to 3.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for rapid fire source location as described in any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for rapid fire source location as described in any one of claims 1 to 2.

Images