A data acquisition method, device, medium, equipment and product of an aircraft

Abstract

The application relates to the technical field of computer vision, and particularly provides a data acquisition method and device of an aircraft, a medium, equipment and products, the method can comprise constructing a surrounding target function and a constraint condition of a fire field; wherein the surrounding target function represents a surrounding area formed by a surrounding track and the fire field; the constraint condition represents that a fire field outer contour is located on an inner side of the surrounding track; the surrounding target function is solved based on the constraint condition and a contour point of the fire field outer contour, so that the surrounding track and a speed direction of the aircraft around the fire field are obtained; wherein the surrounding track is a circular arc segment; in the process that the aircraft flies according to the surrounding track and the speed direction, a plurality of images of the fire field acquired by the aircraft are acquired; wherein the plurality of images are used for determining the area of the fire field. The embodiments of the application can realize automatic planning of a patrol track and direction of a fire field region, and provide support for effective measurement of the area of the fire field.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a data acquisition method, apparatus, medium, equipment, and product for an aircraft. Background Technology

[0002] To ensure safety in the field or large areas, areas prone to fire are typically inspected. This allows for timely intervention in the event of a fire, mitigating the associated risks. Currently, for fires in the field or factory areas, ground-based handheld devices are commonly used to measure the fire area as quickly as possible. However, this method has a limited field of view, and the accuracy of the measurement cannot be guaranteed. In response, existing technologies have proposed using drones for aerial photography. This method often employs single-frame segmentation and proportional conversion to estimate the fire area, but it is sensitive to errors in height and tilt angle. Furthermore, multi-frame fusion lacks strict imaging geometric constraints, leading to boundary drift and preventing precise localization of the fire's extent.

[0003] Therefore, how to provide a technical solution for a highly accurate data acquisition method for aircraft has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of some embodiments of this application is to provide a data acquisition method, device, medium, equipment and product for aircraft. The technical solutions of the embodiments of this application can improve the accuracy and reliability of fire scene range positioning, and provide reliable decision-making basis for aircraft fire extinguishing bomb throwing planning, personnel mobilization and on-site command.

[0005] In a first aspect, some embodiments of this application provide a data acquisition method for an aircraft, comprising: constructing an orbital objective function and constraints for a fire field; wherein the orbital objective function characterizes the orbital area constructed by the orbital trajectory and the fire field; the constraints characterize the outer contour of the fire field located inside the orbital trajectory; solving the orbital objective function based on the constraints and the contour points of the outer contour of the fire field to obtain the orbital trajectory and velocity direction of the aircraft around the fire field; wherein the orbital trajectory is an arc segment; the velocity direction is determined by the arc mid-angle of the orbital trajectory and the orbital radius of the orbital trajectory; and acquiring multiple frames of images of the fire field collected by the aircraft during the flight of the aircraft according to the orbital trajectory and the velocity direction.

[0006] Some embodiments of this application obtain the orbital trajectory and velocity direction output by the orbital objective function by combining the contour points of the outer contour of the fire field with the orbital objective function and constraints. This enables precise planning of the orbital trajectory and direction of the aircraft, thereby allowing the aircraft to acquire multiple frames of images of the fire field and providing support for accurate positioning of the fire area.

[0007] In some embodiments, the velocity direction is obtained by the following steps: obtaining the mid-arc angle of the orbital trajectory and determining the tangential unit vector and normal unit vector corresponding to the mid-arc angle; determining the velocity direction based on the tangential unit vector, the normal unit vector, and the orbital radius of the orbital trajectory.

[0008] Some embodiments of this application determine the velocity direction by using the tangential unit vector, the normal unit vector, and the orbital radius at the midpoint of the orbital arc, thus achieving precise planning of the flight direction.

[0009] In some embodiments, after acquiring multiple frames of images of the fire scene captured by the aircraft, the method further includes: determining a closed polygon corresponding to the fire scene using a set of points in the multiple frames of images; and calculating the area of ​​the closed polygon to obtain the area of ​​the fire scene.

[0010] This application embodiment analyzes the point set in multiple frames of images to form a closed quadrilateral. This geometric constraint can improve the accuracy and reliability of fire area positioning.

[0011] In some embodiments, determining the closed polygon corresponding to the fire site using the point set in the multi-frame images includes: obtaining a ground point set after projecting the fire site's outer contour point set from the multi-frame images onto the ground; removing discrete points from the ground point set to obtain a removed point set; and constructing the closed polygon corresponding to the removed point set.

[0012] Some embodiments of this application obtain closed polygons by projecting the fire area's outer contour points from multiple frames of images onto the ground, removing discrete points, and constructing a corresponding set of polyline segments. This achieves geometric constraints on the fire area and provides accurate data support for subsequent calculations of the fire area.

[0013] In some embodiments, obtaining the ground point set after projecting the fire field outer contour point set from the multi-frame images onto the ground includes: extracting the maximum connected component outer contour point set of the fire field in each frame of the multi-frame images; projecting the fire field outer contour point set in each frame of the images onto the ground, and combining it with the maximum connected component outer contour point set to obtain the projection point set of each frame of the images; the projection point set of the multi-frame images constitutes the ground point set.

[0014] Some embodiments of this application determine the ground point set by extracting the largest connected outer contour point set in each frame of the image and combining it with the result of projecting the fire field outer contour point set onto the ground, thus providing support for the subsequent construction of closed polygons.

[0015] In some embodiments, the step of removing discrete points from the ground point set to obtain a set of points after removal includes: constructing a circular region centered on any point in the ground point set; confirming that the number of ground points in the circular region is less than a point count threshold, then taking the any point as the discrete point; or, using a proximity algorithm to analyze the nearest neighbor distance of the any point to determine the discrete point; removing the discrete points from the ground point set to obtain the set of points after removal.

[0016] Some embodiments of this application reduce the interference of noise points (or isolated points) by identifying and removing discrete points from the ground point set to obtain a point set after removal.

[0017] In some embodiments, constructing the closed polygon corresponding to the removed point set includes: performing optimal triangulation on the removed point set to obtain a candidate boundary set; concatenating the edges in the candidate boundary set to obtain an initial line segment set; reconnecting the initial line segment set to obtain a reconnected initial line segment set; and optimizing the reconnected initial line segment set to obtain the closed polygon.

[0018] Some embodiments of this application obtain a set of polyline segments by performing optimal triangulation, edge splicing, reconnection, and optimization on the removed point set, thereby achieving effective data processing and providing support for obtaining effective closed quadrilaterals.

[0019] In some embodiments, reconnecting the initial set of line segments to obtain a reconnected initial set of line segments includes: determining candidate stitch pairs formed by any two endpoints in the initial set of line segments; merging target candidate stitch pairs whose angles satisfy an angle threshold into the edge of the fire field; wherein the edge of the fire field is a polyline segment; and constructing the reconnected initial set of line segments based on multiple polyline segments.

[0020] Some embodiments of this application obtain an initial set of polyline segments by analyzing the line segments formed by any two endpoints in the initial set of line segments; then, the polyline segments in the initial set of polyline segments are optimized to obtain an ordered, single, closed quadrilateral.

[0021] In some embodiments, optimizing the reconnected initial line segment set to obtain the closed polygon includes: identifying and filling holes in the reconnected initial line segment set, and then removing pseudo-loops in the reconnected initial line segment set; wherein, the pseudo-loop is a loop or inner loop formed by broken line segments with an area less than an area threshold; and sorting the points in the reconnected initial line segment set to obtain the closed polygon.

[0022] Secondly, some embodiments of this application provide a data acquisition device for an aircraft, comprising: a construction module for constructing an orbital objective function and constraints for a fire field; wherein the orbital objective function characterizes the orbital area constructed by the orbital trajectory and the fire field; the constraints characterize the outer contour of the fire field located inside the orbital trajectory; a solution module for solving the orbital objective function based on the constraints and the contour points of the outer contour of the fire field to obtain the orbital trajectory and velocity direction of the aircraft around the fire field; wherein the orbital trajectory is an arc segment; the velocity direction is determined by the arc mid-angle of the orbital trajectory and the orbital radius of the orbital trajectory; and a determination module for acquiring multiple frames of images of the fire field acquired by the aircraft during the flight of the aircraft according to the orbital trajectory and the velocity direction.

[0023] Thirdly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0024] Fourthly, some embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the method as described in any embodiment of the first aspect.

[0025] Fifthly, some embodiments of this application provide a computer program product, the computer program product including a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of some embodiments of this application, the accompanying drawings used in some embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 Flowcharts of a data acquisition method for an aircraft provided for some embodiments of this application;

[0028] Figure 2 Schematic diagram of the optimal arc segment within a fire zone provided for some embodiments of this application;

[0029] Figure 3 Flowcharts of methods for determining the extent of a fire scene provided for some embodiments of this application;

[0030] Figure 4 Block diagrams of data acquisition devices for aircraft provided for some embodiments of this application;

[0031] Figure 5 A schematic diagram of an electronic device provided for some embodiments of this application. Detailed Implementation

[0032] The technical solutions of some embodiments of this application will now be described with reference to the accompanying drawings.

[0033] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0034] Among related technologies, the main rapid mapping methods for field or factory fires include the following: one is to use ground-based handheld infrared or visible equipment for mapping, but this method has a limited field of view, is easily affected by heat waves and smoke, and has high requirements for personnel safety; another is to use fixed-wing or multi-rotor drones to take aerial photos, which often uses single-frame segmentation and scale conversion to estimate the area, but is sensitive to height and tilt angle errors; and multi-frame fusion lacks strict imaging geometric constraints, resulting in boundary drift and making it impossible to achieve accurate mapping; another method is to use thermal infrared equipment and temperature threshold segmentation, but this method is prone to false positives in strong convection and high background temperatures, and requires time-series robustness and terrain constraints.

[0035] As can be seen from the above-mentioned technologies, the existing methods for rapid mapping of fire areas cannot accurately measure the fire area, resulting in poor reliability and affecting the accuracy and efficiency of rescue efforts.

[0036] In view of this, some embodiments of this application provide a data acquisition method for an aircraft. This method can plan the aircraft's orbital trajectory and velocity direction, and then, as the aircraft flies accordingly, it can acquire multiple frames of images of the fire scene. The point sets in the multiple frames are then processed to construct a closed polygon; finally, the area of ​​the closed polygon is calculated to obtain the fire area. The data acquisition method provided in this application can achieve automated orbital inspection of the fire area. After acquiring multiple frames of images, the range is calculated by fusing the multi-frame image data to obtain a precise and reliable fire area, providing a reliable decision-making basis for aircraft fire extinguishing bomb deployment planning, personnel mobilization, and on-site command.

[0037] The following is in conjunction with the appendix Figure 1The present application provides exemplary methods for implementing data acquisition of a fire-fighting aircraft through some embodiments. This data acquisition process can be executed by a processor mounted on the aircraft or by a terminal connected to the aircraft; the aircraft can be a fixed-wing or multi-rotor type of unmanned aerial vehicle (UAV). It should be understood that the embodiments of this application do not specifically limit the implementation process of data acquisition or the type of aircraft.

[0038] To facilitate the explanation of the implementation process of this application, an example of a drone will be used below for illustrative purposes.

[0039] Please see the appendix Figure 1 , Figure 1 A flowchart illustrating a data acquisition method for an aircraft, provided for some embodiments of this application. The data acquisition method for the aircraft may include:

[0040] S110, construct the objective function and constraints for the fire field's encirclement; wherein, the objective function represents the encirclement area of ​​the encirclement trajectory and the fire field; the constraints represent that the outer contour of the fire field is located inside the encirclement trajectory.

[0041] For example, in a specific embodiment of this application, a target function (hereinafter referred to as the target function) and inner conditions (as a specific example of constraint conditions) are constructed in the orbital trajectory module. The inner constraints are constructed as follows: an arc segment is described by the center o, radius R, and angle interval [θs, θe], i.e., arc segment A(o, R, θs, θe) = {o + R[cosθ, sinθ]}. T : θ=[θs, θe]}. The normal direction of this arc segment (pointing to the center of the circle) is:

[0042] ;

[0043] Contour subset covered by candidate arc segments C loc The fire site must be located inside the arc segment, and the inner constraint is as follows: ,σ(x) ∈ {+1, 1}: The "inner / outer" discrimination result of the contour points (+1 indicates that the fire is on the normal side);

[0044] The objective function is constructed by defining the surrounding region formed by the contour sub-polygon H and the arc closure within the local region of interest. The optimization objective is to minimize the surrounding area: The polygon area calculation function Area(·) can be used in engineering by discretizing the arc segment H into a polygonal line, forming a union of the polygonal lines with H, and then calculating the area using the shoelace formula. However, since lower precision is required here, the closed area can be discretized into n points. ,in (i=k) represents the coordinates of a discrete point in the pixel coordinate system. The final objective function is A= argminJ ( A ).

[0045] S120, based on the constraints and the contour points of the fire scene's outer contour, solve the orbital objective function to obtain the orbital trajectory and the velocity direction; wherein, the orbital trajectory is a circular arc segment.

[0046] For example, in a specific embodiment of this application, the contour points C of the fire scene outer contour (image plane or projection onto the ground coordinate plane) obtained by segmentation by a monocular camera are input into the orbital trajectory module, and the inner constraints in the orbital trajectory module are... This allows for the "inner / outer" discrimination of the contour point x (+1 indicates the fire is on the normal side); by solving the objective function, the orbital trajectory module can output the optimal arc segment. (As a specific example of an orbital trajectory) and the desired horizontal resultant velocity vector v des (As a specific example of the direction of velocity); specifically, as follows Figure 2 The diagram shows the optimal arc segment.

[0047] The velocity direction is obtained through the following steps: obtaining the mid-arc angle of the orbital trajectory and determining the tangential unit vector and normal unit vector corresponding to the mid-arc angle; determining the velocity direction based on the tangential unit vector, the normal unit vector, and the orbital radius of the orbital trajectory.

[0048] For example, in a specific embodiment of this application, the desired horizontal velocity vector of the UAV is obtained through tangential guidance at the midpoint of the arc and radius error feedback. Specifically, its optimal arc midpoint angle (as a specific example of the arc midpoint angle) is... The location of the midpoint The tangential unit vector is: Normal unit vector Define the desired horizontal net velocity: ; where v d For the desired tangential velocity, R d For the desired radius of circumference, k r >0 represents the radius error feedback gain.

[0049] The purpose of obtaining this desired horizontal resultant velocity vector is to: use the tangential velocity vector to determine whether the UAV can fly around the fire area, thereby acquiring multi-frame data (i.e., multi-frame images) through a monocular camera; and use the normal unit vector and the desired orbital radius R. d Constraints ensure that the drone remains outside the fire zone, preventing airflow caused by the fire from causing it to go out of control. R... dThe selection can be set to + C, where C>0 represents a configurable safe range. Simultaneously, to ensure the boundedness of the velocity command (i.e., the desired horizontal resultant velocity vector), consistency of the input direction needs to be achieved after amplitude limiting, and this is achieved through the intrinsic parameter matrix K and the rotation matrix R from the camera coordinate system to the UAV body coordinate system. c2b The rotation matrix R from the UAV body coordinate system to the inertial coordinate system (NED) b2n The speed command is then converted. Furthermore, the drone's heading angle needs to be constrained: if the angle between the drone's current heading and the heading of the desired speed command is within ±90 degrees, the speed command is executed; otherwise, the speed command is reversed. This ensures that the drone can circle the fire area. Additionally, the drone's flight speed can be flexibly set according to actual needs.

[0050] S130: During the flight of the aircraft according to the predetermined orbital trajectory and speed direction, the aircraft acquires multiple frames of images of the fire scene.

[0051] For example, in a specific embodiment of this application, the drone, while flying along a circular trajectory and speed direction, acquires images of the fire area through its onboard camera, obtaining multiple frames of images.

[0052] In some embodiments of this application, after executing S130, the data acquisition method of the aircraft may include:

[0053] S140, using the point set in the multi-frame images, determine the closed polygon corresponding to the fire site.

[0054] For example, in a specific embodiment of this application, the points at the edge of the fire are first identified frame by frame and projected onto the ground coordinate system for the multiple frames of images obtained during the drone's orbital flight. Then, data filtering and geometric fusion are performed, and a single closed polygon G is obtained after multi-frame incremental fusion.

[0055] In some embodiments of this application, S140 may include:

[0056] S141, Obtain the ground point set after projecting the fire scene outer contour point set from multiple frames of images onto the ground.

[0057] For example, in a specific embodiment of this application, a ground point set is obtained by analyzing and projecting each frame of a multi-frame image.

[0058] In some embodiments of this application, S141 may include: extracting the maximum connected domain outer contour point set of the fire field in each of the multiple frames of images; projecting the fire field outer contour point set in each frame of images onto the ground, and combining it with the maximum connected domain outer contour point set to obtain the projection point set of each frame of images; the projection point set of the multiple frames of images constitutes the ground point set.

[0059] For example, in a specific embodiment of this application, the process of obtaining the projection point set is illustrated using any frame (e.g., frame t) from a multi-frame image. Specifically, the process of obtaining the frame t image I is described. t Camera intrinsic parameters K, rotation matrix R c2b and R b2n Image I t The segmentation process yields a binary mask, which is then used to extract the set of outer contour points C of the largest connected component. t ={(u k ,v k Then, using the imaging geometric projection method, image I is... t The fire scene's outer contour point set is projected onto the ground to obtain the projection point set for frame t. P t :

[0060] ;

[0061] in, These are the coordinates of the camera position in the UAV's body coordinate system. for The z-axis component is the camera's height relative to the drone in the NED coordinate system; H AGL d represents the ground altitude of the UAV in the NED coordinate system. gz For d g The z-axis component.

[0062] By processing all the frame images taken by the UAV, a ground point set is obtained. .

[0063] S142, the discrete points in the ground point set are removed to obtain the point set after removal.

[0064] For example, in a specific embodiment of this application, after obtaining the ground point set P through the above processing, discrete and outlier points (collectively referred to as discrete points) are removed to facilitate the subsequent acquisition of relevant boundary line segments.

[0065] In some embodiments of this application, S142 may include: constructing a circular region with any point in the set of ground points as the center; if the number of ground points in the circular region is less than a point number threshold, then taking the any point as the discrete point; or, using a proximity algorithm to analyze the proximity distance of the any point to determine the discrete point; removing the discrete points in the set of ground points to obtain the removed point set.

[0066] For example, in a specific embodiment of this application, the ground grid resolution is set to g(m), and the radius neighborhood is defined. It represents that the neighborhood satisfies the distance p i Point p less than r j Where r = 2g. Let point p... i We filter circles with center i (i.e., taking i as any point) and radius r, and if the number of points inside the circle is satisfied... Then point p i It is determined to be an isolated point (i.e., a discrete point). Where k min This is the minimum number of points threshold, which can be set according to actual needs. Alternatively, the KNN distance algorithm (as a concrete example of a nearest neighbor algorithm) can be used to identify discrete points. Specifically, let the distance to the Kth nearest neighbor be denoted as... With all the median of The median absolute deviation (MAD) is used as the discrimination criterion; if > +2MAD, then this point is treated as a discrete point. Where MAD = median(| - |) represents the Median Absolute Deviation (MAD). The discrete points identified above are then removed, resulting in the set of points P after removal. .

[0067] S143, construct the closed polygon corresponding to the removed point set.

[0068] For example, in a specific embodiment of this application, after removing the corresponding discrete points, a set of corresponding polyline segments is constructed to facilitate the acquisition of closed polygons.

[0069] In some embodiments of this application, S143 may include:

[0070] S1431, Perform optimal triangulation on the removed point set to obtain a candidate boundary set; concatenate the edges in the candidate boundary set to obtain an initial line segment set.

[0071] For example, in a specific embodiment of this application, the candidate boundary set is first confirmed. Specifically, the initial boundary is determined by α-shape, and the parameter α is set to control the boundary detail scale (m), which is related to the flight altitude and sampling density. For example, its empirical value range is: α ∈ [1.0, 5.0]. For P Construct a Delaunay triangulation T, preserving the circumcircle radius. The triangle (and its sides) form α The complex is defined by taking the edges "used only by a single triangle" as the candidate boundary set E, and concatenating them to obtain several open or closed zigzag segment sets L (as a specific example of the initial segment set). Here, α-shape is the boundary composed of a subset of simplexes satisfying the scale condition in the Delaunay triangulation of the point set. For each simplex (e.g., point / edge / triangle / tetrahedron), the radius of its circumcircle or circumsphere is calculated, and simplexes with circumcircle radii less than a threshold (i.e., triangles in the above example) are retained.

[0072] S1432, the initial set of line segments is reconnected to obtain a reconnected initial set of line segments; the reconnected initial set of line segments is optimized to obtain a closed polygon.

[0073] For example, in a specific embodiment of this application, since the L obtained above may contain broken segments and small holes, subsequent reconnection and cleaning are required to obtain a closed polygon.

[0074] In some embodiments of this application, S1432 may include: determining candidate stitch pairs formed by any two endpoints in the initial line segment set; merging target candidate stitch pairs whose angles satisfy an angle threshold into the edge of the fire field; wherein the edge of the fire field is a polyline segment; and constructing the reconnected initial line segment set based on multiple polyline segments.

[0075] The optimization process for the reconnected initial line segment set to obtain the closed polygon includes: identifying and filling holes in the reconnected initial line segment set, and removing pseudo-loops in the reconnected initial line segment set; wherein, the pseudo-loop is a loop or inner loop formed by broken line segments with an area less than an area threshold; and sorting the points in the reconnected initial line segment set to obtain the closed polygon.

[0076] For example, in a specific embodiment of this application, the set of open-fold line segments in L is set as {S}. m}, the two endpoints of each broken line segment are a m and b m For any two endpoints ,when The line segment formed by any two endpoints is included in the candidate (i.e., candidate stitch pair), where the distance threshold is... Let the tangent vector of the polyline segment at its endpoints be... and ,like Then, it is detected through angle continuity; among which, This is the angle threshold, which can be set according to actual conditions. After the angle requirements are met, This is added to the boundary graph as a new edge (as a concrete example of the initial set of polyline segments after reconnection). The above constraints effectively suppress erroneous connections that "cross holes or traverse the inside". Then, a minimum area threshold A is used... min Small holes in the boundary map are filtered out; for area A hole min Small holes are filled directly. If the boundaries intersect, a sweeping motion is used to divide the polygons at the intersection points, retaining the largest outer ring and removing inner rings or pseudo-rings with too small an area (e.g., less than the area threshold). To ensure that the output vertices are sorted counterclockwise (CCW), the directed area of ​​the polygons is used as needed. The correction meets the requirements, and the output is a reasonable closed polygon G.

[0077] If new batches of points are subsequently collected, T and T can be updated within the aforementioned Delaunay conflict domain. To avoid full reconstruction every time, only local reconnection and purification of the affected segments are performed.

[0078] In addition, before calculating the fire area, it can be determined whether the drone has completed its circumnavigation. For example, changes in the drone's status and data collected by the camera can be used to determine whether the fire scene circumnavigation has been completed. For instance, confirming changes in the drone's heading angle. When the maximum boundary gap gap of the outer contour G after the α-shape boundary reconnection is ≤3g (g is the ground resolution / grid side length) and there is no self-intersection, the wrapping is confirmed to be complete.

[0079] S150, calculate the area of ​​the closed polygon to obtain the fire area.

[0080] For example, in a specific embodiment of this application, after determining the surrounding area, the fire area is obtained by solving the area of ​​the closed polygon using the shoelace formula. This facilitates resource scheduling. For example, if the vertex sequence of G is v1, ... v N v N+1 =v1, the shoelace formula is:

[0081] .

[0082] The following is in conjunction with the appendix​ Figure 3 The implementation process of determining the fire area provided by some embodiments of this application is illustrated by way of example.

[0083] Please see the appendix Figure 3 , Figure 3 A flowchart illustrating a method for determining the extent of a fire scene, provided for some embodiments of this application.

[0084] The above process is illustrated below by example.

[0085] S310: Obtain the contour points of the fire scene outer contour obtained by image segmentation from a monocular camera.

[0086] S320 inputs the contour points of the fire scene's outer outline into the orbital trajectory module and outputs the optimal arc segment and velocity direction.

[0087] The orbital trajectory module includes the orbital objective function and constraints.

[0088] The S330 acquires multiple frames of images of the fire scene as the drone flies along the optimal arc and speed direction.

[0089] S340, acquire the ground point set after projecting the fire scene outline point set from multiple frames of images onto the ground.

[0090] S350 removes discrete points from the ground point set to obtain the point set after removal.

[0091] S360, based on the point set after removal, uses the α-shape method to construct the initial boundary, resulting in a set of several polyline segments.

[0092] S370 performs boundary reconnection, hole filling, self-intersection resolution, and direction consistency processing on the polyline segments in a set of polyline segments to obtain closed polygons.

[0093] S380, solve for the area of ​​the closed polygon to obtain the fire area.

[0094] It is understood that the specific implementation process of S310~S380 can be referred to the method embodiment provided above. To avoid repetition, detailed descriptions are omitted here.

[0095] Please refer to Figure 4 , Figure 4 The diagram illustrates a block diagram of a data acquisition device for an aircraft provided in some embodiments of this application. It should be understood that this data acquisition device corresponds to the method embodiments described above and is capable of performing the various steps involved in the method embodiments. The specific functions of this data acquisition device can be found in the description above; detailed descriptions are omitted here to avoid repetition.

[0096] Figure 4The data acquisition device of the aircraft includes at least one software functional module that can be stored in a memory or embedded in the data acquisition device of the aircraft in the form of software or firmware. The data acquisition device of the aircraft includes: a construction module 410, used to construct the orbital objective function and constraints of the fire field; wherein, the orbital objective function represents the orbital trajectory and the orbital area of ​​the fire field; the constraints represent that the outer contour of the fire field is located inside the orbital trajectory; a solution module 420, used to solve the orbital objective function based on the constraints and the contour points of the outer contour of the fire field to obtain the orbital trajectory and velocity direction of the aircraft around the fire field; wherein, the orbital trajectory is a circular arc segment; and a determination module 430, used to acquire multiple frames of images of the fire field acquired by the aircraft during the flight of the aircraft according to the orbital trajectory and the velocity direction.

[0097] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0098] Some embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can perform the operation of any of the methods corresponding to the methods provided in the above embodiments.

[0099] Some embodiments of this application also provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operation of any of the methods corresponding to the above embodiments provided in the above embodiments.

[0100] like Figure 5 As shown, some embodiments of this application provide an electronic device 500, which includes a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520. When the processor 520 reads the program from the memory 510 via a bus 530 and executes the program, it can implement the methods of any of the above embodiments.

[0101] Processor 520 can process digital signals and can include various computing architectures. For example, it can be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 520 can be a microprocessor.

[0102] The memory 510 can be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 520 of this disclosure embodiment can be used to execute the instructions in the memory 510 to implement the methods shown above. The memory 510 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0103] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0105] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data acquisition method for an aircraft, characterized in that, include: Construct a fire scene encirclement objective function and constraints; wherein, the encirclement objective function represents the encirclement area of ​​the encirclement trajectory and the fire scene; the constraints represent that the outer contour of the fire scene is located inside the encirclement trajectory; the optimization objective of the encirclement objective function is to minimize the encirclement area; the encirclement area is obtained by discretizing the arc segment into a polygonal line, forming a plane polygon union with the arc segment, and then using the shoelace formula; The orbital objective function is solved based on the constraints and the contour points of the fire area to obtain the orbital trajectory and velocity direction of the aircraft around the fire area; wherein the orbital trajectory is a circular arc segment; the velocity direction is determined by the mid-arc angle of the orbital trajectory and the orbital radius of the orbital trajectory. During the flight of the aircraft in accordance with the orbital trajectory and the speed direction, multiple frames of images of the fire site are acquired by the aircraft.

2. The method as described in claim 1, characterized in that, The velocity direction is obtained through the following steps: Obtain the mid-arc angle of the orbital trajectory, and determine the tangential unit vector and normal unit vector corresponding to the mid-arc angle; The velocity direction is determined based on the tangential unit vector, the normal unit vector, and the radius of the orbital trajectory.

3. The method according to any one of claims 1-2, characterized in that, After acquiring the multiple frames of images of the fire scene captured by the aircraft, the method further includes: The closed polygon corresponding to the fire site is determined by the point set in the multi-frame images; The area of ​​the fire zone is obtained by solving for the area of ​​the closed polygon.

4. The method as described in claim 3, characterized in that, The step of determining the closed polygon corresponding to the fire scene using the point set in the multi-frame images includes: Obtain the ground point set after projecting the fire scene outer contour point set from the multi-frame images onto the ground; The discrete points in the ground point set are removed to obtain the point set after removal; Construct the closed polygon corresponding to the set of points after the removal.

5. The method as described in claim 4, characterized in that, The step of obtaining the ground point set after projecting the fire scene outline point set from the multi-frame images onto the ground includes: Extract the set of the largest connected component out-of-contact contour points of the fire scene in each of the multiple frames of images; The fire scene outline point set in each frame image is projected onto the ground, and combined with the outline point set of the largest connected region, to obtain the projection point set of each frame image; the projection point sets of the multiple frames image constitute the ground point set.

6. The method as described in claim 4, characterized in that, The step of removing discrete points from the ground point set to obtain a set of points after removal includes: Construct a circular region with any point in the set of ground points as the center; If the number of ground points within the circular area is less than a point count threshold, then any one of those points is taken as the discrete point; or, the discrete point is determined by analyzing the nearest neighbor distances of any one of those points using a proximity algorithm. The discrete points in the ground point set are removed to obtain the point set after removal.

7. The method as described in claim 5 or 6, characterized in that, The construction of the closed polygon corresponding to the removed point set includes: Perform optimal triangulation on the removed point set to obtain a candidate boundary set; By concatenating the edges in the candidate boundary set, an initial set of line segments is obtained; The initial set of line segments is reconnected to obtain a reconnected initial set of line segments; The initial set of reconnected line segments is optimized to obtain the closed polygon.

8. The method as described in claim 7, characterized in that, The step of reconnecting the initial set of line segments to obtain a reconnected initial set of line segments includes: Determine candidate stitch pairs formed by any two endpoints in the initial set of line segments; The candidate suture pairs whose angles satisfy the angle threshold are merged into the edge of the fire field; wherein, the edge of the fire field is a polyline segment; Based on multiple polyline segments, construct the initial set of line segments after reconnection.

9. The method as described in claim 7, characterized in that, The optimization process of the reconnected initial line segment set to obtain the closed polygon includes: After identifying and filling the holes in the reconnected initial line segment set, remove the pseudo-loops in the reconnected initial line segment set; wherein, the pseudo-loop is a loop or inner loop formed by broken line segments with an area smaller than an area threshold; The points in the initial set of reconnected line segments are sorted to obtain the closed polygon.

10. A data acquisition device for an aircraft, characterized in that, include: A construction module is used to construct the objective function and constraints of the fire field's encirclement; wherein, the objective function represents the encirclement area constructed by the encirclement trajectory and the fire field; the constraints represent that the outer contour of the fire field is located inside the encirclement trajectory; the optimization objective of the objective function is to minimize the encirclement area; the encirclement area is obtained by discretizing the arc segment into a polygonal line, forming a plane polygon union with the arc segment, and then calculating it using the shoelace formula; The solution module is used to solve the orbital objective function based on the constraints and the contour points of the outer contour of the fire field, to obtain the orbital trajectory and velocity direction of the aircraft around the fire field; wherein, the orbital trajectory is a circular arc segment; the velocity direction is determined by the mid-arc angle of the orbital trajectory and the orbital radius of the orbital trajectory; The determination module is used to acquire multiple frames of images of the fire site collected by the aircraft during the flight of the aircraft according to the orbital trajectory and the speed direction.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to perform the method as described in any one of claims 1-9.

12. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program is executed by the processor to perform the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program, wherein the computer program is executed by a processor to perform the method as described in any one of claims 1-9.

Images