Fire fighting access occupation detection method and system based on unmanned aerial vehicle cluster

Abstract

The invention discloses a fire fighting access occupation detection method and system based on an unmanned aerial vehicle cluster. The method comprises the following steps: controlling the unmanned aerial vehicle cluster to fly; in the unmanned aerial vehicle inspection process, ground images are obtained in real time through imaging equipment carried on an unmanned aerial vehicle holder; identifying the fire fighting access and the motor vehicle in the same ground image, outputting segmentation information of the fire fighting access, and obtaining segmentation mask information of the fire fighting access according to the segmentation information of the fire fighting access; and outputting position information of the motor vehicle in the ground image, and judging whether the motor vehicle is occupied by the fire fighting access according to the segmentation mask of the fire fighting access and the position information of the motor vehicle. On the basis of the unmanned aerial vehicle cluster, the segmentation algorithm and the target detection technology, the ground image is automatically analyzed, and the fire fighting access and the occupied motor vehicle are identified, so that the detection efficiency can be greatly improved, the human input is reduced, and the detection cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to a fire passage occupancy detection method and system based on a drone cluster. Background Art

[0002] In the prior art, there are technical solutions for detecting fire passage occupancy based on drones, and technical solutions for detecting illegal occupation of emergency lanes. For example, application number 202410362084.7 and patent name "AI detection method and device for illegal occupation of emergency lanes based on drones" disclose a similar solution, which requires a pre-trained multi-task detection model to perform vehicle detection tasks and lane segmentation tasks on inspection images. As a result, its calculation process is complicated and resource consumption is large, and the drone shooting angle is not taken into account, resulting in serious misjudgment and missed detection results. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a fire passage occupancy detection method and system based on drone clusters, which automatically analyzes ground images based on drone clusters, segmentation algorithms, and target detection technology to identify fire passages and occupied motor vehicles, thereby greatly improving detection efficiency, reducing manpower input, and reducing detection costs.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] On the one hand, a method for detecting fire passage occupancy based on a drone cluster is provided, which includes:

[0006] Control the flight of the drone swarm so that each drone in the drone swarm inspects the same designated area according to the corresponding cruising altitude and route;

[0007] During the drone inspection process, the imaging device on the drone gimbal acquires ground images in real time;

[0008] Respectively identifying the fire passage and the motor vehicle in the same ground image, and outputting the segmentation information of the fire passage, and obtaining the segmentation mask information of the fire passage according to the segmentation information of the fire passage; and outputting the position information of the motor vehicle in the ground image;

[0009] It is determined whether a motor vehicle occupies the fire passage based on the segmentation mask of the fire passage and the position information of the motor vehicle.

[0010] Preferably, judging whether a motor vehicle occupies a fire passage according to the segmentation mask of the fire passage and the position information of the motor vehicle comprises the following steps:

[0011] Determining whether the motor vehicle occupies the fire passage in the ground image according to the segmentation mask information of the fire passage in the same ground image and the position information of the motor vehicle;

[0012] Determine whether the same motor vehicle occupies the same fire lane in ground images obtained by different drones;

[0013] All the judgment results obtained are summarized and a final determination is made through voting whether a motor vehicle occupies the fire passage.

[0014] Preferably, if the fire passage is finally considered to be occupied by a motor vehicle, the fire passage occupancy information is sent to the monitoring end and an alarm message is generated.

[0015] Preferably, obtaining the segmentation mask information of the fire passage according to the segmentation information of the fire passage in the ground image comprises the following steps:

[0016] The segmentation mask area S of the fire passage is calculated based on the segmentation information of the fire passage in the ground image. M , and obtain the center point coordinates P of the segmentation mask area m =(x m ,y m ),and

[0017]

[0018] Among them, x max 、x min ,y max and min Represent the segmentation mask area S of the fire channel respectively M The maximum abscissa, minimum abscissa, maximum ordinate, and minimum ordinate in the ground image.

[0019] Preferably, judging whether a motor vehicle occupies a fire passage in the ground image according to the segmentation mask information of the fire passage in the same ground image and the position information of the motor vehicle comprises the following steps:

[0020] The key point coordinates of the vehicle are calculated by the position information of the vehicle, and the key point coordinates include: the center point P of the vehicle c =(x c ,y c ), a first edge point P1, a second edge point P2, a third edge point P3, a fourth edge point P4 (i.e., the midpoint of the line connecting the first edge point P1 and the sixth edge point P6), a fifth edge point P5, a sixth edge point P6, a seventh edge point P7, and an eighth edge point P8, wherein the first edge point P1 to the eighth edge point P8 are all points on the motor vehicle detection frame in the ground image;

[0021] If ym -y c ≥0, then judge the center point P c , the fourth edge point P4, the fifth edge point P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least two points are in the segmentation mask area S M If yes, the motor vehicle is considered to occupy the fire passage; otherwise, the motor vehicle is considered not to occupy the fire passage.

[0022] Preferably, if y m -y c <0, then determine the center point P c , the first edge point P1, the second edge point P2, the third edge point P3, the fourth edge point P4, the right center P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least 4 points are in the segmentation mask area S M If yes, the motor vehicle is considered to occupy the fire passage; otherwise, the motor vehicle is considered not to occupy the fire passage.

[0023] Preferably, if the center point P c In the segmentation mask area S M In the case of fire, the motor vehicle is considered to be occupying the fire passage.

[0024] Preferably, the fire passage occupancy information includes: one or more of the following: position information of the fire passage in the ground image, position information of the motor vehicle, a ground image containing the motor vehicle and the fire passage, fire passage contour feature points, and a fire passage segmentation mask map.

[0025] Preferably, the position information of the motor vehicle in the ground image includes one or more of the position coordinates of the motor vehicle in the ground image, the length and the width of the motor vehicle in the image.

[0026] On the other hand, a fire passage occupancy detection system is also provided, which is used to implement the above-mentioned fire passage occupancy detection method, and comprises:

[0027] Drones;

[0028] An imaging device, which is carried on the drone via a drone gimbal;

[0029] A fire passage segmentation unit, which is used to identify the fire passage in the ground image and output the segmentation information of the fire passage in the ground image;

[0030] A motor vehicle identification unit, which is used to identify a motor vehicle in the ground image and output position information of the motor vehicle in the ground image;

[0031] A fire passage occupancy detection unit, which is used to determine whether there is a motor vehicle occupying the fire passage based on the fire passage segmentation information and the motor vehicle position information;

[0032] And a monitoring terminal, which is used to receive the fire channel occupancy information sent by the drone and generate alarm information.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The algorithm of the present invention is simple and easy to implement. It can automatically analyze ground images based on drone clusters, segmentation algorithms and target detection technology, identify fire lanes and occupied vehicles, and determine whether motor vehicles occupy fire lanes, thereby greatly improving detection efficiency. Compared with manual inspections, the system can significantly reduce manpower input and reduce detection costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the steps of the fire escape detection method in the present invention;

[0036] Figure 2 The fire passage and the motor vehicle image identified in the ground image of the present invention;

[0037] Figure 3 It is the segmentation mask area of ​​the fire passage of the present invention;

[0038] Figure 4 A schematic diagram of key points of the motor vehicle of the present invention;

[0039] Figure 5 This is a schematic diagram of the relative positions of motor vehicles and fire passages. In the figure, the light grey quadrilateral represents a motor vehicle, and the dark grey quadrilateral represents a fire passage.

[0040] Figure 6 It is a structural schematic diagram of the fire passage occupancy detection system in the present invention. DETAILED DESCRIPTION

[0041] Embodiment 1:

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Embodiment 1:

[0044] like Figure 1As shown, this embodiment provides a fire passage occupancy detection method based on a drone cluster, which includes the following steps:

[0045] S1. Control the flight of the drone cluster so that each drone in the drone cluster inspects the same designated area according to the corresponding cruising altitude and route; the cruising altitude and route of each drone during the inspection can be different;

[0046] S2. During the inspection process of the drone, the ground image is obtained in real time through the imaging device carried on the drone gimbal; in this embodiment, the imaging device includes a high-resolution camera, and the imaging device obtains the ground image at a 45°-90° downward angle to obtain a clear and complete ground image; at the same time, the gimbal of each drone in the drone cluster is equipped with an imaging device, and each drone obtains the ground image in real time through the imaging device carried by it during the inspection, and a number of ground images are obtained;

[0047] S3, respectively identifying the fire passage P and the motor vehicle V in the same ground image, and outputting the segmentation information of the fire passage P, and obtaining the segmentation mask information of the fire passage P according to the segmentation information of the fire passage P; and outputting the position information of the motor vehicle V in the ground image;

[0048] S4, judging whether the motor vehicle V occupies the fire passage P in the ground image according to the segmentation mask information of the fire passage P in the same ground image and the position information of the motor vehicle V;

[0049] S5, repeating the above steps S3-S4 to determine whether the same motor vehicle V occupies the same fire passage P in the ground images obtained by different drones;

[0050] S6. Summarize all the obtained judgment results and finally determine whether the motor vehicle V occupies the fire passage P by voting method, so as to improve the accuracy of detection and reduce the false alarm rate;

[0051] S7. If it is finally determined that the fire passage P is occupied by a motor vehicle V, the fire passage occupation information is sent to the monitoring end and an alarm message is generated.

[0052] Preferably, the imaging device acquires ground images at a downward angle of 45°-90°.

[0053] Specifically, in step S3, the fire passage P in the same ground image is identified, and the segmentation information of the fire passage P is output, and the segmentation mask information of the fire passage P is obtained according to the segmentation information of the fire passage P, including:

[0054] like Figure 2As shown, dynamic target segmentation is performed based on the YOLOv8 target detection algorithm or the like, or the fire channel P in the ground image is automatically identified based on the real-time semantic segmentation network model or the like, and the segmentation information of the fire channel P in the ground image is output, and the segmentation mask information of the fire channel P is obtained according to the segmentation information of the fire channel P in the ground image;

[0055] Specifically, obtaining the segmentation mask information of the fire passage P in the ground image according to the segmentation information of the fire passage P includes the following steps:

[0056] like Figure 3 The segmentation mask area S of the fire passage P is calculated by using the segmentation information of the fire passage P in the ground image (the segmentation information is the segmented fire passage image). M , and obtain the center point coordinates P of the segmentation mask area m =(x m ,y m ),and

[0057]

[0058] Among them, x max 、x min ,y max and min Respectively represent the segmentation mask area S of the fire channel P M The maximum abscissa, minimum abscissa, maximum ordinate, and minimum ordinate in the ground image.

[0059] Specifically, in step S4, identifying the motor vehicle V in the ground image and outputting the position information of the motor vehicle V in the ground image includes:

[0060] like Figure 2 As shown, based on the YOLOv8 target detection algorithm or the dense target detection algorithm, the motor vehicle V in the ground image is automatically identified, and the position information of the motor vehicle V in the ground image is output. In this embodiment, the position information of the motor vehicle V in the ground image includes the position coordinates of the motor vehicle V in the ground image, the length of the motor vehicle V in the image, and one or more of the width.

[0061] Specifically, in step S4, judging whether the motor vehicle V occupies the fire passage P in the ground image according to the segmentation mask information of the fire passage P in the same ground image and the position information of the motor vehicle V includes the following steps:

[0062] The key point coordinates of the motor vehicle V are calculated by the position information of the motor vehicle V, such as Figure 4 As shown, the key point coordinates include: the center point P of the motor vehiclec =(x c ,y c ), the first edge point P1, the second edge point P2 (i.e. the midpoint of the line connecting the first edge point P1 and the third edge point P3), the third edge point P3, the fourth edge point P4 (i.e. the midpoint of the line connecting the first edge point P1 and the sixth edge point P6), the fifth edge point P5 (i.e. the midpoint of the line connecting the third edge point P3 and the eighth edge point P8), the sixth edge point P6, the seventh edge point P7 (i.e. the midpoint of the line connecting the first edge point P1 and the sixth edge point P6), the eighth edge point P8, a total of 9 key points, among which the first edge point P1 to the eighth edge point P8 are all points on the motor vehicle V detection frame in the ground image;

[0063] like Figure 5 As shown, judging whether the motor vehicle V occupies the fire passage P specifically includes the following contents:

[0064] If y m -y c ≥0, then judge the center point P c , the fourth edge point P4, the fifth edge point P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least two points are in the segmentation mask area S M If yes, then the motor vehicle V is considered to occupy the fire passage P (such as Figure 5 Otherwise, the motor vehicle is deemed not to occupy the fire lane P (e.g. Figure 5 (a)-(b), (d), and (f) of the Regulations);

[0065] If y m -y c <0, then determine the center point P c , the first edge point P1, the second edge point P2, the third edge point P3, the fourth edge point P4, the right center P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least 4 points are in the segmentation mask area S M If yes, then the motor vehicle V is considered to occupy the fire passage P (such as Figure 5 Otherwise, the motor vehicle is deemed not to occupy the fire passage P; Figure 5 (h) of the Regulations);

[0066] If the center point P c In the segmentation mask area S M In (such as Figure 5 As shown in (e) in the figure, the motor vehicle V is considered to occupy the fire passage P.

[0067] Furthermore, in step S7, the fire passage occupancy information includes: one or more of the position information of the fire passage in the ground image, the position information of the motor vehicle in the ground image, the ground image containing the motor vehicle and the fire passage, the fire passage contour feature points, the fire passage segmentation mask map, etc.

[0068] Therefore, this embodiment automatically analyzes ground images based on segmentation algorithms and target detection technology to identify fire passages and occupied motor vehicles, which can greatly improve detection efficiency, reduce manpower investment, and reduce detection costs. At the same time, it does not need to establish a complex detection model through training. It can quickly determine whether a motor vehicle occupies the fire passage only through the location information of the motor vehicle and the mask information of the fire passage. The entire judgment logic is simple, the amount of calculation and resource consumption is small, and the judgment result is accurate.

[0069] Embodiment 2:

[0070] This embodiment provides a fire passage occupancy detection system, which can implement the fire passage occupancy detection method described in Embodiment 1. Figure 6 As shown, it includes:

[0071] A plurality of drones 1 are formed into a drone cluster;

[0072] An imaging device 2, which is carried on the drone 1 and is used to obtain ground images, and each drone 1 in the drone cluster is equipped with an imaging device 2;

[0073] A fire passage segmentation unit 3 is used to identify the fire passage P in the ground image, output the segmentation information of the fire passage P, and obtain the segmentation mask information of the fire passage P according to the segmentation information of the fire passage P. The specific steps are the same as step S3;

[0074] A motor vehicle identification unit 4 is used to identify the motor vehicle V in the ground image and output the position information of the motor vehicle V in the ground image. The specific steps are the same as those of step S3.

[0075] The fire passage occupancy detection unit 5 is used to determine whether the motor vehicle V occupies the fire passage P in the ground image according to the segmentation mask information of the fire passage P in the same ground image and the position information of the motor vehicle V, and finally determine whether the motor vehicle V occupies the fire passage P by a voting method according to all the obtained judgment results. The specific steps are the same as step S5;

[0076] And the monitoring terminal 6 is used to receive the fire passage occupancy information sent by the drone and generate alarm information.

[0077] To sum up, the algorithm of the present invention is simple and easy to implement. It automatically analyzes ground images based on segmentation algorithms and target detection technology to identify fire passages and occupied motor vehicles, thereby greatly improving detection efficiency, reducing manpower investment, and reducing detection costs. At the same time, it does not require training to establish a complex detection model. It can quickly determine whether a motor vehicle occupies a fire passage only through the location information of the motor vehicle and the mask information of the fire passage, and the judgment result is accurate.

[0078] It should be noted that the technical features in the above-mentioned embodiments 1-2 can be combined arbitrarily, and the combined technical solutions all belong to the protection scope of the present application. In this article, terms such as "including", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0079] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fire passage occupancy detection method based on drone clusters, characterized in that: include: Control the flight of the drone swarm so that each drone in the drone swarm inspects the same designated area according to the corresponding cruising altitude and route; During the drone inspection process, the imaging device on the drone gimbal acquires ground images in real time; Recognize the fire passage and the motor vehicle in the same ground image respectively, and output the segmentation information of the fire passage, and obtain the segmentation mask information of the fire passage according to the segmentation information of the fire passage; and outputting position information of the motor vehicle in the ground image; It is determined whether a motor vehicle occupies the fire passage based on the segmentation mask of the fire passage and the position information of the motor vehicle.

2. The fire passage occupancy detection method according to claim 1, characterized in that: Judging whether a motor vehicle occupies a fire passage according to the segmentation mask of the fire passage and the position information of the motor vehicle includes the following steps: Determining whether the motor vehicle occupies the fire passage in the ground image according to the segmentation mask information of the fire passage in the same ground image and the position information of the motor vehicle; Determine whether the same motor vehicle occupies the same fire lane in ground images obtained by different drones; All the judgment results obtained are summarized and a final determination is made through voting whether a motor vehicle occupies the fire passage.

3. The fire passage occupancy detection method according to claim 2, characterized in that: If it is finally determined that the fire passage is occupied by a motor vehicle, the fire passage occupancy information will be sent to the monitoring end and an alarm message will be generated.

4. The fire passage occupancy detection method according to claim 1, characterized in that: The segmentation mask information of the fire passage is obtained according to the segmentation information of the fire passage in the ground image, comprising the following steps: The segmentation mask area S of the fire passage is calculated based on the segmentation information of the fire passage in the ground image. M , and obtain the center point coordinates P of the segmentation mask area m =(x m ,y m ),and Among them, x max 、x min ,y max and min Represents the segmentation mask area S of the fire channel M The maximum abscissa, minimum abscissa, maximum ordinate, and minimum ordinate in the ground image.

5. The fire passage occupancy detection method according to claim 1, characterized in that: Judging whether a motor vehicle occupies a fire passage in the ground image according to the segmentation mask information of the fire passage in the same ground image and the position information of the motor vehicle comprises the following steps: The key point coordinates of the vehicle are calculated by the position information of the vehicle, and the key point coordinates include: the center point P of the vehicle c =(x c ,y c ), a first edge point P1, a second edge point P2, a third edge point P3, a fourth edge point P4 (i.e., the midpoint of the line connecting the first edge point P1 and the sixth edge point P6), a fifth edge point P5, a sixth edge point P6, a seventh edge point P7, and an eighth edge point P8, wherein the first edge point P1 to the eighth edge point P8 are all points on the motor vehicle detection frame in the ground image; If y m -y c ≥0, then judge the center point P c , the fourth edge point P4, the fifth edge point P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least two points are in the segmentation mask area S M If yes, the motor vehicle is considered to occupy the fire passage; otherwise, the motor vehicle is considered not to occupy the fire passage.

6. The fire passage occupancy detection method according to claim 5, characterized in that: If y m -y c <0, then determine the center point P c , the first edge point P1, the second edge point P2, the third edge point P3, the fourth edge point P4, the right center P5, the sixth edge point P6, the seventh edge point P7, and the eighth edge point P8, whether at least 4 points are in the segmentation mask area S M If yes, the motor vehicle is considered to occupy the fire passage; otherwise, the motor vehicle is considered not to occupy the fire passage.

7. The fire passage occupancy detection method according to claim 5, characterized in that: If the center point P c In the segmentation mask area S M In the case of fire, the motor vehicle is considered to be occupying the fire passage.

8. The fire passage occupancy detection method according to claim 2, characterized in that: The fire passage occupancy information includes: one or more of the position information of the fire passage in the ground image, the position information of the motor vehicle, the ground image containing the motor vehicle and the fire passage, the fire passage contour feature points, the fire passage segmentation mask map, etc.

9. The fire passage occupancy detection method according to claim 1, characterized in that: The position information of the motor vehicle in the ground image includes one or more of the position coordinates of the motor vehicle in the ground image, the length and the width of the motor vehicle in the image.

10. A fire passage occupancy detection system, used to implement the fire passage occupancy detection method according to any one of claims 1 to 9, characterized in that: include: Drones; An imaging device, which is carried on the drone via a drone gimbal; A fire passage segmentation unit, which is used to identify the fire passage in the ground image and output the segmentation information of the fire passage in the ground image; A motor vehicle identification unit, which is used to identify a motor vehicle in the ground image and output position information of the motor vehicle in the ground image; A fire passage occupancy detection unit, which is used to determine whether there is a motor vehicle occupying the fire passage based on the fire passage segmentation information and the motor vehicle position information; And a monitoring terminal, which is used to receive the fire channel occupancy information sent by the drone and generate alarm information.

Images