An unmanned aerial vehicle-based city building fire hazard identification method

Abstract

The application discloses an unmanned aerial vehicle (UAV)-based city building fire hazard identification method, and belongs to the technical field of intelligent fire fighting, wherein a three-dimensional model of a city building is used to generate an autonomous cruise route, an UAV is controlled to fly and simultaneously collect visible light and infrared multi-modal images, a space geographic coordinate label is added in combination with an RTK positioning module, and the images are input into a fire hazard identification model pre-trained by a time-varying fuzzy intelligent optimization algorithm to output hazard information, so that the identification precision and robustness of the model for a complex fuzzy scene are significantly improved; subsequently, hazards are mapped into a three-dimensional model according to the space geographic coordinate label to generate a stereoscopic visual report, hovering and rephotographing are triggered when high-risk hazards or abnormally high temperatures are identified, and an alarm is pushed in real time, so that high-precision, dead-angle-free and space-visualized intelligent investigation and dynamic response of fire hazards are realized.

Description

Technical Field

[0001] This application belongs to the field of smart fire protection technology, specifically relating to a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs). Background Technology

[0002] With the acceleration of urbanization, the number of high-rise and super high-rise buildings in cities is increasing, and their complex structures make fire hazard inspections extremely difficult. Traditional fire inspections mainly rely on manual ground patrols, which are inefficient and have blind spots (such as the facades and rooftops of high-rise buildings).

[0003] In recent years, drone technology has been introduced into the field of fire inspection, but existing technology still has the following shortcomings: Unreasonable flight path planning: Existing drone patrols are mostly manually controlled or based on simple GPS tracking, lacking deep integration with 3D city models. This makes it easy to miss inspections due to building obstruction and cannot guarantee maximum coverage of building facades.

[0004] Low target detection accuracy: Drones are easily affected by airflow during flight, resulting in time-varying blurring of images; at the same time, due to factors such as changes in lighting and occlusion, traditional target detection models (such as the YOLO series) have insufficient generalization ability and detection accuracy in complex urban scenes, which easily leads to missed detections or false detections.

[0005] Lack of spatial geographic mapping: Existing systems can usually only output the location of hazards on two-dimensional images, which cannot intuitively reflect the exact location of hazards in three-dimensional urban space, making it inconvenient for fire commanders to quickly locate and dispatch them.

[0006] The emergency response mechanism is inadequate: after a potential hazard is discovered, it can usually only be reported afterward, lacking a real-time dynamic response mechanism for discovery, confirmation, and notification, and lacking automatic verification methods for abnormal high temperatures in infrared thermal imaging. Summary of the Invention

[0007] The purpose of this application is to provide a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs) to address issues such as unreasonable flight path planning, low target detection accuracy, lack of spatial geographic mapping, and / or imperfect emergency response mechanisms.

[0008] This application provides a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs), including: Based on the preset 3D model of urban buildings and the inspection task, the corresponding autonomous patrol route of the drone is generated; The drone is controlled to fly along the autonomous cruise route, and multimodal image data of the building exterior is collected simultaneously by the onboard visible light camera and infrared thermal imager. Spatial geographic coordinate labels are added to the image data in combination with the RTK positioning module. The multimodal image data is input into a fire hazard identification model pre-trained by a time-varying fuzzy intelligent optimization algorithm, and the model outputs the hazard category, confidence level, and bounding box coordinates; wherein, the fire hazard identification model is a multi-task detection model built based on the YOLOv8 deep learning network; Based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto a three-dimensional city model to generate a fire hazard report containing the location, type, and thermal anomaly status of the hazard. When a specific high-risk hazard is identified or an abnormally high temperature point is detected by an infrared thermal imager, the drone is triggered to hover and take a follow-up photo, and the follow-up photo data and alarm information are pushed to the fire command center in real time.

[0009] Furthermore, based on a pre-set 3D model of urban buildings and the inspection task, an autonomous patrol route for the drone is generated, including: Analyze the patrol task to determine the location, height, and facades of the target buildings that require special attention during the patrol. Based on the aforementioned 3D urban building model, the 3D outline of the target building and the obstacle avoidance space of the surrounding airspace are extracted. Combining the drone's flight performance parameters with the constraints of maximizing coverage of building facades and avoiding obstruction, a path planning algorithm is used to generate an autonomous cruise route that includes shooting waypoints and flight attitude.

[0010] Furthermore, the method for obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm includes: The model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population; the training population includes multiple different individuals. Obtain the fitness values ​​of all individuals and determine the individual with the highest fitness value as the globally optimal individual; Based on the globally optimal individual, the individuals in the training population are jointly optimized by using a time-varying information chain acquisition method to obtain the first target individual; A time-varying fuzzy information acquisition method is used to perform fuzzy information joint optimization on the first target individual to obtain the second target individual; An adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain the third target individual; The process is re-optimized until the training termination condition is met, and the final globally optimal individual is determined based on the latest third target individual. The model parameters in the final global optimal individual are used as the target parameters of the fire hazard identification model, thus obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm.

[0011] Furthermore, the model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population, including: The model parameters of the fire hazard identification model are randomly initialized between the upper and lower limits, and the randomly initialized model parameters are encoded into vectors to obtain individuals; Repeat the initialization encoding multiple times to obtain multiple individuals and form a training population.

[0012] Furthermore, the fitness values ​​of all individuals are obtained, and the individual with the highest fitness value is determined as the globally optimal individual, including: Obtain the root mean square loss function value for all individuals; The fitness value of an individual is obtained by either taking a negative value of the root mean square loss function or by adding the root mean square loss function value to a preset constant term and taking the reciprocal. Iterate through all individuals to obtain the fitness value of each individual, and determine the individual with the largest fitness value as the global optimal individual.

[0013] Furthermore, based on the globally optimal individual, a time-varying information chain acquisition method is used to perform chain-based joint optimization on the individuals in the training population to obtain the first target individual, including: Based on the fitness values ​​of the individuals, the training information factors for all individuals are determined as follows: ; in, To train information factors, For the first j The fitness value corresponding to each individual, N is the total number of individuals in the training population, and ln is the logarithmic function with the natural constant e as the base; Based on the fitness value of the individual and the current number of training iterations, the time-varying control factor is obtained as follows: ; in, For time-varying control factors, This is the minimum value of the time-varying control factor, set as a constant term between 0.01 and 0.1. For the current number of training iterations, To the maximum number of training iterations, This is the balance factor, and it is set to a constant between (0,1). The minimum fitness corresponding to the individual. The maximum fitness corresponding to the individual; Arrange all individuals in descending order of fitness value to obtain the sorted individuals; Based on the training information factor, the globally optimal individual, and the time-varying control factor, chain-based joint optimization is performed on the individuals in the training population to obtain the first target individual: ; in, The globally optimal individual. For the first j The individuals after the arrangement, For the first j The first target individual, It is the first learning factor, and is set to 1 or 2; This is the second learning factor, and it is set to 0.75 or 1.5; The first random number between (0,1) The second random number between (0,1) Indicates the first j -1 individuals after the permutation, Indicates the first j +1 more individuals after the permutation.

[0014] Furthermore, a time-varying fuzzy information acquisition method is used to perform fuzzy information joint optimization on the first target individual to obtain the second target individual, including: The time-varying correlation function is obtained as follows: ; in, For the first t Time-varying correlation function during the training process This is the maximum value of the time-varying correlation function, and it is set to 0.95; This is the minimum value of the time-varying correlation function, and it is set to 0.01; Let π be the mathematical constant, T be the maximum number of training iterations, and cos be the cosine function. Based on the time-varying correlation function, the fuzzy information acquisition factor is obtained as follows: ; in, For the first t Fuzzy information acquisition factors during the training process. This is the maximum value of the fuzzy information collection factor, and it is set to 0.95. Based on the fuzzy information acquisition factor, the first target individual is subjected to fuzzy information joint optimization to obtain the second target individual: ; ; ; ; in, For the first n The first target individual, For the first n A second target individual, For the first n The first target individual in the first t Update speed during each training process For the first n The first target individual in the first t +1 update speed during training. As the first joint optimization control factor, As the second joint optimization control factor, To and Different individuals with the first random number, To and Different individuals with second random numbers, The third random number between (0,1) The fourth random number between (0, 1); The maximum value of the first joint optimization control factor is set to 2; The minimum value of the first joint optimization control factor is set to 0.4; The maximum value of the second joint optimization control factor is set to 2; This is the minimum value of the second joint optimization control factor, and it is set to 0.4.

[0015] Furthermore, an adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain the third target individual, including: Three other second target individuals are randomly matched for the second target individual to obtain the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual corresponding to the second target individual; Based on the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual, the first global mutation information and the second global mutation information are obtained as follows: ; ; ; in, For the first global information, jointly optimize the individual's first... d The model parameters are d = 1, 2, ..., D, where D is the total number of model parameters in an individual. For the second global information joint optimization of the individual's firstd 3D model parameters, For the third global information joint optimization of the individual's first d 3D model parameters, For the first m The second target individual's first d The first global mutation information corresponding to the dimensional model parameters. For the first m The second target individual's first d The second global mutation information corresponding to the dimensional model parameters. The solution spatial location parameters of an individual are jointly optimized based on the first global information. The solution spatial location parameters of the individual are jointly optimized using the second global information. The solution spatial location parameters of the individual are jointly optimized using third global information. This is a preset constant term, set to 0.01; For the first k The solution spatial location parameters of an individual are jointly optimized using global information, k=1,2,3. For the first k The fitness value of an individual is jointly optimized using global information. The fitness value of the globally optimal individual is given by , and T is the maximum number of training iterations. Based on the first and second global mutation information, the second target individual is jointly optimized using global information to obtain the third target individual: ; in, For the first m The second target individual's first d 3D model parameters, For the first m The third target individual's first d 3D model parameters, It is a natural constant. The first of the globally optimal individuals d 3D model parameters; Determine whether the fitness value of the third target individual has increased. If so, retain the result of this global information joint optimization; otherwise, filter this global information joint optimization and directly use the original second target individual as the third target individual.

[0016] Furthermore, based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto a 3D city model, generating a fire hazard report that includes the hazard location, type, and thermal anomaly status, including: Extract the pixel coordinates of the four vertices of the bounding box of the potential hazard in the image; Using the mapping relationship of the spatial geographic coordinate labels, the pixel coordinates of the four vertices are converted into real-world three-dimensional geographic coordinates; The converted 3D geographic coordinates are registered with the coordinate system of the 3D urban building model, and a hazard visualization icon or 3D bounding box is generated at the corresponding spatial location of the 3D urban model.

[0017] Furthermore, when a specific high-risk hazard is identified or an abnormally high temperature point is detected by the infrared thermal imager, the drone is triggered to hover and take repeat photos, including: Determine whether the hazard category belongs to the preset high-risk hazard set (such as illegal occupation of fire lanes or stacking of flammable materials), or whether the temperature data collected by the infrared thermal imager exceeds the preset temperature threshold. If any of the above conditions are met, control the drone to interrupt the current cruise route and hover at the current position; Control the gimbal to adjust the shooting angle, and continuously capture images from multiple angles of the potential hazard area as reshoot data.

[0018] The beneficial effects of this application are as follows: This application provides a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs). It generates an autonomous flight path from a 3D model of the urban building, controls the UAV to fly, and simultaneously acquires visible light and infrared multimodal images. Spatial geographic coordinate labels are added using an RTK positioning module, and the images are input into a fire hazard identification model pre-trained by a time-varying fuzzy intelligent optimization algorithm to output hazard information. This significantly improves the model's accuracy and robustness in complex and fuzzy scenes. Subsequently, the hazards are mapped onto the 3D model based on the spatial geographic coordinate labels to generate a stereoscopic visualization report. When high-risk hazards or abnormally high temperatures are identified, hovering and re-enhancing are triggered, and alarms are pushed in real time. This achieves high-precision, comprehensive, and spatially visualized intelligent investigation and dynamic response to fire hazards. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] Figure 1 A flowchart of a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs) provided in this application.

[0021] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] The embodiments of this application are described in detail below with reference to the accompanying drawings.

[0024] like Figure 1 As shown in the figure, this application provides a method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs), including: S101. Based on the preset 3D model of urban buildings and the inspection task, generate the corresponding autonomous cruise route for the UAV. In some possible embodiments, based on a preset 3D model of urban buildings and the inspection task, an autonomous drone navigation route is generated, including: Analyze the patrol task to determine the location, height, and facades of the target buildings that require special attention during the patrol. Based on the aforementioned 3D urban building model, the 3D outline of the target building and the obstacle avoidance space of the surrounding airspace are extracted. Combining the drone's flight performance parameters with the constraints of maximizing coverage of building facades and avoiding obstruction, a path planning algorithm is used to generate an autonomous cruise route that includes shooting waypoints and flight attitude.

[0025] For example, traditional UAV flight path planning often employs simple two-dimensional planar orbiting or fixed-height gridded flight. This approach is highly prone to collisions when facing complex high-rise buildings and cannot guarantee that the camera's viewpoint is always perpendicular to the building facade, resulting in significant image distortion and a large amount of invalid background. Therefore, this invention proposes a refined flight path generation method based on three-dimensional model constraints, with the following specific steps: Upon receiving an inspection task, the system first performs semantic parsing. For example, the task instruction is "Inspect the south facade and rooftop fire escape of Building A". The system retrieves the geographical location (latitude and longitude) and absolute height (e.g., 120 meters) of Building A from the city's 3D building model, and divides the south facade into multiple "sub-areas to be inspected" according to the building's geometry, based on every 10 floors or specific building concave and convex structures. At the same time, the rooftop is divided into a separate sub-area.

[0026] Due to the dense spacing between buildings in the city, drones cannot fly close to the walls. Based on a pre-set 3D city model, the system extracts the detailed 3D outline of Building A (including balconies, external decorative elements, and other protruding structures). For safety, the system "expands" this 3D outline outwards along the normal direction by a safe distance (e.g., expanding outwards by 3 to 5 meters, depending on RTK positioning accuracy and obstacle avoidance radar performance), generating a virtual "3D safe flight envelope." Simultaneously, the system divides the airspace outside the envelope into a 3D grid, marking the "no-fly grids" occupied by other adjacent buildings.

[0027] In this stage, the system densely arranges "candidate waypoints" on the aforementioned "3D safe flight envelope" based on the focal length and field of view (FOV) of the visible light / infrared camera on the UAV, as well as the image overlap rate (e.g., 80% in the heading direction and 60% in the lateral direction) required to ensure subsequent 3D reconstruction or hazard identification. For each candidate waypoint, the system performs "line-of-sight occlusion detection": that is, a ray is emitted from the candidate point to the wall sub-area it is responsible for. If the ray hits the grid of other buildings before reaching the wall, the waypoint is determined to be occluded and is subject to a very high penalty.

[0028] Finally, a multi-objective optimization function is established: the objective is to maximize the total coverage area of ​​all effective waypoints (constraint 1), minimize the flight distance between the waypoints to save power (constraint 2, i.e., UAV flight performance parameters), and ensure that all waypoints are within the safe envelope and unobstructed (constraint 3).

[0029] The system employs an improved A* (A-star) algorithm or ant colony algorithm to solve the above model, connecting the points to form a line. The output is not merely a string of latitude and longitude coordinates, but a four-dimensional flight path encompassing "time-space-attitude." For example: Waypoint 1 (E longitude XXX, N latitude XXX, altitude 50 meters, speed 2 m / s, gimbal pitch angle -90°, i.e., looking vertically down at the rooftop) →→ Waypoint 2 (E longitude XXX, N latitude XXX, altitude 60 meters, speed 1.5 m / s, gimbal yaw angle +15°, pitch angle -45°, i.e., a blind spot looking at the balcony). This dynamic attitude adjustment ensures the camera is always facing the wall, significantly improving the effective pixel ratio for subsequent image recognition.

[0030] By analyzing the patrol mission and extracting 3D contours and obstacle avoidance spaces from urban building 3D models, an autonomous patrol route is generated with the constraint of maximizing building facade coverage and avoiding obstruction. The technical benefits are: it completely changes the blindness of traditional 2D planar point marking, enabling automatic planning of wall-hugging flight trajectories based on the actual concave and convex structures of buildings, ensuring that visible light and infrared cameras can cover key patrol facades without blind spots, greatly improving the patrol coverage and data acquisition quality of a single flight.

[0031] It is worth noting that existing path planning and obstacle avoidance algorithms can also be used to generate autonomous cruise routes for the drone while flying at a fixed altitude. For example, obstacles at the flight altitude can be determined using a 3D model of urban buildings, and the task nodes (such as coordinates) recorded in the patrol mission can be used as target points, with path planning algorithms employed for path planning.

[0032] S102. The camera synchronously acquires multimodal image data of the building's exterior and adds spatial geographic coordinate labels to the image data in conjunction with the RTK positioning module. During this process, the UAV flight control system strictly follows the waypoints, and the visible light and infrared instruments maintain strict synchronization in terms of timestamps to ensure that the two modal characteristics of the same potential hazard can be accurately aligned. The RTK module receives differential signals from the base station and writes latitude, longitude, and elevation information into the image data.

[0033] Simultaneously acquiring visible light and infrared images, and adding high-precision spatial geographic coordinate labels using an RTK positioning module. The technical benefits are: RTK's centimeter-level positioning accuracy eliminates the drift error of ordinary GPS; multimodal data can not only identify hidden dangers such as illegal parking and clutter under visible light, but also detect invisible hazards such as aging and overheating electrical wiring and unauthorized use of open flames through infrared thermal imaging, achieving precise anchoring between the physical world and image data.

[0034] S103. Input the multimodal image data into a fire hazard identification model pre-trained by a time-varying fuzzy intelligent optimization algorithm, and output the hazard category, confidence level, and bounding box coordinates; wherein, the fire hazard identification model is a multi-task detection model built based on the YOLOv8 deep learning network; In some possible embodiments, the method for obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm includes: The model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population; the training population includes multiple different individuals. Obtain the fitness values ​​of all individuals and determine the individual with the highest fitness value as the globally optimal individual; Based on the globally optimal individual, the individuals in the training population are jointly optimized by using a time-varying information chain acquisition method to obtain the first target individual; A time-varying fuzzy information acquisition method is used to perform fuzzy information joint optimization on the first target individual to obtain the second target individual; An adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain the third target individual; The process is re-optimized until the training termination condition is met (such as reaching the maximum number of training iterations), and the final globally optimal individual is determined based on the latest third target individual. The model parameters in the final global optimal individual are used as the target parameters of the fire hazard identification model, thus obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm.

[0035] In some possible embodiments, the model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population, including: The model parameters of the fire hazard identification model are randomly initialized between the upper and lower limits, and the randomly initialized model parameters are encoded into vectors to obtain individuals; Repeat the initialization encoding multiple times to obtain multiple individuals and form a training population.

[0036] In some possible embodiments, the fitness values ​​of all individuals are obtained, and the individual with the highest fitness value is determined as the globally optimal individual, including: Obtain the root mean square loss function value for all individuals; The fitness value of an individual is obtained by either taking the negative value of the root mean square loss function or by adding the root mean square loss function value to a preset constant term (such as 0.01) and taking the reciprocal. Iterate through all individuals to obtain the fitness value of each individual, and determine the individual with the largest fitness value as the global optimal individual.

[0037] In some possible embodiments, based on the globally optimal individual, a time-varying information chain acquisition method is used to perform chain joint optimization on the individuals in the training population to obtain the first target individual, including: Based on the fitness values ​​of the individuals, the training information factors for all individuals are determined as follows: ; in, To train information factors, For the first j The fitness value corresponding to each individual, N is the total number of individuals in the training population, and ln is the logarithmic function with the natural constant e as the base; Based on the fitness value of the individual and the current number of training iterations, the time-varying control factor is obtained as follows: ; in, For time-varying control factors, This is the minimum value of the time-varying control factor, set as a constant term between 0.01 and 0.1. For the current number of training iterations, To the maximum number of training iterations, This is the balance factor, and it is set to a constant between (0,1). The minimum fitness corresponding to the individual. The maximum fitness corresponding to the individual; Arrange all individuals in descending order of fitness value to obtain the sorted individuals; Based on the training information factor, the globally optimal individual, and the time-varying control factor, chain-based joint optimization is performed on the individuals in the training population to obtain the first target individual: ; in, The globally optimal individual. For the first j The individuals after the arrangement, For the first j The first target individual, It is the first learning factor, and is set to 1 or 2; This is the second learning factor, and it is set to 0.75 or 1.5; The first random number between (0,1) The second random number between (0,1) Indicates the first j -1 individuals after the permutation, Indicates the first j +1 more individuals after the permutation.

[0038] By using chain perturbations on adjacent individuals after sorting, combined with a time-varying control factor, the effects are: maintaining population diversity in the early stages of training (when the time-varying control factor is large) to avoid getting trapped in local optima; and enhancing local fine-grained search in the later stages of training (when the time-varying control factor is small), improving model training performance and ensuring the model's ability to extract fuzzy edge features after training.

[0039] In some possible embodiments, a time-varying fuzzy information acquisition method is used to perform fuzzy information joint optimization on the first target individual to obtain a second target individual, including: The time-varying correlation function is obtained as follows: ; in, For the first t Time-varying correlation function during the training process This is the maximum value of the time-varying correlation function, and it is set to 0.95; This is the minimum value of the time-varying correlation function, and it is set to 0.01; Let π be the mathematical constant, T be the maximum number of training iterations, and cos be the cosine function. Based on the time-varying correlation function, the fuzzy information acquisition factor is obtained as follows: ; in, For the first t Fuzzy information acquisition factors during the training process. This is the maximum value of the fuzzy information collection factor, and it is set to 0.95. Based on the fuzzy information acquisition factor, the first target individual is subjected to fuzzy information joint optimization to obtain the second target individual: ; ; ; ; in, For the first n The first target individual, For the first n A second target individual, For the first n The first target individual in the first t Update speed during each training process For the first n The first target individual in the first t +1 update speed during training. As the first joint optimization control factor, As the second joint optimization control factor, To and Different individuals with the first random number, To and Different individuals with second random numbers, The third random number between (0,1) The fourth random number between (0, 1); The maximum value of the first joint optimization control factor is set to 2; The minimum value of the first joint optimization control factor is set to 0.4; The maximum value of the second joint optimization control factor is set to 2; This is the minimum value of the second joint optimization control factor, and it is set to 0.4.

[0040] By introducing a sinusoidal time-varying correlation function and a fuzzy information acquisition factor, the individual update speed is dynamically adjusted. The effect is that by dynamically balancing self-cognition and social cognition through fuzzy logic, the algorithm can maintain diversity in the early and mid-stages and ensure convergence precision in the later stages. This gives the model extremely strong robustness and anti-interference ability when facing fuzzy and uncertain information such as sudden changes in illumination and shadows.

[0041] In some possible embodiments, an adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain a third target individual, including: Three other second target individuals are randomly matched for the second target individual to obtain the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual corresponding to the second target individual; Based on the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual, the first global mutation information and the second global mutation information are obtained as follows: ; ; ; in, For the first global information, jointly optimize the individual's first... d The model parameters are d = 1, 2, ..., D, where D is the total number of model parameters in an individual. For the second global information joint optimization of the individual's first d 3D model parameters, For the third global information joint optimization of the individual's first d 3D model parameters, For the first m The second target individual's first d The first global mutation information corresponding to the dimensional model parameters. For the first m The second target individual's first d The second global mutation information corresponding to the dimensional model parameters. The solution spatial location parameters of an individual are jointly optimized based on the first global information. The solution spatial location parameters of the individual are jointly optimized using the second global information. The solution spatial location parameters of the individual are jointly optimized using third global information. This is a preset constant term, set to 0.01; For the first k The solution spatial location parameters of an individual are jointly optimized using global information, k=1,2,3. For the first kThe fitness value of an individual is jointly optimized using global information. The fitness value of the globally optimal individual is given by , and T is the maximum number of training iterations. Based on the first and second global mutation information, the second target individual is jointly optimized using global information to obtain the third target individual: ; in, For the first m The second target individual's first d 3D model parameters, For the first m The third target individual's first d 3D model parameters, It is a natural constant. The first of the globally optimal individuals d 3D model parameters; Determine whether the fitness value of the third target individual has increased. If so, retain the result of this global information joint optimization; otherwise, filter this global information joint optimization and directly use the original second target individual as the third target individual.

[0042] Global mutation information is calculated by randomly matching three individuals, and adaptive scaling is performed in conjunction with the solution space location parameters. The effect is that a deep, cross-dimensional search is performed in the parameter space, and a greedy selection mechanism is used to retain individuals whose fitness increases and filter them out otherwise. This not only effectively enhances the global search capability but also ensures that the model loss function can converge stably and quickly to the global minimum, ultimately resulting in a significant improvement in the confidence of hazard identification and the accuracy of bounding box regression.

[0043] Optionally, after each update of an individual, an over-limit processing can be performed to ensure the validity of the individual.

[0044] S104. Based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto the three-dimensional city model to generate a fire hazard report containing the location, type, and thermal anomaly status of the hazard. In some possible embodiments, based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto a three-dimensional city model to generate a fire hazard report containing the hazard location, type, and thermal anomaly status, including: Extract the pixel coordinates of the four vertices of the bounding box of the potential hazard in the image; Using the mapping relationship of the spatial geographic coordinate labels, the pixel coordinates of the four vertices are converted into real-world three-dimensional geographic coordinates; The converted 3D geographic coordinates are registered with the coordinate system of the 3D urban building model, and a hazard visualization icon or 3D bounding box is generated at the corresponding spatial location of the 3D urban model.

[0045] For example, the pixel coordinates of the four vertices of the hazard bounding box in the image are extracted; using the mapping relationship between the camera intrinsic matrix, extrinsic matrix and RTK spatial geographic coordinate label, the pixel coordinates are converted into real-world three-dimensional geographic coordinates (X,Y,Z) through collinearity equations; the converted coordinates are registered with the WGS84 or CGCS2000 coordinate system of the urban building three-dimensional model (such as the OSGB format model), and a red hazard visualization icon or a semi-transparent three-dimensional bounding box is generated at the corresponding spatial location.

[0046] By extracting the vertex pixel coordinates of the bounding box and combining them with spatial geographic coordinate labels, the data is converted into real-world 3D geographic coordinates and registered to a 3D city model. The technical effect is that it breaks the limitations of traditional 2D reports, allowing fire commanders to directly see on a 3D digital twin platform which floor and window a hazard is located on, achieving a WYSIWYG three-dimensional command and dispatch system.

[0047] S105. When a specific high-risk hazard is identified or an abnormally high temperature point is detected by the infrared thermal imager, the drone is triggered to hover and take a follow-up photo, and the follow-up photo data and alarm information are pushed to the fire command center in real time.

[0048] In some possible embodiments, when a specific high-risk hazard is identified or an abnormally high temperature point is detected by the infrared thermal imager, the drone is triggered to hover and take a follow-up shot, including: Determine whether the hazard category belongs to the preset high-risk hazard set (such as illegal occupation of fire lanes or stacking of flammable materials), or whether the temperature data collected by the infrared thermal imager exceeds the preset temperature threshold. If any of the above conditions are met, control the drone to interrupt the current cruise route and hover at the current position; Control the gimbal to adjust the shooting angle, and continuously capture images from multiple angles of the potential hazard area as reshoot data.

[0049] The system has pre-set sets of high-risk hazards (such as illegal occupation of fire lanes, stacking of flammable materials, and obstruction of fire hydrants) and temperature thresholds (such as 80℃). When the category output by the YOLOv8 model belongs to the high-risk set, or when there is a connected domain exceeding 80℃ in the raw temperature matrix of the infrared thermal imager, the UAV flight control immediately interrupts the current flight path and hovers at the current position. At the same time, the control gimbal continuously takes 9 photos at a set step size (such as every 10°) within the pitch angle [-30°, +30°] and yaw angle [-45°, +45°] as repeat data, and pushes the alarm information including coordinates to the fire command center in real time via 5G / 4G network.

[0050] The system triggers hovering and re-shooting, along with multi-angle adjustments to the gimbal, targeting high-risk hazards or abnormally high temperatures. The technical benefits include: effectively eliminating false positives caused by poor shooting angles during a single drone flight or interference from instantaneous heat sources (such as reflected sunlight); continuous multi-angle re-shooting provides more comprehensive decision-making information; and combined with real-time push notifications, it reduces the time from hazard discovery to confirmation to the second level, buying precious time for fire suppression.

[0051] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0055] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for identifying fire hazards in urban buildings based on unmanned aerial vehicles (UAVs), characterized in that, include: Based on the preset 3D model of urban buildings and the inspection task, the corresponding autonomous patrol route of the drone is generated; The drone is controlled to fly along the autonomous cruise route, and multimodal image data of the building exterior is collected simultaneously by the onboard visible light camera and infrared thermal imager. Spatial geographic coordinate labels are added to the image data in combination with the RTK positioning module. The multimodal image data is input into a fire hazard identification model pre-trained by a time-varying fuzzy intelligent optimization algorithm, and the hazard category, confidence level, and bounding box coordinates are output. Based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto a three-dimensional city model to generate a fire hazard report containing the location, type, and thermal anomaly status of the hazard. When a specific high-risk hazard is identified or an abnormally high temperature point is detected by an infrared thermal imager, the drone is triggered to hover and take a follow-up photo, and the follow-up photo data and alarm information are pushed to the fire command center in real time.

2. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on a pre-set 3D model of urban buildings and inspection tasks, an autonomous drone patrol route is generated, including: Analyze the patrol task to determine the location, height, and facades of the target buildings that require special attention during the patrol. Based on the aforementioned 3D urban building model, the 3D outline of the target building and the obstacle avoidance space of the surrounding airspace are extracted. Combining the drone's flight performance parameters with the constraints of maximizing coverage of building facades and avoiding obstruction, a path planning algorithm is used to generate an autonomous cruise route that includes shooting waypoints and flight attitude.

3. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The method for obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm includes: The model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population; the training population includes multiple different individuals. Obtain the fitness values ​​of all individuals and determine the individual with the highest fitness value as the globally optimal individual; Based on the globally optimal individual, the individuals in the training population are jointly optimized by using a time-varying information chain acquisition method to obtain the first target individual; A time-varying fuzzy information acquisition method is used to perform fuzzy information joint optimization on the first target individual to obtain the second target individual; An adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain the third target individual; The process is re-optimized until the training termination condition is met, and the final globally optimal individual is determined based on the latest third target individual. The model parameters in the final global optimal individual are used as the target parameters of the fire hazard identification model, thus obtaining the fire hazard identification model pre-trained by the time-varying fuzzy intelligent optimization algorithm.

4. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The model parameters of the fire hazard identification model are initialized and encoded multiple times to obtain a training population, including: The model parameters of the fire hazard identification model are randomly initialized between the upper and lower limits, and the randomly initialized model parameters are encoded into vectors to obtain individuals; Repeat the initialization encoding multiple times to obtain multiple individuals and form a training population.

5. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, Obtain the fitness values ​​for all individuals and determine the individual with the highest fitness value as the globally optimal individual, including: Obtain the root mean square loss function value for all individuals; The fitness value of an individual is obtained by either taking a negative value of the root mean square loss function or by adding the root mean square loss function value to a preset constant term and taking the reciprocal. Iterate through all individuals to obtain the fitness value of each individual, and determine the individual with the largest fitness value as the global optimal individual.

6. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, Based on the globally optimal individual, a time-varying information chain acquisition method is used to perform chain joint optimization on the individuals in the training population to obtain the first target individual, including: Based on the fitness values ​​of the individuals, the training information factors for all individuals are determined as follows: ； in, To train information factors, For the first j The fitness value corresponding to each individual, N is the total number of individuals in the training population, and ln is the logarithmic function with the natural constant e as the base; Based on the fitness value of the individual and the current number of training iterations, the time-varying control factor is obtained as follows: ； in, For time-varying control factors, This is the minimum value of the time-varying control factor, set as a constant term between 0.01 and 0.

1. For the current number of training iterations, To the maximum number of training iterations, This is the balance factor, and it is set to a constant between (0,1). The minimum fitness corresponding to the individual. The maximum fitness corresponding to the individual; Arrange all individuals in descending order of fitness value to obtain the sorted individuals; Based on the training information factor, the globally optimal individual, and the time-varying control factor, chain-based joint optimization is performed on the individuals in the training population to obtain the first target individual: ； in, The globally optimal individual. For the first j The individuals after the arrangement, For the first j The first target individual, It is the first learning factor, and is set to 1 or 2; This is the second learning factor, and it is set to 0.75 or 1.5; The first random number between (0,1) The second random number between (0,1) Indicates the first j -1 individuals after the permutation, Indicates the first j +1 more individuals after the permutation.

7. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The first target individual is obtained by performing fuzzy information joint optimization using a time-varying fuzzy information acquisition method, including: The time-varying correlation function is obtained as follows: ； in, For the first t Time-varying correlation function during the training process This is the maximum value of the time-varying correlation function, and it is set to 0.95; This is the minimum value of the time-varying correlation function, and it is set to 0.01; Let π be the mathematical constant, T be the maximum number of training iterations, and cos be the cosine function. Based on the time-varying correlation function, the fuzzy information acquisition factor is obtained as follows: ； in, For the first t Fuzzy information acquisition factors during the training process. This is the maximum value of the fuzzy information collection factor, and it is set to 0.

95. Based on the fuzzy information acquisition factor, the first target individual is subjected to fuzzy information joint optimization to obtain the second target individual: ； ； ； ； in, For the first n The first target individual, For the first n A second target individual, For the first n The first target individual in the first t Update speed during each training process For the first n The first target individual in the first t +1 update speed during training. As the first joint optimization control factor, As the second joint optimization control factor, To and Different individuals with the first random number, To and Different individuals with second random numbers, The third random number between (0,1) The fourth random number between (0, 1); The maximum value of the first joint optimization control factor is set to 2; The minimum value of the first joint optimization control factor is set to 0.4; The maximum value of the second joint optimization control factor is set to 2; This is the minimum value of the second joint optimization control factor, and it is set to 0.

4.

8. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, An adaptive multi-point mutation information acquisition method is used to perform global information joint optimization on the second target individual to obtain the third target individual, including: Three other second target individuals are randomly matched for the second target individual to obtain the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual corresponding to the second target individual; Based on the first global information joint optimization individual, the second global information joint optimization individual, and the third global information joint optimization individual, the first global mutation information and the second global mutation information are obtained as follows: ； ； ； in, For the first global information, jointly optimize the individual's first... d The model parameters are d = 1, 2, ..., D, where D is the total number of model parameters in an individual. For the second global information joint optimization of the individual's first d 3D model parameters, For the third global information joint optimization of the individual's first d 3D model parameters, For the first m The second target individual's first d The first global mutation information corresponding to the dimensional model parameters. For the first m The second target individual's first d The second global mutation information corresponding to the dimensional model parameters. The solution spatial location parameters of an individual are jointly optimized based on the first global information. The solution spatial location parameters of the individual are jointly optimized using the second global information. The solution spatial location parameters of the individual are jointly optimized using third global information. This is a preset constant term, set to 0.01; For the first k The solution spatial location parameters of an individual are jointly optimized using global information, k=1,2,3. For the first k The fitness value of an individual is jointly optimized using global information. The fitness value of the globally optimal individual is given by , and T is the maximum number of training iterations. Based on the first and second global mutation information, the second target individual is jointly optimized using global information to obtain the third target individual: ； in, For the first m The second target individual's first d 3D model parameters, For the first m The third target individual's first d 3D model parameters, It is a natural constant. The first of the globally optimal individuals d 3D model parameters; Determine whether the fitness value of the third target individual has increased. If so, retain the result of this global information joint optimization; otherwise, filter this global information joint optimization and directly use the original second target individual as the third target individual.

9. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on the spatial geographic coordinate labels and bounding box coordinates, the identified fire hazards are mapped onto a 3D city model, generating a fire hazard report that includes the hazard location, type, and thermal anomaly status, including: Extract the pixel coordinates of the four vertices of the bounding box of the potential hazard in the image; Using the mapping relationship of the spatial geographic coordinate labels, the pixel coordinates of the four vertices are converted into real-world three-dimensional geographic coordinates; The converted 3D geographic coordinates are registered with the coordinate system of the 3D urban building model, and a hazard visualization icon or 3D bounding box is generated at the corresponding spatial location of the 3D urban model.

10. The method for identifying urban building fire hazards based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, When a specific high-risk hazard is identified or an abnormally high temperature point is detected by the infrared thermal imager, the drone is triggered to hover and take repeat photos, including: Determine whether the hazard category belongs to the preset high-risk hazard set or whether the temperature data collected by the infrared thermal imager exceeds the preset temperature threshold. If any of the above conditions are met, control the drone to interrupt the current cruise route and hover at the current position; Control the gimbal to adjust the shooting angle, and continuously capture images from multiple angles of the potential hazard area as reshoot data.

Images