Fire-fighting unmanned aerial vehicle monitoring method and system based on deep learning

Abstract

The application discloses a kind of fire-fighting unmanned aerial vehicle monitoring method and system based on deep learning, comprising: first response to the fire alarm instruction for specific to be monitored area, subsequently instructs fire-fighting unmanned aerial vehicle to collect the video data of the area according to monitoring request.Through the analysis to the target building structure video data collected, in combination with the preset simulated fire image feature library, the fire situation analysis result is analyzed and obtained using deep learning algorithm.According to the analysis result, the system further generates fire strategy search request, and selects the most suitable fire handling strategy from the strategy storage warehouse prepared in advance to guide the on-site fire action.Such design can accurately judge and strategy formulation for fire situation in real time and automatically, greatly improve the emergency response speed and processing efficiency of fire department.In addition, through the application of deep learning technology, the accuracy of fire situation analysis can be continuously optimized, providing a new and efficient solution for fire safety management.

Description

Technical Field

[0001] This invention relates to the field of smart fire protection technology, and more specifically, to a fire protection drone monitoring method and system based on deep learning. Background Technology

[0002] With the acceleration of urbanization and the increasing number of high-rise buildings, the probability and risk of fires are also increasing. Traditional fire monitoring and response methods rely on manual patrols and fixed monitoring facilities, which suffer from problems such as long response times and numerous blind spots, making it difficult to meet the rapidly developing urban safety needs. In recent years, the rapid development of drone technology has brought new solutions to the field of fire monitoring. Drones can quickly reach the fire scene and transmit video data in real time, but how to quickly and accurately extract useful information from massive amounts of video data is the key to improving the efficiency of fire monitoring, and it is also a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0003] The purpose of this invention is to provide a fire-fighting drone monitoring method and system based on deep learning.

[0004] In a first aspect, embodiments of the present invention provide a fire-fighting drone monitoring method based on deep learning, comprising:

[0005] In response to a fire alarm command for the area to be monitored, a monitoring request is sent to the fire-fighting drone;

[0006] Receive video data of the target building structure in the area to be monitored from the firefighting drone based on the monitoring request;

[0007] Based on the target building structure video data and the features of the pre-set simulated fire building images, the fire situation analysis results of the area to be monitored are obtained.

[0008] A fire strategy search request is generated based on the fire analysis results.

[0009] The target fire handling strategy is determined from the strategy storage repository based on the fire strategy search request.

[0010] In a second aspect, embodiments of the present invention provide a server system, including a server, the server being used to execute the method described in the first aspect.

[0011] Compared to existing technologies, the beneficial effects of this invention include: The fire-fighting drone monitoring method and system disclosed in this invention, based on deep learning, includes: firstly, responding to a fire alarm command for a specific monitored area; then, directing a fire-fighting drone to collect video data of the area according to the monitoring request. By analyzing the collected target building structure video data and combining it with a pre-set simulated fire image feature library, deep learning algorithms are used to analyze and derive fire situation analysis results. Based on the analysis results, the system further generates a fire strategy search request and selects the most suitable fire handling strategy from a pre-prepared strategy storage repository to guide on-site fire-fighting actions. This design enables real-time and automatic accurate judgment and strategy formulation of fire situations, significantly improving the emergency response speed and processing efficiency of fire departments. Furthermore, the application of deep learning technology can continuously optimize the accuracy of fire situation analysis, providing a new and efficient solution for fire safety management. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating the steps of a deep learning-based fire-fighting drone monitoring method provided in an embodiment of the present invention;

[0014] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0016] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] In order to solve the technical problems mentioned in the background art Figure 1 This is a flowchart illustrating the deep learning-based fire-fighting drone monitoring method provided in this embodiment. The following is a detailed description of the deep learning-based fire-fighting drone monitoring method.

[0018] Step S201: In response to a fire alarm command for the area to be monitored, a monitoring request is sent to the fire-fighting drone;

[0019] Step S202: Receive video data of the target building structure in the area to be monitored from the fire-fighting drone based on the monitoring request;

[0020] Step S203: Based on the target building structure video data and the features of the preset simulated fire building image, analyze the fire situation analysis results of the area to be monitored.

[0021] Step S204: Generate a fire strategy search request based on the fire analysis results;

[0022] Step S205: Determine the target fire handling strategy from the strategy storage warehouse according to the fire strategy search request.

[0023] In an embodiment of the invention, for example, a fire alarm on a high-rise building in a commercial district of a city is triggered. This information is quickly transmitted to the fire department's central server. Upon receiving the fire alarm command for the high-rise building in the commercial district, the server responds immediately and sends a monitoring request to a nearby deployed fire-fighting drone. Upon receiving the server's monitoring request, the fire-fighting drone immediately starts, flies to the area of ​​the high-rise building that triggered the alarm, and begins collecting video data of the target building structure in that area. The server receives the video data transmitted back by the fire-fighting drone in real time. This data clearly shows the fire intensity, smoke conditions, and possible fire source locations outside the building. The server uses deep learning algorithms, combined with pre-set simulated fire building image features, to analyze the received video data. Through analysis, the server can identify the size of the fire, its spread speed, and the potentially affected parts of the building. The server also uses historical data and real-time weather information to assess the possible development trend of the fire. Based on the fire analysis results, the server needs to determine an effective fire response strategy. Based on the conclusions of the fire analysis, the server generates a fire strategy search request, which includes key information such as the specific location of the fire, its size, and the building type. The server stores multiple preset processing strategies for different types of fires. Based on the search request, the server retrieves and matches the most suitable target fire response strategy for the current fire situation from the strategy store. This strategy may include information such as the specific flight path of the drone, the type of extinguishing agent to be used, and whether ground fire brigades need to be notified. Through this process, the server can effectively utilize firefighting drones and deep learning technology to respond quickly and intelligently to fires, thereby minimizing the losses caused by fires.

[0024] To more clearly describe the solutions provided in the embodiments of this application, a more detailed explanation is provided below.

[0025] In this embodiment of the invention, the monitored area, for example, refers to a specific geographical area or interior space of a building that requires fire monitoring. This area can be a commercial district, residential area, or industrial area in a city. For instance, if a city's central business district becomes a high-risk fire area due to its dense high-rise buildings and large pedestrian traffic, then this central business district can be defined as a monitored area. A fire alarm command is an alarm signal automatically or manually triggered when a fire detector, smoke alarm, or other fire monitoring device detects signs of fire. This command typically includes information such as the location and time of the fire, and the possible severity of the fire. In a large shopping mall, if a smoke alarm on a certain floor detects abnormal smoke concentration, it will automatically send a fire alarm command to the fire protection system, indicating that a fire may have occurred on that floor. A monitoring request is an instruction issued by the server to a fire-fighting drone, requesting the drone to proceed to the monitored area for aerial surveillance and data collection. This request typically includes specific information such as the drone's flight path, monitoring focus, and data collection type. After confirming a fire in the shopping mall, the central server will send a monitoring request to nearby fire-fighting drones. This request instructs a drone to fly over the shopping mall, collecting real-time data such as fire scene video, smoke concentration, and temperature, and transmitting this data back to the server for analysis. A firefighting drone is an unmanned aerial vehicle equipped with cameras, sensors, and other firefighting equipment. It can quickly reach the fire scene, providing real-time fire information to assist firefighters in fire reconnaissance and firefighting operations. Upon receiving the monitoring request from the server, the firefighting drone will immediately start and fly to the fire scene according to a predetermined flight path. During flight, the drone will use its onboard cameras and sensors to collect various information about the fire scene in real time and transmit this information back to the server wirelessly. Simultaneously, the drone can adjust its flight path and monitoring focus based on changes in the fire situation and instructions from the server.

[0026] Furthermore, "receiving" here refers to the process by which a server or data processing center acquires or receives information from firefighting drones. This process is typically achieved through wireless communication technologies such as Wi-Fi, 4G / 5G networks, and satellite communication. When a firefighting drone is in the air capturing video and collecting data from the area to be monitored, it encodes this information in real time and sends it to a ground server via a wireless communication network. The receiving equipment or software on the server decodes this information and stores it for further analysis and processing. "Target building structure video data" refers to real-time video data about the building structure collected by the firefighting drone in the area to be monitored. This data can display detailed information such as the building's appearance, structure, and damage, which is crucial for assessing the fire's spread and developing firefighting strategies. In an urban fire, a firefighting drone can fly over a burning building and use its onboard high-definition camera to capture video data of the building's appearance and structure. This data can be transmitted to a server in real time for analysis and processing, helping rescue personnel understand the impact of the fire on the building and potential safety hazards inside. Simultaneously, this data can also be used for subsequent fire cause investigations and liability determination. "Target building structure video data" refers to real-time video data collected by firefighting drones regarding a specific building structure (such as the exterior and interior structure). This data can reveal important information such as the current state of the building, the spread of fire, and the distribution of smoke. For example, if a fire breaks out in a building, a firefighting drone quickly arrives at the scene and begins filming. The video data it captures shows flames emerging from windows on a certain floor, the spread of smoke, and the extent of damage to the building's facade. This data constitutes the target building structure video data. "Pre-set simulated fire building image features" refers to fire-related building image feature data pre-stored in the system or database. This data is typically extracted from a large number of historical fire images using machine learning or deep learning algorithms to help the system identify and analyze fire signs in real-time video data. In fire monitoring systems, various simulated fire building image features can be pre-set, such as the color, shape, and dynamic change patterns of flames, and the density, color, and spread rate of smoke. When the system receives real-time video data, it compares and matches these features with the pre-set features to more accurately identify and analyze the fire. After receiving video data of the target building structure transmitted by the firefighting drone, the server runs specialized fire analysis software. This software analyzes the image content in the video frame by frame, comparing and matching it with preset simulated fire building image features. By analyzing features such as the size, shape, and color of the flames, as well as the density and diffusion of smoke, the software can generate a detailed analysis report on the fire situation. The "fire analysis results" refer to the conclusions or reports drawn from the above analysis process regarding the fire situation.This result typically includes information such as the size of the fire, its rate of spread, the areas that may be affected, and recommended response measures. After analysis, the system can arrive at the following fire analysis result: the fire is spreading from one floor of the building to adjacent floors, smoke has spread to multiple floors and corridors, and it is recommended to immediately evacuate people from the building and dispatch more firefighting forces for firefighting operations. This result will be quickly communicated to on-site commanders and rescue teams to guide their actions and decisions.

[0027] Furthermore, a "fire strategy" refers to a detailed response plan or scheme developed for a specific fire situation. This strategy typically includes multiple aspects such as firefighting methods, resource allocation, personnel coordination, and safety assurance, aiming to control the fire most effectively and protect the safety of people and property. A "search request" here refers to an instruction or request that instruct the system or database to search for information matching specific conditions or parameters. In a fire monitoring context, this request is usually based on fire analysis results, aiming to find the most suitable response strategy or scheme for the current fire situation. For example, if a fire breaks out in a high-rise building in a city, the fire strategy search request could include data such as the building's structural information, number of floors, fire source location, current wind direction and speed, etc. The system will use this information to search and filter for the most suitable firefighting strategy. A "strategy repository" is a database or system that stores and manages various fire handling strategies. It contains a large number of pre-developed response strategies for different fire scenarios, which may be based on historical experience, expert advice, or simulation exercises. A "target fire handling strategy" refers to the specific strategy or scheme selected after a search and determination process to guide the current fire response. This strategy should be the optimal option selected from the strategy repository based on the actual situation and needs of the current fire. In the scenario of the aforementioned urban high-rise fire, after a search and determination process, the system can select a target fire handling strategy that includes specific measures such as using high-altitude fire extinguishing equipment, evacuating people inside the building, and ensuring the safety of surrounding buildings. This strategy will serve as a guiding plan for subsequent firefighting operations.

[0028] In this embodiment of the invention, the aforementioned step S203 can be implemented through the following examples.

[0029] Acquire target building structure video data obtained from monitoring the target building in the area to be monitored;

[0030] Based on video content feature attributes that match the feature attribute category of the target building, the building structure features corresponding to each video frame are extracted from the target building structure video data; the video content feature attributes are attributes that meet the requirements after integrating the drone shooting angles corresponding to the target video frames; the target video frame is the video frame corresponding to the feature extraction operation.

[0031] Based on the building structure features corresponding to each video frame, local fire building image features of the monitored area are obtained.

[0032] The local fire building image features are compared and analyzed with the preset simulated fire building image features, and the fire analysis results for the monitored area are obtained based on the comparison and analysis results; the preset simulated fire building image features are local fire building image features obtained by performing pre-fire simulation analysis on the target building structure video data corresponding to the target building in the fire monitoring area.

[0033] In this embodiment of the invention, for example, the server receives real-time video streams from firefighting drones. These video data are taken of target buildings in the area to be monitored. For instance, if a fire breaks out in a large commercial complex, the firefighting drone quickly flies to the fire scene and takes high-definition video of the building's exterior and structure. This video data is transmitted to the server in real time for subsequent fire analysis and processing. The server processes the received video data, first identifying video content feature attributes that match the target building's characteristic attribute categories. For example, the server analyzes features such as the shooting angle, building appearance, and structural details in the video frames and matches these features with a pre-stored building feature library. Then, the server extracts the corresponding building structural features from each video frame, such as the building's outline, the position and size of windows, and the height of floors. The server uses the extracted building structural features, combined with fire monitoring algorithms, to analyze possible fire signs in the video frames. For example, the server detects thick smoke and flames emanating from a window in a certain video frame, while the building structural features around the window also show abnormal heat radiation and deformation. Based on these features, the server generates localized fire image features of the monitored area, including flame color, shape, dynamic changes, and smoke density and diffusion rate. The server then compares these generated localized fire image features with pre-stored simulated fire image features. For example, the server might find that the flame color and shape in the current video frame are highly similar to a specific type of fire in a pre-stored simulated fire image. Based on this similarity comparison, the server determines that the fire is spreading rapidly and may affect other parts of the building. Based on the comparison analysis results, the server generates fire analysis results for the monitored area, including the fire size, spread rate, potentially affected areas, and recommended response measures. This information is promptly communicated to on-site commanders and rescue teams to guide their firefighting operations and ensure personnel safety.

[0034] In a more detailed implementation, suppose there is a 20-story commercial building called "Shengshi Building" in the city center. This building houses various shops, offices, and entertainment facilities. One afternoon, a fire breaks out in the kitchen of a restaurant on the 5th floor. The fire department responds quickly and dispatches a firefighting drone for aerial monitoring. The drone flies over Shengshi Building and begins capturing high-definition video of it. The drone pays particular attention to the location of the 5th-floor restaurant, as that is the source of the fire. The video data is transmitted in real time to the fire department's server. After receiving the video data from the drone, the server begins processing it. First, the server uses image recognition technology to identify the building's outline, floors, windows, and other structural features. Due to the change in the drone's shooting angle, the server also needs to calibrate and integrate these features to ensure the accuracy of subsequent analysis. While analyzing the video data, the server detects thick smoke and flames emanating from the windows of the 5th-floor restaurant. The server further analyzes the characteristics of these flames and smoke, such as color, shape, and dynamic changes. Simultaneously, the server, combined with the building's structural features, determines the areas and spread paths that the flames and smoke may affect. The server compares the real-time flame and smoke features with pre-stored images of simulated fire buildings. The comparison reveals a high degree of similarity between the current fire's characteristics and a specific type of fire in the simulation database—fires typically characterized by rapid spread and high smoke density. Based on this information, the server generates a fire analysis result: the fire is spreading rapidly and may soon affect adjacent areas on the 6th and 4th floors; smoke has already spread to multiple floors, necessitating immediate evacuation of residents. Based on this analysis, the fire department quickly formulates a rescue strategy: dispatching more firefighters to the scene for firefighting and rescue operations; and guiding residents to safety via broadcasts and evacuation routes. Simultaneously, drones continue to monitor the fire's progress from the air, providing real-time information support for subsequent rescue operations.

[0035] In this embodiment of the invention, the target building is the main building in the fire, and the video content feature attribute is the flame and smoke attribute; the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data according to the video content feature attribute matching the target building feature attribute category can be implemented through the following example.

[0036] Based on the flame and smoke attributes from the drone's shooting angle, the building structure features corresponding to each video frame are extracted from the target building structure video data.

[0037] In this embodiment of the invention, exemplarily, a 15-story office building named "Yuelai Building" caught fire in a real fire incident. The fire department responded quickly and dispatched a fire-fighting drone to monitor the fire from the air. The video data captured by the drone was transmitted in real time to the fire department's server for analysis and processing. The server received the video data of the Yuelai Building fire transmitted by the drone. This data clearly showed flames and smoke emerging from certain windows and vents of the building. Since flames and smoke are the most obvious features of a fire, the server focused its analysis on these features. The server first preprocessed the video data to identify and extract the attributes of the flames and smoke. These attributes may include the color, brightness, and shape of the flames, as well as the concentration and spread rate of the smoke. By analyzing these attributes, the server can more accurately determine the size and spread of the fire. Next, the server extracted the building structural features from each video frame based on changes in the drone's shooting angle. These features may include the building's floor height, window positions, and vent layout. Since changes in the shooting angle affect the accuracy and consistency of the features, the server needs to use image processing and computer vision technologies to calibrate and integrate these features. For example, in one video frame, a drone captures a shot from above showing thick smoke billowing from a window on the 5th floor of the Yuelai Building. The server identifies the window's location and size, and, combined with the building's structural features, determines the area the smoke might affect and its spread path. This information is crucial for developing effective firefighting strategies. Finally, the server combines the extracted structural features with flame and smoke attributes to generate a comprehensive fire analysis. This result includes not only the size and spread of the fire but also detailed information about the fire's location, potentially affected areas, and optimal firefighting routes. This information is promptly communicated to on-site commanders and rescue teams to guide their firefighting operations and ensure personnel safety.

[0038] In this embodiment of the invention, the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data based on the flame and smoke attributes of the drone's shooting angle can be implemented through the following example.

[0039] Based on the flame and smoke attributes from multiple drone shooting angles, the drone shooting angle building structure features corresponding to the flame and smoke attributes of each video frame and the multiple drone shooting angles are extracted from the target building structure video data.

[0040] For each video frame in each video frame, the drone shooting angle building structure features corresponding to the flame and smoke attributes of the multiple drone shooting angles are merged to obtain the drone shooting angle merged features of each video frame.

[0041] The corresponding building structure features are obtained by merging the drone shooting angle features corresponding to each video frame.

[0042] In this embodiment of the invention, for example, the server receives video data from three drones, which capture images of the Yuelai Building from the front, side, and diagonally above. Each drone's video clearly shows different characteristics of the flames and smoke. The server first extracts the building structure features corresponding to each video frame based on the drone's shooting angle and the attributes of the flames and smoke. For example, in the drone video data captured from the front, the server can clearly see features such as the building's entrance, floor layout, and window arrangement; in the drone video data captured from the side, the server can identify features such as the building's side profile, ventilation openings, and emergency exits; in the drone video data captured from diagonally above, the server can observe features such as the building's roof structure, upper-floor windows, and smoke diffusion. For each video frame, the server merges the building structure features from different drones. This process needs to consider the overlap and complementarity of shooting angles, as well as the spatial relationships and temporal consistency between features. By merging these features, the server obtains a more comprehensive and accurate description of the building structure features. For example, in a certain video frame, both the front and side drones captured thick smoke emanating from a window on the 5th floor of the Yuelai Building. By merging the features from these two angles, the server can more accurately determine the source, spread path, and potentially affected areas of the smoke. Simultaneously, the drone video data taken from an oblique angle provides a bird's-eye view of the smoke's spread in the air, further validating the smoke's diffusion trend. After merging, the server obtains the merged drone shooting angle features for each video frame. These features not only include information from a single drone shooting angle but also incorporate complementary information from multiple angles. Finally, the server uses these merged features to obtain the final building structure features for each video frame. These final building structure features provide fire departments with more comprehensive and accurate information about the fire scene. Through the analysis and processing of these features, fire departments can quickly formulate effective firefighting strategies, evacuation routes, and rescue plans to minimize fire damage and casualties.

[0043] In this embodiment of the invention, the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data based on the video content feature attributes that match the feature attribute category of the target building can be implemented through the following example.

[0044] Obtain video content feature attributes that match the feature attribute category of the target building in the video content feature dimension;

[0045] Based on the video content feature attributes under the video content feature dimension, the building structure features corresponding to each video frame are extracted from the target building structure video data.

[0046] In an embodiment of the invention, for example, in a fire monitoring task, the server needs to analyze a segment of real-time video data about a commercial building to determine if a fire exists. This video data is captured by surveillance cameras installed on the exterior of the commercial building and transmitted to the server in real time for processing. The server first identifies the characteristic attribute category of the target building; for example, in this case, the characteristic attribute category could be "commercial building." Then, the server obtains video content characteristic attributes that match the characteristic attribute category "commercial building" under the video content characteristic dimension. These characteristic attributes may include the building's appearance features (such as window layout, floor height, etc.), structural features (such as wall materials, load-bearing structures, etc.), and environmental features (such as surrounding roads, other buildings, etc.). Next, the server processes the target building's structural video data based on the obtained video content characteristic attributes. The server uses image processing and computer vision technology to analyze each video frame and extract the building structural features related to the "commercial building" characteristic attributes. For example, the server can identify features such as the window layout, floor height, and wall materials of the commercial building in the video frame and represent and store these features in the form of data. In practice, servers can use deep learning models to extract features from video frames. These models, after extensive training, can accurately identify various building structural features and transform them into data formats suitable for subsequent analysis. Through this process, the server can obtain video content features that match the target building's characteristic attribute categories, thus providing crucial information for subsequent fire analysis and judgment. These building structural features not only help identify the location of the fire source and determine the fire's spread trend, but also provide strong support for fire departments to formulate effective firefighting and rescue plans. Furthermore, this method can also be applied to monitoring tasks in other similar scenarios, such as fire early warning and monitoring in factories and warehouses.

[0047] In this embodiment of the invention, the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data based on the video content feature attributes under the video content feature dimension can be implemented through the following example.

[0048] Based on the video content feature attributes under multiple video content feature dimensions, extract the building structure features to be processed corresponding to each video frame and each of the multiple video content feature dimensions from the target building structure video data;

[0049] For each video frame in each video frame, each video frame is merged with the building structure features to be processed corresponding to the multiple video content feature dimensions to obtain the multidimensional merged features of each video frame.

[0050] The corresponding building structure features are obtained based on the multidimensional merging features corresponding to each video frame.

[0051] In this embodiment of the invention, for example, in a fire monitoring task, the server is processing real-time video data of a commercial complex. This video data is captured by multiple cameras from different angles and transmitted to the server for processing in real time. Commercial complexes encompass various building types, such as shopping malls, office buildings, and hotels, resulting in complex and diverse architectural structural features. The server first identifies the characteristic attribute category of the target building; for example, in this case, the characteristic attribute category could be "commercial complex." Then, the server extracts the corresponding architectural structural features for each video frame from the target building's structural video data across multiple video content feature dimensions, such as appearance features, structural features, and environmental features. For example, in the appearance feature dimension, the server can identify features such as the window layout, floor height, and wall color of the commercial complex in the video frame; in the structural feature dimension, the server can identify features such as the load-bearing structure and building materials; and in the environmental feature dimension, the server can identify features such as surrounding roads, other buildings, and pedestrian traffic. For each video frame, the server merges it with the architectural structural features corresponding to the multiple video content feature dimensions. During the merging process, the server needs to consider the spatial relationships and temporal consistency between features of different dimensions to ensure that the merged features accurately reflect the structural characteristics of the target building. For example, in a certain video frame, the server simultaneously identifies the window layout, load-bearing structure, and surrounding road conditions of a commercial complex. By merging these features, the server can obtain a more comprehensive and accurate description of the building's structural features, including the location and size of windows, the type and distribution of load-bearing structures, and the relationship between roads and buildings. After merging, the server obtains multi-dimensional merged features corresponding to each video frame. These features not only include information from individual video content feature dimensions but also integrate complementary information from multiple dimensions. Finally, the server obtains the final building structural features corresponding to each video frame based on these multi-dimensional merged features. These final building structural features provide more comprehensive and accurate information support for fire monitoring tasks. Through the analysis and processing of these features, the server can more accurately identify the location of the fire source, judge the trend of fire spread, and issue alarms in a timely manner, providing strong auxiliary decision-making basis for fire departments. This method can also be applied to monitoring tasks in other similar scenarios, such as fire early warning and monitoring in large public places and transportation hubs.

[0052] In this embodiment of the invention, the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data based on the video content feature attributes that match the feature attribute category of the target building can be implemented through the following example.

[0053] The convolutional units in the pre-trained fire building recognition model are used to extract the building structure features corresponding to each video frame from the target building structure video data.

[0054] The convolutional unit is used to perform feature extraction operations based on video content feature attributes that match the feature attribute category of the target building.

[0055] The process of obtaining local fire building image features of the monitored area based on the building structure features corresponding to each video frame includes:

[0056] The local fire building feature extraction unit in the fire building identification model extracts the local fire building image features of the monitored area based on the building structure features corresponding to each video frame.

[0057] In this embodiment of the invention, for example, the server deploys a pre-trained fire building recognition model, which has the ability to identify and extract building structural features from video data. In a fire monitoring task, the server receives real-time video stream data from a commercial district and needs to analyze this data to determine if there is a fire risk. The server first preprocesses the received real-time video stream data, including decoding and format conversion, to ensure that the data can be correctly processed by the fire building recognition model. Then, the server inputs the preprocessed video data into the fire building recognition model. The convolutional units in the model begin working, performing feature extraction operations on each video frame based on video content feature attributes (such as flames, smoke, etc.) that match the target building feature attribute category. The convolutional units gradually extract low-level features such as edges and textures from the video frames through convolution operations and activation functions, and further combine them into higher-level building structural features. After completing the extraction of building structural features for each video frame, the local fire building feature extraction unit in the model begins working. This unit further processes the building structural features corresponding to each video frame, focusing on extracting local image features related to the fire. These local features can include the shape, color, and dynamic changes of flames, as well as key information such as the spread and concentration of smoke. Finally, the server integrates the local fire building image features of each video frame to form a fire risk feature sequence for the area over a period of time. These feature sequences provide crucial information for subsequent fire detection, alarm, and location. Through these steps, the server can efficiently extract building structure features and local fire building image features that match the target building's feature attribute categories from real-time video stream data using a pre-trained fire building recognition model. These features provide critical information support for fire monitoring tasks, facilitating accurate and timely fire detection and alarm. Furthermore, this method can also be applied to monitoring tasks in other similar scenarios, such as fire early warning and monitoring in factories and warehouses.

[0058] In this embodiment of the invention, the fire building identification model is obtained in the following manner.

[0059] Acquire video data of multiple sample target building structures;

[0060] The convolutional units in the initial fire building identification model are used to extract the sample building structure features corresponding to each sample video frame from the sample target building structure video data.

[0061] The local fire building feature extraction unit in the initial fire building identification model extracts the sample fire building features for the sample building structure features corresponding to each sample video frame.

[0062] The training error is obtained based on the fire-related building characteristics and structural characteristics of the sample buildings.

[0063] Based on the training error, the convolutional units and local fire building feature extraction units in the initial fire building recognition model are optimized, and the training process is re-executed until the fire building recognition model that meets the preset conditions is obtained.

[0064] In this embodiment of the invention, for example, to construct an accurate and reliable fire-related building identification model, the server needs to perform a series of model training and optimization operations. These operations are based on a large amount of sample target building structure video data, aiming to enable the model to accurately identify building structure features and fire-related features from the video data. The server first acquires multiple sample target building structure video data. This data comes from different building scenarios, including commercial buildings, residential buildings, factories, etc., covering various possible building structures and fire situations. Each sample video contains multiple video frames used to extract building structure features and fire features. The server uses the convolutional unit in the initial fire-related building identification model to extract building structure features from each sample video frame. The convolutional unit extracts low-level features such as edges and corners in the video frame through convolution operations and pooling operations, and gradually combines them into higher-level building structure features. Subsequently, the server uses the local fire building feature extraction unit in the initial model to further extract fire-related features for the building structure features corresponding to each sample video frame. These features may include the shape, color, and dynamic changes of flames, as well as key information such as the diffusion range and concentration of smoke.

[0065] Based on the extracted fire-related building features and structural features of the sample buildings, the server calculates the training error. The training error reflects how well the model fits the sample data under the current parameter settings. A large error indicates a significant difference between the model's predictions and the actual situation, requiring optimization. Based on the calculated training error, the server optimizes the convolutional units and local fire-related building feature extraction units in the initial fire-related building identification model. Optimization operations may include adjusting model parameters, increasing or decreasing the number of network layers, and changing the activation function. After optimization, the server re-executes the training process, using the optimized model again to extract features and calculate errors on the sample data. Through multiple iterations of training and optimization, the model's performance gradually improves, and its ability to identify building structural features and fire features is continuously enhanced. During training, the server periodically evaluates whether the model's performance meets preset conditions. These conditions may include achieving certain levels in metrics such as accuracy, recall, and F1 score. When a model performs well in the evaluation and meets the preset conditions, the server selects it as the final fire-related building identification model. Once a satisfactory fire-related building identification model is obtained, the server deploys it to the real-world application environment. In real-time monitoring tasks, the server uses this model to process and analyze received video data to accurately identify fire risks and issue timely alarms. Through the above steps, the server can construct an accurate and reliable fire building identification model. This model can automatically extract building structural features and fire characteristics from video data, providing strong support for fire monitoring and early warning. Furthermore, this method can also be applied to monitoring tasks in other similar scenarios, such as fire early warning and monitoring in factories and warehouses.

[0066] In this embodiment of the invention, the aforementioned step of obtaining training error based on the sample fire building features and the sample building structure features can be implemented through the following examples.

[0067] The feature error of the fire building image is obtained based on the sample fire building features;

[0068] Obtain counterexample pairs, which include sample target building structure video data labeled with multiple fire types;

[0069] Based on the sample building structure features corresponding to the sample target building structure video data of the counterexample, the sample instance pair error is obtained;

[0070] The training error is obtained based on the feature error of the fire building image and the error of the sample instance.

[0071] In this embodiment of the invention, exemplarily, during the training of the fire building recognition model, the server needs to accurately calculate the training error in order to optimize the model. The training error reflects the difference between the model's prediction and the true label. To more comprehensively evaluate the model's performance, the server not only considers the fire building feature error under normal circumstances but also introduces counterexample pairs to evaluate the model's ability to handle complex and easily confused situations. The server first calculates the fire building image feature error based on sample fire building features. This is done by comparing the differences between the fire building image features predicted by the model and the actual fire building image features. For example, in a sample video frame, the model may incorrectly identify a non-flame area as a flame or miss a real flame area. These errors lead to an increase in the fire building image feature error. To further improve the robustness of the model, the server obtains a set of counterexample pairs. These counterexample pairs include sample target building structure video data labeled with various fire types. For example, in some video frames, there may be other objects with similar shapes and colors to flames (such as red flags, lights, etc.), which are easily misidentified as flames by the model. By introducing these easily confused scenarios as counterexamples into the training process, the server can help the model better distinguish real fire features from other interfering factors. For the sample target building structure video data in the counterexample pairs, the server extracts the corresponding sample building structure features and calculates the sample instance pair error. This is done by comparing the model's predictions in these easily confused scenarios with the true labels. For example, if the model incorrectly identifies a red flag as a flame, it will lead to an increase in the sample instance pair error. Finally, the server calculates the training error based on the fire building image feature error and the sample instance pair error. This training error reflects the model's performance on the overall sample data. By continuously optimizing model parameters and adjusting the structure, the server can reduce the training error and improve the model's ability to recognize fire building features. Through these steps, the server can more comprehensively evaluate the performance of the fire building recognition model and make targeted optimizations to address any shortcomings. This not only helps improve the model's accuracy in recognizing fire building features under normal conditions but also enhances the model's robustness in handling complex and easily confused scenarios. Ultimately, this will provide more reliable technical support for fire monitoring and early warning.

[0072] In this embodiment of the invention, the aforementioned step of obtaining the local fire building image features of the area to be monitored based on the building structure features corresponding to each video frame can be implemented through the following example.

[0073] Based on the drone shooting angle corresponding to each video frame in the target building structure video data, the building structure features corresponding to each video frame are combined to obtain the building structure features of the target building structure video data.

[0074] A feature extraction operation is performed on the building structure features of the target building structure video data to obtain the local fire building image features of the area to be monitored.

[0075] In an embodiment of the invention, exemplarily, in a fire monitoring system, a server receives target building structure video data from a drone. This data, captured by the drone from different shooting angles, provides a comprehensive view of the building structure. The server needs to process this data to identify and extract local building features related to the fire. The server receives target building structure video data from a drone. This data contains multiple video frames, each capturing the structural details of the building from a different angle. For each video frame, the server uses a pre-trained model or algorithm to extract its corresponding building structure features. These features may include the building's edges, corners, textures, etc., which together constitute the building's structural outline and appearance. The server knows the drone shooting angle corresponding to each video frame in the target building structure video data. Using this information, the server combines the building structure features corresponding to each video frame. This combination considers feature overlap and complementarity at different angles, thereby obtaining a more comprehensive and accurate representation of the building structure features. For example, one video frame may capture the front of the building, while another video frame captures the side. By combining the features of these two video frames, the server can obtain a more complete view of the building structure. After obtaining the building structure features from the target building structure video data, the server further performs feature extraction operations. The purpose of this step is to extract fire-related local building features from the combined features. This involves identifying features such as the color, shape, and dynamic changes of flames, as well as the diffusion patterns and concentration changes of smoke. For example, the server may detect that the brightness and color changes in a certain area match the characteristics of flames, or detect that the smoke diffusion pattern is similar to the smoke diffusion pattern during a fire. These features are all considered fire-related local building features. By performing the above feature extraction operation, the server finally obtains the local fire-related building image features of the area to be monitored. These features provide important basis for subsequent fire detection, alarm, and location. If the server detects that certain features match known fire features or exceed a preset threshold, it can trigger an alarm system or notify relevant personnel. Through the above steps, the server can effectively process target building structure video data from drones and extract fire-related local building features. This method improves the accuracy and reliability of fire monitoring systems, helping to detect and respond to fire risks in a timely manner. At the same time, it also demonstrates the potential application value of drone technology in public safety fields such as fire monitoring.

[0076] In this embodiment of the invention, the aforementioned step of extracting the building structure features corresponding to each video frame from the target building structure video data based on the video content feature attributes that match the feature attribute category of the target building can be implemented through the following example.

[0077] Key objects are obtained from the video data of the target building structure;

[0078] Obtain video content feature attributes that match the feature attribute category of the target building;

[0079] Based on the video content feature attributes, extract the building structure features corresponding to each video frame in the key object.

[0080] In an embodiment of the invention, exemplarily, in a fire monitoring system, the server needs to process video data of a target building structure captured by a drone. To effectively identify building structural features, the server first determines the key object in the video, i.e., the target building, and obtains video content feature attributes that match the feature attribute categories of the target building. Based on these feature attributes, the server can accurately extract building structural features from each video frame. The server identifies and obtains the key object, i.e., the target building, from the target building structure video data. This can be achieved through an image recognition algorithm, which can identify the main building in a video frame and distinguish it from the background or other objects. For example, a drone captures a video of a city block, including multiple buildings. The server uses an image recognition algorithm to accurately identify one specific building as the key object. The server obtains video content feature attributes that match the feature attribute categories of the target building. These feature attributes can include the building's shape, size, color, texture, etc. To obtain these feature attributes, the server can pre-train on different types of buildings to learn their unique feature attributes. Taking the aforementioned city block video as an example, the server identifies the key object as a high-rise building with a specific color and texture. The server determines that these feature attributes match the high-rise building based on a pre-trained knowledge base. Based on the acquired video content features, the server uses image processing and computer vision techniques to extract the architectural structural features of key objects from each video frame. These features can include detailed information such as the building's edge contours, window and door layouts, and floor divisions. Continuing with the example of the high-rise building mentioned above, the server uses image processing algorithms to accurately extract the architectural structural features of the high-rise building, such as its edge contours, window layouts, and floor divisions, from each video frame. Through these steps, the server can accurately extract the architectural structural features corresponding to each video frame from the target building's video data. These features provide crucial foundational data for subsequent fire detection and building structure analysis. The server's processing power and accuracy are of great significance for the timely detection of fire hazards and the protection of people's lives and property. Simultaneously, this method also provides a useful reference for video surveillance and feature extraction in other similar scenarios.

[0081] In this embodiment of the invention, the target building is the main building affected by the fire; the aforementioned step of obtaining key objects from the structural video data of the target building can be implemented through the following example.

[0082] The location of each fire source between floors in the main building of the fire was detected from the video data of the target building structure;

[0083] Based on the location information of each fire source feature location and the distance measurement between each fire source feature location, key center points and object boundary parameters are obtained from the target building structure video data;

[0084] The key object is obtained from the target building structure video data based on the key center point and the object boundary parameters.

[0085] In this embodiment of the invention, for example, when a fire occurs, a drone captures video data containing the main building on fire. This data is sent to a server for processing to accurately identify the location of the fire source within the main building and further obtain the key object, namely the main building itself. The server first processes the video data of the target building structure, using image recognition and fire detection algorithms to detect the characteristic locations of each fire source between floors in the main building. These characteristic locations can manifest as obvious fire signs such as flames and smoke. For example, in a fire video captured by a drone, the server uses an algorithm to identify that flames are emerging from windows between the third and fourth floors of the building, and the locations of these windows are marked as fire source characteristic locations. Based on the location information of each detected fire source characteristic location and the distance measurement between these locations, the server further analyzes the video data to obtain key center points and object boundary parameters. The key center point can be the geometric center or centroid of the fire source characteristic location, while the object boundary parameters define the boundary range of the main building in the video frame. In the aforementioned fire video, the server calculates the geometric center of the windows emitting flames as the key center point and determines the boundary parameters of the main building based on the location and size of these windows. Finally, the server extracts the key object, namely the main building of the fire, from the target building structure video data based on the acquired key center points and object boundary parameters. This step can be achieved through image segmentation or cropping, separating the main building from the video frames for subsequent analysis and processing. Continuing with the above fire video example, the server uses key center points and boundary parameters to crop the video frames, accurately extracting the area containing the main building as the key object. Through these steps, the server can accurately obtain the key object—the main building of the fire—from the target building structure video data. This is of great significance for subsequent fire spread trend analysis, rescue strategy formulation, and post-disaster assessment. At the same time, this method also provides a useful reference for video surveillance and object extraction in similar scenarios.

[0086] In this embodiment of the invention, the aforementioned step of comparing and analyzing the local fire building image features with the preset simulated fire building image features, and obtaining the fire analysis results for the area to be monitored based on the comparison and analysis results, can be implemented through the following example.

[0087] Obtain the pre-set simulated fire building image features corresponding to each fire monitoring area;

[0088] The matching degree between the local fire building image features and each of the preset simulated fire building image features is obtained respectively;

[0089] Fire analysis results for the monitored area are obtained based on the matching degree.

[0090] In this embodiment of the invention, for example, the server has acquired local fire building image features of the area to be monitored. These features are obtained by analyzing video data captured by a drone. To further analyze these features, the server needs to compare them with preset simulated fire building image features. The server first accesses its stored database or knowledge base to retrieve preset simulated fire building image features corresponding to each fire monitoring area. These preset features are obtained by simulating and analyzing different types of buildings and fire scenarios, representing various possible fire situations. For example, the server stores simulated fire image features of different types of buildings, such as high-rise residential buildings, commercial buildings, and industrial plants. These features may include the color, shape, and dynamic changes of flames, the diffusion speed and density of smoke, etc. The server then compares the acquired local fire building image features with each preset simulated fire building image feature. The comparison process involves algorithms such as image recognition, feature extraction, and similarity calculation. The server calculates a matching degree, representing the similarity between the local features and each preset feature. Taking a fire in a high-rise residential building as an example, the server compares the local fire building image features extracted from the drone video with the preset simulated fire image features of the high-rise residential building. If the two images are highly similar in terms of flame color, shape, and dynamic changes, then the matching degree is considered high. Based on the calculated matching degrees, the server further analyzes and derives a fire situation analysis result for the monitored area. This result can be a probability value, a rating, or specific descriptive information, indicating the current fire risk or situation in the area. Continuing with the example of a high-rise residential building fire, if the server finds that local features highly match the features of a pre-set simulated fire image of a high-rise residential building, it can derive a high-risk fire situation analysis result and trigger the corresponding alarm or notification mechanism. This result can promptly remind relevant personnel to take necessary countermeasures to reduce the losses caused by the fire.

[0091] In this embodiment of the invention, the aforementioned step of determining the target fire handling strategy from the strategy storage warehouse according to the fire strategy search request can be implemented through the following example.

[0092] Receive a fire strategy search request for obtaining a target fire handling strategy; the fire strategy search request includes at least search parameters, a starting search point, and a required number of searches;

[0093] The strategy storage bin is polled to obtain the target strategy range marker determined by the starting search point; wherein, each first strategy range marker is bound to a counter value, the counter value is used to indicate the number of fire handling strategies that meet the search parameters among the fire handling strategies corresponding to each marker that has a correlation with the first strategy range markers; the target strategy range marker is used to indicate the last first strategy range marker polled when the sum of the counter values ​​of the polled first strategy range markers exceeds the starting search point;

[0094] Among the markers that are related to the target strategy range, the target markers that satisfy the starting search point are obtained, and starting from the subsequent markers of the target markers, the target fire handling strategies that satisfy the required number of searches are obtained.

[0095] In an embodiment of the invention, exemplarily, in an intelligent fire monitoring system, a server is responsible for storing and managing various fire handling strategies. These strategies are pre-defined based on different fire scenarios, building characteristics, and resource conditions, and are used to guide emergency response and resource allocation during a fire. When the system receives a fire alarm signal, it generates a fire strategy search request based on the current fire situation to find the most suitable handling strategy from the strategy storage repository. When the server receives a fire strategy search request, it first parses the search parameters, starting search point, and required search count in the request. These parameters are dynamically generated based on the current fire situation and system requirements. For example, the server receives a search request where the search parameters are "high-rise residential building, nighttime, moderate fire intensity," the starting search point is 10, and the required search count is 3. This means that the server needs to find a handling strategy suitable for a high-rise residential building experiencing a moderate fire at night from the strategy storage repository, and starts searching from the 10th strategy, returning a maximum of 3 matching strategies. The server then begins polling the various first strategy range markers in the strategy storage repository. Each first policy range marker is bound to a counter value, which represents the number of fire handling strategies that meet the search parameters among the fire handling strategies corresponding to each marker that is related to that marker. The server determines the target policy range marker by comparing the sum of the counter values ​​with the starting search point. Suppose that during the polling process, the server finds that the sum of the counter values ​​of the first three first policy range markers is 9, and the sum of the counter values ​​after adding the fourth marker is 12. Since 10 (the starting search point) is between 9 and 12, the server determines the fourth marker as the target policy range marker. After determining the target policy range marker, the server further searches for target markers that meet the starting search point among the markers that are related to that marker. Then, it retrieves the target fire handling strategies that meet the required search number, starting from the subsequent markers of the target marker. Continuing the example above, suppose that there are two policy markers that meet the search parameters among the related markers corresponding to the fourth marker (target policy range marker). The server will select one of these two policy markers as the starting point (which may be randomly selected or sorted according to some priority, depending on the specific implementation), and then return up to three target fire handling strategies, including the starting point. These strategies will be used to guide the current fire emergency response.

[0096] In this embodiment of the invention, the first strategy range marker is obtained by the following steps.

[0097] Based on the summary information of the pre-defined summaries included in each fire handling strategy stored in the strategy storage repository, tags are constructed for the corresponding fire handling strategies.

[0098] According to the preset tag group allocation strategy, each obtained tag is allocated into multiple tag groups, and a corresponding first strategy range tag is constructed for each tag group, resulting in multiple first strategy range tags;

[0099] Each of the first strategy range tags is bound to a counter value pair. The counter value pair is identified by the search parameters and the corresponding tag group, and the counter value is the number of fire handling strategies that conform to the search parameters included in the corresponding tag group.

[0100] In this embodiment of the invention, for example, a large number of fire handling strategies are stored in the policy storage repository of the server. Each fire handling strategy contains a pre-defined summary, which includes the key attributes and characteristics of the strategy. To efficiently manage and retrieve these strategies, the server tags and groups them. The server first traverses each fire handling strategy in the policy storage repository. For each strategy, the server constructs a tag based on the summary information in its pre-defined summary. This tag is unique and can represent the characteristics and attributes of the strategy. For example, when the server traverses to a fire handling strategy applicable to "high-rise residential building, daytime, small fire," it constructs a tag based on the pre-defined summary of this strategy (which may include information such as building type, time, and fire intensity), such as "high-rise residential building_daytime_small fire." The server then assigns the constructed tags to multiple tag groups according to the pre-defined tag group allocation strategy. Each tag group contains a set of tags with similar characteristics or attributes. The purpose of this is to further organize and manage the tags and improve search efficiency. Continuing the example above, the server might assign the tag "High-rise residential building_Daytime_Small fire" to a tag group specifically for "High-rise residential buildings". This tag group might also contain other tags related to high-rise residential buildings, such as "High-rise residential building_Nighttime_Medium fire", etc. For each tag group, the server constructs a corresponding first policy scope tag. This first policy scope tag represents all tags in the tag group and their corresponding fire handling policies. Simultaneously, each first policy scope tag is bound to a counter value pair. This counter value pair uses the search parameters and the corresponding tag group as counter identifiers, and the number of fire handling policies in the corresponding tag group that match the search parameters as the counter value. In the example above, for the tag group "High-rise residential building", the server constructs a first policy scope tag and binds it to a counter value pair. Assuming the search parameters are "High-rise residential building, any time, any fire", then the counter value is the number of all fire handling policies in the tag group that match these search parameters. This number is dynamically updated as new policies are added or old policies are deleted. Through these steps, the server can efficiently manage and retrieve fire handling strategies, and quickly find the most suitable handling strategy based on the actual fire situation.

[0101] In this embodiment of the invention, the preset tag group allocation strategy includes:

[0102] The amount of data in each labeled group is consistent.

[0103] The data deviation between the number of fire handling strategies that meet the search parameters included in each tag group is less than the data deviation threshold.

[0104] In each of the fire handling strategies that conform to the search parameters included in each tag group, the word frequency of the preset search terms is consistent;

[0105] In each fire handling strategy that conforms to the search parameters and is included in each tag group, the range of the summary information of the preset summary is the average of the ranges of the summary information of the preset summary in all fire handling strategies that conform to the search parameters and are included in the strategy storage bin.

[0106] In this embodiment of the invention, for example, the server first ensures that the number of tags in each tag group is consistent. This is to balance the data load of each tag group, so that each tag group can provide a similar number of strategies to choose from during subsequent search and retrieval processes. For example, if the server has 1000 tags to allocate, and there are 10 preset tag groups, then each tag group should contain 100 tags. The server will distribute the tags evenly among the tag groups according to this principle. When allocating tags, the server will also consider the data deviation of the number of fire handling strategies that match the search parameters in each tag group. This deviation value is used to measure the uniformity of the distribution of the number of strategies among the tag groups. The server will ensure that this deviation value is less than a preset data deviation threshold to ensure consistency in the number of strategies among the tag groups. Assuming the search parameters are "high-rise residential building, nighttime, medium fire intensity", the server will calculate the number of strategies that match these search parameters in each tag group and compare the differences between these numbers. If the number of strategies in a certain tag group is significantly more or less than that in other tag groups, the server will adjust the allocation of tags to reduce the data deviation. In addition to considering the number of strategies, the server also pays attention to the term frequency of pre-defined search terms in fire handling strategies that match the search parameters within each tag group. Term frequency refers to the frequency with which a search term appears in the strategy summary. The server ensures that the term frequency is consistent across all tag groups to guarantee even matching of strategies across all tag groups during searches. For example, using the search term "fire extinguisher," the server counts the number of strategies containing "fire extinguisher" in each tag group and ensures that these numbers are similar across different tag groups. Finally, the server also considers the range of the summary information of fire handling strategies that match the search parameters within each tag group. The range is the difference between the maximum and minimum values ​​of a certain attribute in the strategy summary. The server calculates the range for each tag group and ensures that the average of these ranges is equal to or close to the average range of fire handling strategies that match the search parameters across the entire strategy repository. For example, assuming the summary information includes the attribute "evacuation time," the server calculates the difference between the maximum and minimum values ​​of "evacuation time" in each tag group (i.e., the range) and then calculates the average of these ranges. The server adjusts the tag allocation so that the average value is equal to or close to the average range of all strategies in the entire strategy repository that match the search parameters. Through this pre-defined tag group allocation strategy, the server ensures that the data volume, data bias, term frequency, and range of summary information in each tag group meet certain requirements, thereby improving the efficiency and accuracy of strategy retrieval.

[0107] In this embodiment of the invention, the step of obtaining the target marker that satisfies the starting search point from among the markers that have a correlation with the target strategy range markers can be implemented through the following example.

[0108] Starting from the first tag that is related to the target strategy range tag, sequentially poll each tag that is related to the target strategy range tag until the sum of the counter values ​​of the polled first strategy range tags other than the target strategy range tag, and the sum of the number of polled tags that meet the search parameters and are related to the target strategy range tag, equals the starting search point, thus obtaining the target tag; wherein, the target tag is the last tag among the polled tags.

[0109] In this embodiment of the invention, for example, the target policy range markers have already been determined in the server's policy storage repository according to the previous steps. Now, the server needs to find the target marker that satisfies the starting search point among the markers that are related to this target policy range marker. The starting search point is a specified value used to determine which marker to start retrieving the target fire handling policy from. The server starts from the first marker that is related to the target policy range marker and polls these markers sequentially. During the polling process, the server increments the counter value of the polled first policy range marker (these counter values ​​represent the number of fire handling policies that meet the search parameters among the fire handling policies corresponding to each marker that is related to this marker), and simultaneously calculates the number of polled markers that meet the search parameters. Assuming the starting search point is 10, the server starts polling the markers related to the target policy range marker. First, it polls the first marker, whose counter value is 3, indicating that there are 3 fire handling policies that meet the search parameters related to this marker. Then the server continues to poll the next marker, whose counter value is 2, and the counter value of the next marker is 5. At this point, the sum of the counter values ​​of the polled first policy range markers is 3 + 2 + 5 = 10, which is exactly equal to the starting search point. However, the server does not immediately stop polling because it also needs to ensure that the number of polled markers matching the search parameters also meets the requirement. In this example, suppose the first and second markers each have 1 marker matching the search parameters, and the third marker has 3 markers matching the search parameters. Then, the sum of the number of polled markers matching the search parameters is 1 + 1 + 3 = 5, which is not equal to the starting search point. Therefore, the server will continue polling the next marker until it finds a marker such that the sum of the counter values ​​of the polled first policy range markers (excluding the target policy range marker) plus the sum of the number of polled markers matching the search parameters that are related to the target policy range marker equals the starting search point. When the server finds such a marker, it will designate it as the target marker. In this example, assuming the fourth marker's counter value is 2, and there are 2 markers matching the search parameters associated with it, then the sum of the number of markers matching the search parameters that have been polled is 5 + 2 = 7, which is still not equal to the starting search point 10. The server will continue polling subsequent markers until the conditions are met. During the polling process, the server only focuses on whether the sum of the counter values ​​of the first policy range markers polled has reached the starting search point, and stops polling when it does, determining the last polled marker as the target marker.

[0110] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned deep learning-based fire-fighting drone monitoring method. Figure 2As shown, Figure 2 This is a structural block diagram of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0111] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A deep learning-based method for monitoring firefighting drones, characterized in that, include: In response to a fire alarm command for the area to be monitored, a monitoring request is sent to the fire-fighting drone; Receive video data of the target building structure in the area to be monitored from the firefighting drone based on the monitoring request; Based on the target building structure video data and the features of the pre-set simulated fire building images, the fire situation analysis results of the area to be monitored are obtained. A fire strategy search request is generated based on the fire analysis results. The target fire handling strategy is determined from the strategy storage repository based on the fire strategy search request. The step of determining the target fire handling strategy from the strategy storage repository based on the fire strategy search request includes: Receive a fire strategy search request for obtaining a target fire handling strategy; the fire strategy search request includes at least search parameters, a starting search point, and a required number of searches; The strategy storage bin is polled to obtain the target strategy range marker determined by the starting search point; wherein, each first strategy range marker is bound to a counter value, the counter value is used to indicate the number of fire handling strategies that meet the search parameters among the fire handling strategies corresponding to each marker that has a correlation with the first strategy range markers; the target strategy range marker is used to indicate the last first strategy range marker polled when the sum of the counter values ​​of the polled first strategy range markers exceeds the starting search point; Starting from the first tag that is related to the target strategy range tag, sequentially poll each tag that is related to the target strategy range tag until the sum of the counter values ​​of the polled first strategy range tags (excluding the target strategy range tag) and the sum of the number of polled tags that meet the search parameters and are related to the target strategy range tag equal to the starting search point, thus obtaining the target tag; wherein, the target tag is the last tag among the polled tags, and starting from the subsequent tags of the target tag, obtain the target fire handling strategy that meets the required search number; The first policy scope marker is obtained by the following steps, including: Based on the summary information of the pre-defined summaries included in each fire handling strategy stored in the strategy storage repository, tags are constructed for the corresponding fire handling strategies. According to the preset tag group allocation strategy, each obtained tag is allocated into multiple tag groups, and a corresponding first strategy range tag is constructed for each tag group, resulting in multiple first strategy range tags; Each of the first strategy range tags is bound to a counter value pair. The counter value pair is identified by the search parameters and the corresponding tag group, and the counter value is the number of fire handling strategies that conform to the search parameters included in the corresponding tag group. The preset tag group allocation strategy includes: The amount of data in each labeled group is consistent. The data deviation between the number of fire handling strategies that meet the search parameters included in each tag group is less than the data deviation threshold. In each of the fire handling strategies that conform to the search parameters included in each tag group, the word frequency of the preset search terms is consistent; In each fire handling strategy that conforms to the search parameters and is included in each tag group, the range of the summary information of the preset summary is the average of the ranges of the summary information of the preset summary in all fire handling strategies that conform to the search parameters and are included in the strategy storage bin.

2. The method according to claim 1, characterized in that, The analysis of the fire situation in the monitored area, based on the target building structure video data and pre-set simulated fire building image features, includes: Acquire target building structure video data obtained from monitoring the target building in the area to be monitored; Based on video content feature attributes that match the feature attribute category of the target building, the building structure features corresponding to each video frame are extracted from the target building structure video data; the video content feature attributes are attributes that meet the requirements after integrating the drone shooting angles corresponding to the target video frames; the target video frame is the video frame corresponding to the feature extraction operation. Based on the building structure features corresponding to each video frame, local fire building image features of the monitored area are obtained. The local fire building image features are compared and analyzed with the preset simulated fire building image features, and the fire analysis results for the monitored area are obtained based on the comparison and analysis results; the preset simulated fire building image features are local fire building image features obtained by performing pre-fire simulation analysis on the target building structure video data corresponding to the target building in the fire monitoring area.

3. The method according to claim 2, characterized in that, The target building is the main building of the fire, and the video content feature attribute is the flame and smoke attribute; The step of extracting the building structure features corresponding to each video frame from the target building structure video data based on video content feature attributes that match the feature attribute category of the target building includes: Based on the flame and smoke attributes from multiple drone shooting angles, the drone shooting angle building structure features corresponding to the flame and smoke attributes of each video frame and the multiple drone shooting angles are extracted from the target building structure video data. For each video frame in each video frame, the drone shooting angle building structure features corresponding to the flame and smoke attributes of the multiple drone shooting angles are merged to obtain the drone shooting angle merged features of each video frame. The corresponding building structure features are obtained by merging the drone shooting angle features corresponding to each video frame.

4. The method according to claim 2, characterized in that, The step of extracting the building structure features corresponding to each video frame from the target building structure video data based on video content feature attributes that match the feature attribute category of the target building includes: Obtain video content feature attributes that match the feature attribute category of the target building in the video content feature dimension; Based on the video content feature attributes under multiple video content feature dimensions, extract the building structure features to be processed corresponding to each video frame and each of the multiple video content feature dimensions from the target building structure video data; For each video frame in each video frame, each video frame is merged with the building structure features to be processed corresponding to the multiple video content feature dimensions to obtain the multidimensional merged features of each video frame. The corresponding building structure features are obtained based on the multidimensional merging features corresponding to each video frame.

5. The method according to claim 2, characterized in that, The step of extracting the building structure features corresponding to each video frame from the target building structure video data based on video content feature attributes that match the feature attribute category of the target building includes: The convolutional units in the pre-trained fire building recognition model are used to extract the building structure features corresponding to each video frame from the target building structure video data. The convolutional unit is used to perform feature extraction operations based on video content feature attributes that match the feature attribute category of the target building. The process of obtaining local fire building image features of the monitored area based on the building structure features corresponding to each video frame includes: The local fire building feature extraction unit in the fire building identification model extracts the local fire building image features of the monitored area based on the building structure features corresponding to each video frame. The fire-related building identification model is obtained through the following methods: Acquire video data of multiple sample target building structures; The convolutional units in the initial fire building identification model are used to extract the sample building structure features corresponding to each sample video frame from the sample target building structure video data. The local fire building feature extraction unit in the initial fire building identification model extracts the sample fire building features for the sample building structure features corresponding to each sample video frame. The feature error of the fire building image is obtained based on the sample fire building features; Obtain counterexample pairs, which include sample target building structure video data labeled with multiple fire types; Based on the sample building structure features corresponding to the sample target building structure video data of the counterexample, the sample instance pair error is obtained; The training error is obtained based on the feature error of the fire building image and the error of the sample instance; Based on the training error, the convolutional units and local fire building feature extraction units in the initial fire building recognition model are optimized, and the training process is re-executed until the fire building recognition model that meets the preset conditions is obtained.

6. The method according to claim 2, characterized in that, The process of obtaining local fire building image features of the monitored area based on the building structure features corresponding to each video frame includes: Based on the drone shooting angle corresponding to each video frame in the target building structure video data, the building structure features corresponding to each video frame are combined to obtain the building structure features of the target building structure video data. A feature extraction operation is performed on the building structure features of the target building structure video data to obtain the local fire building image features of the area to be monitored.

7. The method according to claim 2, characterized in that, The target building is the main building affected by the fire. The step of extracting the building structure features corresponding to each video frame from the target building structure video data based on video content feature attributes that match the feature attribute category of the target building includes: The location of each fire source between floors in the main building of the fire was detected from the video data of the target building structure; Based on the location information of each fire source feature location and the distance measurement between each fire source feature location, key center points and object boundary parameters are obtained from the target building structure video data; Based on the key center point and the object boundary parameters, the key object is obtained from the target building structure video data; Obtain video content feature attributes that match the feature attribute category of the target building; Based on the video content feature attributes, extract the building structure features corresponding to each video frame in the key object.

8. The method according to claim 2, characterized in that, The step of comparing and analyzing the features of the local fire building image with the features of the preset simulated fire building image, and obtaining the fire analysis results for the monitored area based on the comparison and analysis results, includes: Obtain the pre-set simulated fire building image features corresponding to each fire monitoring area; The matching degree between the local fire building image features and each of the preset simulated fire building image features is obtained respectively; Fire analysis results for the monitored area are obtained based on the matching degree.

9. A server system, characterized in that, Includes a server, the server being used to perform the method according to any one of claims 1-8.

Images