Air-ground integrated cooperative command forest fire extinguishing system based on 5G network

Abstract

The invention discloses an air-ground integrated cooperative command forest fire extinguishing system based on a 5G network. The air-ground integrated cooperative command forest fire extinguishing system comprises an air-ground integrated cooperative command decision display unit, a fire situation intelligent analysis unit, a comprehensive scheduling unit, a data communication unit and a 5G communication link unit. The data communication unit comprises a data transmission downlink unit and a data transmission uplink unit; the current real-time information of the forest fire is detected and acquired in real time through the load scheduling unit; a fire prediction situation map is obtained through real-time information of the fire situation intelligent analysis unit, and fire extinguishing facility information of current fire extinguishing operation is transmitted to the air-ground integrated cooperative command decision display unit through the comprehensive scheduling unit; an air-ground integrated cooperative command decision display unit formulates an air-ground integrated cooperative fire extinguishing task according to fire extinguishing facility information on the basis of a fire prediction situation map, and a data transmission downlink unit issues a control task instruction, so that the problem of a traditional forest fire extinguishing method is solved; the problems that fire extinguishing efficiency is high and casualties and property loss are increased due to the fact that the requirements for real-time monitoring of live broadcast pictures of a fire scene, real-time fire situation analysis and cooperative command of mechanisms at all levels, various devices, personnel and materials are difficult to meet in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of forest fire fighting technology, and in particular to an air-ground integrated coordinated command forest fire fighting system based on a 5G network. Background Art

[0002] In forest fire fighting scenarios, there are problems such as remote geographical location, large interference from traditional radio signals, rugged roads, vast forest areas, and weak single fire fighting forces. Relying on a single fire fighting method and approach is increasingly unable to meet the actual forest fire fighting needs.

[0003] However, there is currently no mature air-to-ground coordinated forest fire extinguishing system that integrates multiple fire-fighting agencies, multi-purpose drones, fire-fighting helicopters, fire trucks, firefighters and emergency supplies. This makes it impossible to meet the requirements of real-time monitoring of live fire scene images, real-time fire situation analysis, and coordinated command of agencies at all levels, various equipment, personnel and materials, which in turn affects fire-fighting efficiency and increases casualties and property losses. Summary of the invention

[0004] The present invention provides an air-ground integrated coordinated command forest fire extinguishing system based on a 5G network to overcome the above-mentioned technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] An air-ground integrated coordinated command forest fire extinguishing system based on a 5G network, characterized in that it includes: an air-ground integrated coordinated command decision display unit, a fire situation intelligent analysis unit, a comprehensive dispatching unit, a data communication unit, a 5G communication link unit, a video streaming service unit, and a load dispatching unit;

[0007] The load dispatching unit stores fire-fighting and rescue dispatching data and is used to monitor and obtain current real-time information of forest fires in real time;

[0008] The current real-time information includes fire thermal infrared images and fire situation video stream data;

[0009] The 5G communication link unit is used to transmit real-time information to the data communication unit;

[0010] The data communication unit includes an uplink data transmission unit and a downlink data transmission unit;

[0011] The data transmission uplink unit is used to transmit the current real-time information of the forest fire to the fire situation intelligent analysis unit and the comprehensive dispatching unit;

[0012] The fire situation intelligent analysis unit is used to predict the fire situation according to the current real-time information of the forest fire, obtain the fire prediction situation map, and transmit the fire prediction situation map to the air-ground integrated collaborative command decision display unit;

[0013] The comprehensive dispatching unit is used to transmit the fire-fighting and rescue dispatching data of the current fire-fighting operation to the air-ground integrated coordinated command decision-making display unit;

[0014] The firefighting and rescue dispatch data includes firefighting facilities, rescue materials and types / quantities of personnel involved in firefighting; firefighting facilities include the number / types of ground firefighting and rescue facilities and aerial firefighting and rescue facilities involved in firefighting / reserve firefighting;

[0015] The aerial firefighting and rescue facilities include at least firefighting helicopters, firefighting bombs, shouting and lighting drones, firefighting drones and detection drones;

[0016] The air-ground integrated collaborative command decision-making display unit includes a fire extinguishing area division module, an air rescue module and a ground rescue module;

[0017] The fire extinguishing area division module is used to artificially divide the fire situation area according to the fire situation of the fire prediction situation map according to the empirical value;

[0018] The fire situation area includes a strong fire area, a medium fire area and a low fire area; and the division of the fire area is set by the fire thermal infrared image features based on the fire prediction situation map;

[0019] The aerial rescue module is used to determine the number / type of aerial firefighting and rescue facilities currently involved in firefighting / reserve firefighting; the ground rescue module is used to determine the number / type of ground firefighting and rescue facilities currently involved in firefighting / reserve firefighting;

[0020] Based on the fire situation area, combined with the data obtained by the air rescue module and the ground rescue module, the number and type of ground fire fighting and rescue facilities and air fire fighting and rescue facilities are allocated according to experience and the fire fighting performance of the corresponding facilities, so as to obtain the control task instructions of air-ground integrated coordinated fire fighting and send them to the comprehensive dispatching unit; and the control task instructions include air fire fighting control instructions and ground control instructions;

[0021] The integrated scheduling unit is also used to send control task instructions to the 5G communication link unit through the data transmission downlink unit;

[0022] The 5G communication link unit is also used to transmit the issued control task instructions to the load scheduling unit; the load scheduling unit includes a drone load unit and a mobile scheduling control terminal;

[0023] The UAV load unit is used to dispatch aerial firefighting and rescue facilities according to aerial firefighting control instructions, and the mobile dispatching control terminal is used to dispatch aerial and ground firefighting and rescue facilities according to ground firefighting control instructions, thereby realizing the forest fire fighting task of air-ground integrated coordinated command.

[0024] Furthermore, the fire situation intelligent analysis unit performs fire situation prediction to obtain a fire prediction situation map based on the current real-time information of the forest fire, which specifically includes the following steps:

[0025] S01: Real-time monitoring of fire thermal infrared images at the current forest fire scene based on image recognition technology;

[0026] S02: confirming whether the pixel value of the fire thermal infrared image detected exceeds a preset threshold;

[0027] If it is confirmed that the pixel value of the fire thermal infrared image detected exceeds the preset threshold, then continue to execute step S03, otherwise repeat step S01;

[0028] S03: Set the sampling frame period and continuously collect multiple frames of data sampling samples;

[0029] The data sampling samples include at least fire thermal infrared images, high-definition video images and meteorological factor data;

[0030] S04: Extract multiple sample features from data sampling samples through deep learning algorithms;

[0031] And the sample features include image time features, image space features and image meteorological features;

[0032] And multiple sample feature machines are fused to obtain a fused feature map;

[0033] S05: inputting the fused feature map into a pre-trained fire situation prediction model;

[0034] The fire situation prediction model is constructed by using historical fire data based on convolutional neural network (CNN) training;

[0035] The prediction output of the fire situation prediction model is combined with the GIS map engine to obtain a fire prediction situation map.

[0036] Furthermore, the air-ground integrated coordinated command decision display unit is also used to display data on forest fires;

[0037] Including the display of live fire scene video, dynamic display of real-time emergency equipment, personnel and materials, and data display of fire situation / fire change trend;

[0038] The method for displaying the live fire scene video is as follows: based on the 5G communication link unit, the fire situation video stream data in the load unit is retrieved in real time through the video streaming service unit, and the real-time live broadcast picture of the drone is displayed based on the split-screen method;

[0039] The dynamic display method of real-time emergency equipment, personnel and materials is: based on the GIS map engine, multiple layers are used to display the dynamic distribution information of air rescue equipment, ground fire-fighting facilities and rescue and fire-fighting personnel;

[0040] The method for displaying the data of the fire situation / fire change trend is as follows: based on the fire situation intelligent analysis unit, according to the current real-time information of the forest fire, a fire prediction situation diagram for fire situation prediction is dynamically displayed.

[0041] Furthermore, the comprehensive scheduling unit includes a basic information management module, a control command scheduling module, a real-time data transmission module and a message notification module;

[0042] The basic information management module is used to manage the data of rescue emergency equipment, shouting and lighting drones, fire-fighting drones, detection drones, fire trucks, firefighters, fire-fighting and rescue materials, and water sources;

[0043] The control command scheduling module is used to receive the control task instructions of the air-ground integrated coordinated command decision display unit;

[0044] Based on the control task instructions, according to the rescue fire-fighting equipment, rescue fire-fighting personnel and rescue fire-fighting materials at all levels, the number of fire-fighting drones, fire-fighting helicopters, fire trucks and firefighters is planned in real time to configure the fire-fighting route, and the amount of materials to be carried and the planned travel time are confirmed, and sent to the fire-fighting drones, fire-fighting helicopters and mobile dispatch control terminal through the message notification module;

[0045] The real-time data transmission module is used to receive the transmission data between the UAV load unit and the mobile scheduling control end transmitted in real time through the data transmission uplink unit.

[0046] Furthermore, the load dispatching unit also includes a device control / data acquisition module for monitoring the aerial firefighting and rescue facilities and acquiring drone data and sampling data;

[0047] And obtain real-time information on forest fires based on drone data and sampling data;

[0048] The drone data at least includes real-time latitude and longitude position data of the aircraft, flight data of flight speed / roll angle / pitch angle / yaw angle, signal data of satellite signal / remote sensing signal / image transmission signal, battery data of battery temperature / voltage / power, and lens data of gimbal pitch angle / yaw angle / roll angle;

[0049] The sampled data at least includes fire extinguisher bomb inventory quantity data, video image data captured by wide-angle / zoom / infrared cameras, geometric position data collected by lidar, and temperature / humidity / air pressure / wind speed / wind direction data collected by meteorological detectors.

[0050] Furthermore, the aerial fire-fighting and rescue facilities include at least a shouting and lighting drone equipped with an all-in-one shouting and lighting machine and a 5G Celluar module, a fire-fighting drone equipped with a fire-extinguishing bomb throwing device and a 5G Celluar module, and a detection drone equipped with a wide-angle / zoom / infrared camera / lidar / meteorological detector load device and a 5G Celluar module.

[0051] Further, it also includes a data storage unit;

[0052] The data storage unit includes an Hbase column storage database module, a Mysql relational database module and an OSS file storage module;

[0053] The Hbase column storage database module is used to store unstructured data collected by the drone load unit, and the unstructured data includes weather temperature, humidity, air pressure, wind speed, wind direction and real-time longitude & latitude data;

[0054] The Mysql relational database module is used to store structured data obtained by the load scheduling unit; and the structured data includes fire-fighting drones, fire-fighting helicopters, fire trucks, firefighters, fire-fighting emergency supplies, water sources, and drone status data;

[0055] The OSS file storage module is used to store image data, video data and document data of air-ground integrated collaborative fire fighting tasks during the forest fire fighting process.

[0056] Furthermore, it also includes a video streaming service unit, and the video streaming service unit is used to retrieve the fire situation video stream data in the load scheduling unit in real time through the 5G communication link unit, and display the real-time live screen of the fire situation video stream data based on the split-screen method;

[0057] The air-ground integrated coordinated command decision display unit is also used to manually assist and monitor the execution of the air-ground integrated coordinated command forest fire fighting task based on the real-time live broadcast picture.

[0058] Beneficial effect: The present invention provides an air-ground integrated collaborative command forest fire extinguishing system based on 5G network, which solves the problems of short radio communication distance and large signal interference through the 5G communication link unit, and formulates air-ground integrated collaborative fire extinguishing tasks based on the fire prediction situation map and the fire extinguishing facility information through the air-ground integrated collaborative command decision-making display unit, and issues control task instructions through the data transmission downlink unit based on the air-ground integrated collaborative fire extinguishing task, thereby solving the problem of remote command difficulties; and through the fire situation intelligent analysis unit, the fire situation is predicted according to the current real-time information of the forest fire to obtain the fire prediction situation map, which helps to fully explore the potential fire situation, so as to timely and accurately grasp the fire situation through the air-ground integrated collaborative command decision-making display unit and the joint command center, so as to facilitate the implementation of air-ground integrated collaborative rescue command, thereby realizing efficient and rapid emergency command and control of sudden forest fire accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 This is a schematic diagram of the air-ground integrated collaborative command forest fire extinguishing system based on 5G network of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] This embodiment provides an air-ground integrated coordinated command forest fire extinguishing system based on a 5G network, such as Figure 1 As shown, it includes an air-ground integrated collaborative command decision display unit, a fire situation intelligent analysis unit, a comprehensive dispatch unit, a data communication unit, a 5G communication link unit, a data storage unit, a video streaming service unit and a load dispatch unit; and the load dispatch unit includes a UAV load unit and a mobile dispatch control terminal;

[0063] The data communication unit includes a data transmission downlink unit and a data transmission uplink unit;

[0064] The mobile dispatch control terminal is used to control and dispatch ground fire-fighting facilities and rescue fire-fighting personnel to perform fire-fighting tasks. Specifically, the mobile dispatch control terminal includes mobile phones, pads and mobile APP applications; and the APP application is developed using Android and iOS, including a login and exit module, a task message notification module, a real-time live broadcast screen of the fire scene, a fire situation based on a GIS map and multi-layer equipment on a GIS map, and personnel and emergency material distribution information. The development process or method of the APP application is an existing well-known technology and will not be repeated here.

[0065] The load dispatching unit is used to monitor and obtain the current real-time information of forest fires in real time;

[0066] The current real-time information includes fire thermal infrared images, fire situation video stream data, and fire fighting and rescue dispatch data;

[0067] The 5G communication link unit is used to transmit real-time information to the data communication unit;

[0068] The 5G communication link unit, namely the fifth generation mobile communication technology (5G for short), is a new generation of broadband mobile communication technology with the characteristics of high speed, low latency and large connection. The 5G communication facility is the network infrastructure for realizing the interconnection between man, machine and things, and can meet the efficient communication needs in forest fire fighting scenarios.

[0069] The air-ground integrated collaborative command decision-making display unit also includes a video streaming service unit, and the video streaming service unit is used to retrieve the fire situation video stream data in the load scheduling unit in real time through the 5G communication link unit, and display the real-time live screen of the fire situation video stream data based on the split-screen method; the air-ground integrated collaborative command decision-making display unit is also used to manually assist and monitor the execution of the air-ground integrated collaborative command forest fire extinguishing task according to the real-time live screen, and the video streaming service unit is used to retrieve the fire situation video stream data in the load unit in real time through the 5G communication link unit, and display the real-time live screen of the drone based on the split-screen method; specifically, the video streaming service unit can provide rtmp, rtsp, hls, webrtc and other protocol streams to the air-ground integrated collaborative command decision-making display unit, and support live broadcast, recording, playback, capture and other functions; provide GB28181 and GB35114 protocol access to the drone camera load;

[0070] The data communication unit includes an uplink data transmission unit and a downlink data transmission unit;

[0071] The data transmission uplink unit is used to transmit the current real-time information of the forest fire to the fire situation intelligent analysis unit and the comprehensive dispatching unit;

[0072] The fire situation intelligent analysis unit is used to predict the fire situation according to the current real-time information of the forest fire, obtain the fire prediction situation map, and transmit the fire prediction situation map to the air-ground integrated collaborative command decision display unit;

[0073] In a specific embodiment, the fire situation intelligent analysis unit performs fire situation prediction to obtain a fire prediction situation map based on the current real-time information of the forest fire, which specifically includes the following steps:

[0074] S01: Real-time monitoring of fire thermal infrared images at the current forest fire scene based on image recognition technology;

[0075] S02: confirming whether the pixel value of the fire thermal infrared image detected exceeds a preset threshold;

[0076] If it is confirmed that the pixel value of the fire thermal infrared image detected exceeds the preset threshold, then continue to execute step S03, otherwise repeat step S01;

[0077] S03: Set the sampling frame period and continuously collect multiple frames of data sampling samples;

[0078] The data sampling samples include at least fire thermal infrared images, high-definition video images and meteorological factor data;

[0079] S04: Extract multiple sample features from data sampling samples through deep learning algorithms;

[0080] And the sample features include image time features, image space features and image meteorological features;

[0081] And multiple sample feature machines are fused to obtain a fused feature map; the feature fusion method is an existing well-known technical means, such as a deep convolutional neural network and other methods, which will not be described in detail here;

[0082] S05: inputting the fused feature map into a pre-trained fire situation prediction model;

[0083] The fire situation prediction model is constructed by using historical fire data based on convolutional neural network (CNN) training;

[0084] The prediction output of the fire situation prediction model is combined with the GIS map engine to obtain a fire prediction situation map;

[0085] The comprehensive dispatching unit is used to transmit the fire-fighting and rescue dispatching data of the current fire-fighting operation to the air-ground integrated coordinated command decision-making display unit;

[0086] The firefighting and rescue dispatch data includes firefighting facilities, rescue materials and types / quantities of personnel involved in firefighting; firefighting facilities include the number / types of ground firefighting and rescue facilities and aerial firefighting and rescue facilities involved in firefighting / reserve firefighting;

[0087] The aerial firefighting and rescue facilities include at least firefighting helicopters, firefighting bombs, shouting and lighting drones, firefighting drones and detection drones;

[0088] The air-ground integrated collaborative command decision-making display unit includes a fire extinguishing area division module, an air rescue module and a ground rescue module;

[0089] The fire extinguishing area division module is used to artificially divide the fire situation area according to the fire situation of the fire prediction situation map according to the empirical value;

[0090] The fire situation area includes a strong fire area, a medium fire area and a low fire area; and the division of the fire area is set by the fire thermal infrared image features based on the fire prediction situation map;

[0091] The aerial rescue module is used to obtain the number / type of currently participating fire extinguishing / reserve fire extinguishing in the aerial fire extinguishing and rescue facilities, and the ground rescue module is used to obtain the number / type of currently participating fire extinguishing / reserve fire extinguishing in the ground fire extinguishing and rescue facilities;

[0092] Based on the fire situation area, combined with the data obtained by the air rescue module and the ground rescue module, the number and type of ground fire fighting and rescue facilities and air fire fighting and rescue facilities are allocated according to experience and the fire fighting performance of the corresponding facilities, so as to obtain the control task instructions of air-ground integrated coordinated fire fighting and send them to the comprehensive dispatching unit; and the control task instructions include air fire fighting control instructions and ground control instructions;

[0093] For example: for a strong fire area, based on the aerial firefighting and rescue facilities and the ground firefighting and rescue facilities, multiple aerial facilities and ground facilities are configured based on the type and known firefighting performance to perform synchronous firefighting operations. For a low-level fire area, any type of rescue facility from the aerial firefighting and rescue facilities and the ground firefighting and rescue facilities can be selected for firefighting operations based on experience, and firefighting facilities with a smaller number of facilities are configured for rescue, so as to retain other firefighting equipment for dispatching firefighting operations in other fire areas;

[0094] The integrated scheduling unit is also used to send control task instructions to the 5G communication link unit through the data transmission downlink unit;

[0095] The 5G communication link unit is also used to transmit the issued control task instructions to the load scheduling unit; the load scheduling unit includes a drone load unit and a mobile scheduling control terminal;

[0096] The UAV load unit is used to dispatch aerial firefighting and rescue facilities according to aerial firefighting control instructions, and the mobile dispatching control terminal is used to dispatch aerial and ground firefighting and rescue facilities according to ground firefighting control instructions, thereby realizing the forest fire fighting task of air-ground integrated coordinated command.

[0097] This embodiment solves the problems of short radio communication distance and large signal interference through the 5G communication link unit, and formulates air-ground integrated collaborative fire-fighting tasks based on the fire prediction situation map and the fire-fighting facility information through the air-ground integrated collaborative command decision-making display unit, and issues control task instructions through the data transmission downlink unit based on the air-ground integrated collaborative fire-fighting tasks, thereby solving the problem of difficult remote command; and through the fire situation intelligent analysis unit, a fire situation prediction is performed according to the current real-time information of the forest fire to obtain a fire prediction situation map, which helps to fully explore the potential fire situation, so that the air-ground integrated collaborative command decision-making display unit can timely and accurately grasp the fire situation in conjunction with the command center, so as to facilitate the implementation of air-ground integrated collaborative rescue command, thereby realizing efficient and rapid emergency command and control of sudden forest fire accidents.

[0098] In a specific embodiment, the air-ground integrated coordinated command decision display unit is also used to display data on forest fires;

[0099] Including the display of live fire scene video, dynamic display of real-time emergency equipment / personnel / materials, and data display of fire situation / fire change trend;

[0100] The method for displaying the live fire scene video is as follows: based on the 5G communication link unit, the fire situation video stream data in the load unit is retrieved in real time through the video streaming service unit, and the real-time live broadcast picture of the drone is displayed based on the split-screen method;

[0101] The method for dynamic display of real-time emergency equipment / personnel / materials is: based on a GIS map engine, multiple layers are used to display dynamic distribution information of air rescue equipment, ground fire-fighting facilities, and rescue and fire-fighting personnel;

[0102] The method for displaying the data of the fire situation / fire change trend is as follows: based on the fire situation intelligent analysis unit, according to the current real-time information of the forest fire, a fire prediction situation diagram for fire situation prediction is dynamically displayed.

[0103] In a specific embodiment, the comprehensive scheduling unit includes a basic information management module, a control command scheduling module, a real-time data transmission module and a message notification module;

[0104] The basic information management module is used to manage the data of rescue emergency equipment, shouting and lighting drones, fire-fighting drones, detection drones, fire trucks, firefighters, fire-fighting and rescue materials, and water sources;

[0105] The control command scheduling module is used to receive the control task instructions of the air-ground integrated coordinated command decision display unit;

[0106] Based on the control task instructions, according to the rescue fire-fighting equipment, rescue fire-fighting personnel and rescue fire-fighting materials at all levels, the number of fire-fighting drones, fire-fighting helicopters, fire trucks and firefighters is planned in real time to configure the fire-fighting route, and the amount of materials to be carried and the planned travel time are confirmed, and sent to the fire-fighting drones, fire-fighting helicopters and mobile dispatch control terminal through the message notification module;

[0107] The real-time data transmission module is used to receive the transmission data between the UAV load unit and the mobile scheduling control end transmitted in real time through the data transmission uplink unit.

[0108] In a specific embodiment, the data transmission uplink unit transmits the current real-time information of the forest fire to the fire situation intelligent analysis unit and the comprehensive dispatching unit based on the MQTT protocol or HTTP protocol of the 5G network, and the current real-time information of the forest fire also includes drone data and fire extinguishing load data;

[0109] The drone data includes real-time latitude and longitude position data of the aircraft, flight data of flight speed / roll angle / pitch angle / yaw angle, signal data of satellite signal / remote sensing signal / image transmission signal, battery data of battery temperature / voltage / power, and lens data of gimbal pitch angle / yaw angle / roll angle;

[0110] The fire extinguishing load data includes fire extinguishing bomb inventory data, video image data captured by wide-angle / zoom / infrared cameras, geometric position data collected by laser radar, and temperature / humidity / air pressure / wind speed / wind direction data collected by meteorological detectors;

[0111] The data transmission downlink unit sends the control task command and its information dispatched by the integrated dispatching system to the UAV load unit and the mobile dispatching control terminal based on the 5G network.

[0112] In a specific embodiment, the aerial firefighting and rescue facility includes at least a shouting and lighting drone equipped with a shouting and lighting integrated machine and a 5GCelluar module;

[0113] The all-in-one speaker and lighting device is a device that integrates a high-pitched broadcaster and lighting. The all-in-one speaker and lighting device is equipped with control software, which is an application software based on DJI PSDK to realize the control of drones and all-in-one speaker and lighting device, camera video transmission and device status data reporting;

[0114] Firefighting drones equipped with fire-extinguishing bomb throwing devices and 5G Celluar modules;

[0115] Among them, the fire bomb throwing device is mainly used to control the throwing equipment to throw fire bombs. The fire-fighting drone is equipped with control software, which is an application software based on Pegasus SDK to realize the throwing equipment control, camera video transmission and equipment status data reporting;

[0116] A detection drone equipped with a wide-angle & zoom & infrared camera & lidar & weather detector payload and a 5G Celluar module;

[0117] Among them, the detection drone includes DJI M350RTK drone, wide-angle & zoom & thermal infrared camera & laser radar & weather detector and control software;

[0118] Wide-angle & zoom & thermal infrared camera & laser radar & weather detector: mainly including wide-angle & zoom & infrared camera, laser radar, weather detector;

[0119] Wide-angle & zoom camera: capable of capturing high-definition live footage;

[0120] Thermal infrared camera: capable of taking infrared photos of the fire scene;

[0121] LiDAR: can accurately collect geometric position data;

[0122] Meteorological detector: capable of collecting data on five elements: temperature, humidity, air pressure, wind direction and wind speed;

[0123] The detection drone is equipped with control software, which is an application software based on DJI PSDK to realize the control of wide-angle, zoom and infrared cameras, image transmission, and meteorological element collection and control of meteorological detectors;

[0124] In a specific embodiment, it also includes a data storage unit;

[0125] The data storage unit includes an Hbase column storage database module, a Mysql relational database module and an OSS file storage module;

[0126] The Hbase column storage database module is used to store unstructured data collected by the drone load unit, and the unstructured data includes weather temperature, humidity, air pressure, wind speed, wind direction and real-time longitude & latitude data;

[0127] The Mysql relational database module is used to store structured data obtained by the load scheduling unit; and the structured data includes fire-fighting drones, fire-fighting helicopters, fire trucks, firefighters, fire-fighting emergency supplies, water sources, and drone status data;

[0128] The OSS file storage module is used to store image data, video data and document data of air-ground integrated collaborative fire fighting tasks during the forest fire fighting process.

[0129] In this embodiment, by storing structured data, non-structured data, and image / video data of the forest fire extinguishing process, it is convenient to retain historical forest fire extinguishing data for subsequent analysis of decisions on air-ground integrated coordinated fire extinguishing tasks, so as to facilitate better fire extinguishing case study and discussion, and provide a data basis for efficient and rapid emergency command and control of sudden forest fire accidents.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An air-ground integrated coordinated command forest fire extinguishing system based on 5G network, characterized in that: include: Air-ground integrated coordinated command decision-making display unit, fire situation intelligent analysis unit, comprehensive dispatch unit, data communication unit, 5G communication link unit, video streaming service unit and load dispatch unit; The load dispatching unit is used to monitor and obtain the current real-time information of forest fires in real time; The current real-time information includes fire thermal infrared images and fire situation video stream data; The 5G communication link unit is used to transmit real-time information to the data communication unit; The data communication unit includes an uplink data transmission unit and a downlink data transmission unit; The data transmission uplink unit is used to transmit the current real-time information of the forest fire to the fire situation intelligent analysis unit and the comprehensive dispatching unit; The fire situation intelligent analysis unit is used to predict the fire situation according to the current real-time information of the forest fire, obtain the fire prediction situation map, and transmit the fire prediction situation map to the air-ground integrated collaborative command decision display unit; The comprehensive dispatching unit is used to transmit the fire-fighting and rescue dispatching data of the current fire-fighting operation to the air-ground integrated coordinated command decision-making display unit; The firefighting and rescue dispatch data includes firefighting facilities, rescue materials and types / quantities of personnel involved in firefighting; firefighting facilities include the number / types of ground firefighting and rescue facilities and aerial firefighting and rescue facilities involved in firefighting / reserve firefighting; The aerial firefighting and rescue facilities include at least firefighting helicopters, firefighting bombs, shouting and lighting drones, firefighting drones and detection drones; The air-ground integrated collaborative command decision-making display unit includes a fire extinguishing area division module, an air rescue module and a ground rescue module; The fire extinguishing area division module is used to artificially divide the fire situation area according to the fire situation of the fire prediction situation map according to the empirical value; The fire situation area includes a strong fire area, a medium fire area and a low fire area; and the division of the fire area is set by the fire thermal infrared image features based on the fire prediction situation map; The aerial rescue module is used to determine the number / type of currently participating firefighting / reserve firefighting in the aerial firefighting and rescue facilities; the ground rescue module is used to determine the number / type of currently participating firefighting / reserve firefighting in the ground firefighting and rescue facilities; Based on the fire situation area, combined with the data obtained by the air rescue module and the ground rescue module, the number and type of ground fire fighting and rescue facilities and air fire fighting and rescue facilities are allocated according to experience and the fire fighting performance of the corresponding facilities, so as to obtain the control task instructions of air-ground integrated coordinated fire fighting and send them to the comprehensive dispatching unit; and the control task instructions include air fire fighting control instructions and ground control instructions; The integrated scheduling unit is also used to send control task instructions to the 5G communication link unit through the data transmission downlink unit; The 5G communication link unit is also used to transmit the issued control task instructions to the load scheduling unit; the load scheduling unit includes a drone load unit and a mobile scheduling control terminal; The UAV load unit is used to dispatch aerial firefighting and rescue facilities according to aerial firefighting control instructions, and the mobile dispatching control terminal is used to dispatch aerial and ground firefighting and rescue facilities according to ground firefighting control instructions, thereby realizing the forest fire fighting task of air-ground integrated coordinated command.

2. According to the 5G network-based air-ground integrated coordinated command forest fire extinguishing system according to claim 1, it is characterized in that: The fire situation intelligent analysis unit performs fire situation prediction to obtain a fire prediction situation map based on the current real-time information of the forest fire, specifically comprising the following steps: S01: Real-time monitoring of fire thermal infrared images at the current forest fire scene based on image recognition technology; S02: confirming whether the pixel value of the fire thermal infrared image detected exceeds a preset threshold; If it is confirmed that the pixel value of the fire thermal infrared image detected exceeds the preset threshold, then continue to execute step S03, otherwise repeat step S01; S03: Set the sampling frame period and continuously collect multiple frames of data sampling samples; The data sampling samples include at least fire thermal infrared images, high-definition video images and meteorological factor data; S04: Extract multiple sample features from data sampling samples through deep learning algorithms; And the sample features include image time features, image space features and image meteorological features; And multiple sample feature machines are fused to obtain a fused feature map; S05: inputting the fused feature map into a pre-trained fire situation prediction model; The fire situation prediction model is constructed by using historical fire data based on convolutional neural network (CNN) training; The prediction output of the fire situation prediction model is combined with the GIS map engine to obtain a fire prediction situation map.

3. According to claim 2, the 5G network-based air-ground integrated coordinated command forest fire extinguishing system is characterized in that: The air-ground integrated coordinated command decision display unit is also used to display forest fire data; Including the display of live fire scene video, dynamic display of real-time emergency equipment, personnel and materials, and data display of fire situation / fire change trend; The method for displaying the live fire scene video is as follows: based on the 5G communication link unit, the fire situation video stream data in the load unit is retrieved in real time through the video streaming service unit, and the real-time live broadcast picture of the drone is displayed based on the split-screen method; The dynamic display method of real-time emergency equipment, personnel and materials is: based on the GIS map engine, multiple layers are used to display the dynamic distribution information of air rescue equipment, ground fire-fighting facilities and rescue and fire-fighting personnel; The method for displaying the data of the fire situation / fire change trend is as follows: based on the fire situation intelligent analysis unit, according to the current real-time information of the forest fire, a fire prediction situation diagram for fire situation prediction is dynamically displayed.

4. According to claim 3, the 5G network-based air-ground integrated coordinated command forest fire extinguishing system is characterized in that: The comprehensive scheduling unit includes a basic information management module, a control command scheduling module, a real-time data transmission module and a message notification module; The basic information management module is used to manage the data of rescue emergency equipment, shouting and lighting drones, fire-fighting drones, detection drones, fire trucks, firefighters, fire-fighting and rescue materials, and water sources; The control command scheduling module is used to receive the control task instructions of the air-ground integrated coordinated command decision display unit; Based on the control task instructions, according to the rescue fire-fighting equipment, rescue fire-fighting personnel and rescue fire-fighting materials at all levels, the number of fire-fighting drones, fire-fighting helicopters, fire trucks and firefighters is planned in real time to configure the fire-fighting route, and the amount of materials to be carried and the planned travel time are confirmed, and sent to the fire-fighting drones, fire-fighting helicopters and mobile dispatch control terminal through the message notification module; The real-time data transmission module is used to receive the transmission data between the UAV load unit and the mobile scheduling control end transmitted in real time through the data transmission uplink unit.

5. According to claim 4, the 5G network-based air-ground integrated coordinated command forest fire extinguishing system is characterized in that: The load dispatching unit also includes a device control / data acquisition module for monitoring aerial firefighting and rescue facilities to obtain drone data and sampling data; And obtain real-time information on forest fires based on drone data and sampling data; The drone data at least includes real-time latitude and longitude position data of the aircraft, flight data of flight speed / roll angle / pitch angle / yaw angle, signal data of satellite signal / remote sensing signal / image transmission signal, battery data of battery temperature / voltage / power, and lens data of gimbal pitch angle / yaw angle / roll angle; The sampled data at least includes fire extinguisher bomb inventory quantity data, video image data captured by wide-angle / zoom / infrared cameras, geometric position data collected by lidar, and temperature / humidity / air pressure / wind speed / wind direction data collected by meteorological detectors.

6. The 5G network-based air-ground integrated coordinated command forest fire extinguishing system according to claim 5 is characterized in that: The aerial fire-fighting and rescue facilities at least include a shouting and lighting drone equipped with an all-in-one shouting and lighting device and a 5G Celluar module, a fire-fighting drone equipped with a fire-extinguishing bomb throwing device and a 5G Celluar module, and a detection drone equipped with a wide-angle / zoom / infrared camera / lidar / meteorological detector load device and a 5G Celluar module.

7. The 5G network-based air-ground integrated coordinated command forest fire extinguishing system according to claim 5 is characterized in that: Also included is a data storage unit; The data storage unit includes an Hbase column storage database module, a Mysql relational database module and an OSS file storage module; The Hbase column storage database module is used to store unstructured data collected by the drone load unit, and the unstructured data includes weather temperature, humidity, air pressure, wind speed, wind direction and real-time longitude & latitude data; The Mysql relational database module is used to store structured data obtained by the load scheduling unit; and the structured data includes fire-fighting drones, fire-fighting helicopters, fire trucks, firefighters, fire-fighting emergency supplies, water sources, and drone status data; The OSS file storage module is used to store image data, video data and document data of air-ground integrated collaborative fire fighting tasks during the forest fire fighting process.

8. The 5G network-based air-ground integrated coordinated command forest fire extinguishing system according to claim 5 is characterized in that: It also includes a video streaming service unit, and the video streaming service unit is used to retrieve the fire situation video stream data in the load scheduling unit in real time through the 5G communication link unit, and display the real-time live screen of the fire situation video stream data based on the split-screen method; The air-ground integrated coordinated command decision display unit is also used to manually assist and monitor the execution of the air-ground integrated coordinated command forest fire fighting task based on the real-time live broadcast picture.

Images