Space-air-ground integrated intelligent forestry supervision system

Abstract

The invention discloses a space-air-ground integrated intelligent forestry supervision system, which comprises a monitoring end and a service management platform, and is characterized in that the monitoring end returns and converges monitored information to the service management platform in real time through a network, and the service management platform realizes dynamic management and control of forest and grass resources in a three-dimensional scene, including daily monitoring, display and auxiliary decision making. According to the invention, an all-weather and full-coverage forest and grass resource sky, air and ground integrated monitoring network is established, and various monitoring information is transmitted back and converged to a command center in real time through the network and is displayed, browsed, analyzed, counted and the like in a forest and grass resource sky, air and ground integrated intelligent monitoring management platform system. The whole-process and all-directional dynamic management and control service of high integration of forest and grass resource daily monitoring, fire point rapid positioning, fighting command decision making and disaster loss evaluation in a three-dimensional scene is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of forest and grassland supervision and relates to an air-ground-ground integrated intelligent forestry supervision system. Background Art

[0002] The world has entered the information age. Information technology is strongly and profoundly leading the profound changes in all areas of the economy and society. Forest and grassland management and monitoring are also facing a rapid and unstoppable information revolution. Promoting innovation in forest and grassland resource management and monitoring with informatization and driving the scientific, standardized and refined supervision of forests and grasslands has become the only way for the scientific development of forest and grassland supervision.

[0003] In the past decade, my country's forest and grassland resource informatization has made significant progress and achieved results. The forest and grassland resource informatization system has been basically established, effectively improving the ability to supervise resources and participate in macro-control. However, facing the new situation and new requirements of forest and grassland resources, the current level of informatization development is still insufficient compared with the urgent need for informatization in the innovation of forest and grassland resource monitoring methods and the efforts to explore new mechanisms to ensure scientific development:

[0004] (1) The resource monitoring methods are single, and traditional protection and management methods are difficult and ineffective;

[0005] (2) The means of data acquisition and updating are poor, and cannot accurately reflect the current status and dynamic growth and decline of resources in a timely manner;

[0006] (3) Insufficient investment in new equipment and technologies has resulted in long monitoring cycles that cannot meet the requirements of modern forestry dynamic monitoring;

[0007] (4) Data support capabilities are relatively weak, and data accuracy, currency, and integrity need to be improved urgently, which seriously hinders the overall information development of forest and grassland resources. Summary of the invention

[0008] The technical problem solved by the present invention is to provide an integrated air-space-ground intelligent forestry supervision system, which can establish an all-weather, full-coverage air, space and ground integrated monitoring network for forest and grass resources, and provide integrated full-process, all-round dynamic management and control services.

[0009] The present invention is achieved through the following technical solutions:

[0010] An integrated air-ground-air intelligent forestry supervision system includes a monitoring terminal and a service management platform. The monitoring terminal transmits the monitored information back to the service management platform in real time through the network. The service management platform realizes dynamic control of forest and grass resources in a three-dimensional scene, including daily monitoring, display, and auxiliary decision-making.

[0011] The monitoring terminals include remote sensing satellites, drone inspections, forest ranger handheld patrol terminals, and video monitoring and early warning base stations;

[0012] The service management platform obtains high-resolution remote sensing images of forest areas provided by satellite remote sensing, digital elevation models converted from remote sensing images or ground measurements, and oblique photography models generated based on drone aerial photography data, and generates real-scene three-dimensional models in combination with three-dimensional data collected by laser scanning; it follows the J2EE standard and builds a two- and three-dimensional integrated forest area space display platform by using charts, graphics, maps and visualization elements;

[0013] The spatial coordinates collected from video surveillance early warning base stations, drone inspections and forest rangers' handheld patrol terminals are displayed in real time in the form of icons on a two- and three-dimensional integrated forest space display platform; the monitoring videos sent back by video surveillance base stations and drones can be viewed in real time, and the forest rangers' locations, patrol trajectories and reported events can be viewed.

[0014] Furthermore, drone inspection uses unmanned aircraft / unmanned airships equipped with infrared and / or visible light cameras to monitor forest fires and transmit the monitoring images back to the service management platform in real time via 4G / 5G mobile Internet;

[0015] The service management platform monitors forest fires through dual-layer fire identification, and extracts the fire contours based on the edge computing algorithm to calculate the fire area. It determines the spread of the fire based on the changes in the extracted fire contours, monitors whether a forest fire has occurred, and the development of the fire after it has occurred.

[0016] The two-level fire and smoke recognition, the first level is based on the image fire and smoke recognition embedded in the video monitoring equipment of the video monitoring base station to monitor the forest fire; the second level is based on the fire and smoke recognition of the monitoring video stream to perform secondary recognition to monitor the forest fire;

[0017] The edge computing algorithm is the processing and analysis running on the edge monitoring device. On the one hand, the edge monitoring device responds to and processes the monitoring data in real time; on the other hand, the processed data can be transmitted back to the service management platform for further analysis and storage.

[0018] Furthermore, a drone ground station is also set up. The drone ground station is a relay station connecting the drone inspection and service management platform, realizing the connection between the service management platform and the drone ground station; establishing a transmission channel for the return of inspection videos and flight trajectory information, and real-time viewing of information including the number of aircraft carrying out the mission, affiliated units, flight trajectories, and current locations; and providing the drone flight trajectory and return video to the service management platform.

[0019] During the drone inspection, the live footage captured by the drone is transmitted back in real time and displayed live through the streaming media push authentication mechanism:

[0020] After the drone inspection passes the authentication, the video images recorded on site are transmitted back by streaming: the audio and video data are divided into small data blocks, each with its own identification and length information, and transmitted to the service management platform one by one in the form of streams; at the same time, the flight information is synchronously transmitted back; users with permission can pull the stream to watch the live broadcast;

[0021] The live broadcast will also include the flight information including the current coordinates of the drone, including latitude and longitude, altitude, and azimuth, as well as the drone's live images, which will be displayed in real time on the service management platform. Management personnel can view the live inspection or inspection playback function to identify and confirm the fire situation and fire danger, and accurately locate the fire;

[0022] The service management platform can also calculate the number of live streams, network speed, and count the number of people online, live broadcast rate, and live broadcast duration.

[0023] Furthermore, the streaming media service authentication mechanism is:

[0024] 1) A new drone account can be created in the service management platform, and a corresponding sign authentication code will be randomly generated;

[0025] 2) The drone account is used to log in to the drone's operating tablet and automatically detect the real-time video images sent back by the drone; the operating tablet encapsulates the video image data in a streaming request, which contains the drone account's sign authentication code; the operating tablet backend service sends the streaming request to the streaming media server in real time;

[0026] 3) After receiving the push request, the streaming media server extracts the drone account sign authentication code from the push request, queries the database based on the authentication code, and verifies whether the authentication code matches; if the authentication code matches, the streaming media server will actively accept the push request and convert the video image data into the required format for storage or transmission to the viewing client;

[0027] 4) To avoid repeated streaming by the drone account, the streaming media server will check whether the drone account is already streaming; if so, it will reject the new streaming request.

[0028] Furthermore, the service management platform can view the patrol personnel’s location and historical patrol tracks in real time, analyze and process the on-site incident data such as text, voice, photos, and videos reported by patrol personnel;

[0029] Patrol personnel and patrol targets are assigned to specific areas and mountains to implement grid management.

[0030] Furthermore, the auxiliary decision-making service management platform combines the distribution of forest settlements, roads, water systems, fire-fighting teams and forest and grass resource distribution maps, and obtains the distribution of settlements around the disaster site, the nearest water point, the nearest fire-fighting team, the best rescue route, and disaster loss assessment through shortest path analysis, nearest facility analysis, buffer zone analysis, isolation zone setting, field of view analysis, overlay analysis, and loss assessment analysis, thereby providing decision-making assistance for the rapid and standardized formulation of decision-making plans.

[0031] The shortest path analysis is performed by selecting the starting point and the end point coordinates on the map interface, or importing the coordinates of the uploaded event as the starting point and the end point coordinates, and analyzing the shortest distance between two points by the shortest distance and the road priority;

[0032] The nearest facility analysis is to select a point on the map interface or import the coordinates of the uploaded event, set the search radius, and select the shortest distance and highway priority to analyze the nearest roads, rivers, settlements and firefighting force deployment;

[0033] The buffer zone analysis is to select points on the map interface or import the coordinate points of uploaded events, set the buffer radius, level and interval distance, and then analyze the forest area, number of management stations, number of watchtowers and number of settlements within the buffer zone;

[0034] The visual field analysis is to select the coordinates of the observation point on the map interface, set the offset of the observation point and the visual field range to analyze the visual field range that can be observed by the current observation point within the visual field range;

[0035] The overlay analysis is to analyze the small class information overlaid with the graphics by drawing graphics or importing graphics from outside, selecting the overlay method of cutting, including or intersecting, and the overlay results can be statistically summarized and exported;

[0036] The isolation belt is drawn manually on the map by selecting a point on the map interface or importing the coordinates of the uploaded event, setting the width of the isolation belt, and analyzing the length of the isolation belt, the area of ​​the isolation belt, the area of ​​the forest land in the isolation belt, and the distance between the isolation belt and the incident site;

[0037] The loss assessment analysis is carried out by drawing the scope of the loss assessment area or the external import scope, and obtaining the resource patches within the assessment area through spatial overlay relationships. The disaster-stricken area, forest stand damage degree, and timber volume loss can be statistically calculated through the attributes of all the resource patches obtained, including area, land type, forest protection level, number of trees, stocking, and dominant tree species.

[0038] Compared with the prior art, the present invention has the following beneficial technical effects:

[0039] The present invention integrates multiple monitoring measures such as satellite remote sensing, aerial inspection, video surveillance and early warning base stations, and forest ranger patrols to establish an all-weather, full-coverage integrated air, space, and ground monitoring network for forest and grassland resources. Various monitoring information is transmitted back in real time to the command center through the network, and displayed, browsed, analyzed, and counted on the forest and grassland resource air, space, and ground integrated intelligent monitoring and management platform, thus realizing highly integrated full-process, all-round dynamic management and control services for daily monitoring of forest and grassland resources in three-dimensional scenes, rapid positioning of fire points, firefighting command decision-making, and disaster loss assessment.

[0040] The present invention follows the OGC (Open Geospatial Consortium) data and service standards, utilizes multi-source spatial data such as high-resolution remote sensing images, DEM, oblique photography, and real-life three-dimensional models of forest areas, follows the J2EE standard, and builds a two- and three-dimensional integrated forest area spatial display platform by using charts, graphs, maps, and other visualization elements. The collected forest fire video monitoring base station location coordinates, and the spatial location coordinates sent back by drones and forest rangers in real time are uniformly displayed in different icon styles on the two- and three-dimensional integrated forest area spatial display platform, and the monitoring videos sent back by the monitoring base station and drones, the location information of the positioning base station monitoring and early warning, the location of the forest ranger and the patrol track, and the reported events can be viewed in real time;

[0041] The present invention can be combined with the distribution of forest settlements, roads, water systems, fire-fighting teams and forest and grass resource distribution maps. Through the shortest path analysis, the nearest facility analysis, the buffer zone analysis, the isolation zone, the overlay analysis and the loss assessment analysis, the distribution of settlements around the disaster site, the nearest water point, the nearest fire-fighting team, the best rescue route, the disaster loss assessment and the like can be quickly obtained, thereby providing a scientific basis for the rapid and standardized formulation of decision-making plans.

[0042] With the support of the ground support system, drone inspection combines flight control and navigation technology, data transmission and storage technology, monitors forest fires by carrying infrared and visible light cameras, and transmits the monitoring images back to the command center in real time through 4G / 5G high-speed mobile Internet. The fire scene is confirmed by the smoke and fire recognition algorithm, and the fire scene contour is extracted according to the edge computing algorithm. The fire area is calculated, and the fire spread is dynamically determined according to the changes in the extracted fire scene contour, so as to monitor the occurrence and development of forest fires. The use of drones for inspections is not affected by terrain and can effectively clear blind spots, greatly improve construction efficiency, and reduce the risks of operators; in particular, the use of unmanned airships equipped with lighting equipment can also realize night inspections.

[0043] The service management platform can be connected to the drone ground station to establish a transmission channel for the inspection video and flight trajectory information to be transmitted back, and to view the number of aircraft carrying out the mission, the unit to which they belong, the flight trajectory, the current location and other information in real time. The drone flight trajectory and the returned video are provided to the sky-ground integrated intelligent monitoring and management platform, so that decision makers can be aware of the dynamic changes in the remote site in a timely manner.

[0044] The present invention is based on the forest and grass resource database, and can equip each forest ranger with a patrol mobile terminal. The command center can view the patrol personnel's location and historical patrol trajectory in real time, analyze and process the on-site data of events such as text, voice, photos, and videos reported by the patrol personnel. The system implements patrol personnel and patrol objects to specific land and mountain blocks, realizes grid management, and fully solves the problems of patrol personnel's lack of responsibility, inadequate patrol, and untimely supervision. It can effectively strengthen forest fire prevention and pest control work, put an end to indiscriminate logging, protect forest and grass resources and the ecological environment, and provide comprehensive support for management departments in terms of patrol networks, patrol personnel, patrol equipment and facilities, patrol information, and patrol assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the implementation process of real-time positioning and trajectory recording of personnel;

[0046] Figure 2 It is a schematic diagram of the implementation process of event multimedia information collection and feedback;

[0047] Figure 3 Schematic diagram of the implementation process of automated attendance calculation and statistics based on spatial analysis. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below in conjunction with the embodiments, which are intended to explain the present invention rather than to limit it.

[0049] An integrated air-ground-air intelligent forestry supervision system includes a monitoring terminal and a service management platform. The monitoring terminal transmits the monitored information back to the service management platform in real time through the network. The service management platform realizes dynamic control of forest and grass resources in a three-dimensional scene, including daily monitoring, display, and auxiliary decision-making.

[0050] The monitoring terminals include remote sensing satellites, drone inspections, forest ranger handheld patrol terminals, and video monitoring and early warning base stations;

[0051] The service management platform obtains high-resolution remote sensing images of forest areas provided by satellite remote sensing, digital elevation models converted from remote sensing images or ground measurements, and oblique photography models generated based on drone aerial photography data, and generates real-scene three-dimensional models in combination with three-dimensional data collected by laser scanning; it follows the J2EE standard and builds a two- and three-dimensional integrated forest area space display platform by using charts, graphics, maps and visualization elements;

[0052] The spatial coordinates collected from video surveillance early warning base stations, drone inspections and forest rangers' handheld patrol terminals are displayed in real time in the form of icons on a two- and three-dimensional integrated forest space display platform; the monitoring videos sent back by video surveillance base stations and drones can be viewed in real time, and the forest rangers' locations, patrol trajectories and reported events can be viewed.

[0053] The present invention follows the OGC (Open Geospatial Consortium) data and service standards, utilizes multi-source spatial data such as high-resolution remote sensing images, DEM, oblique photography, and real-life three-dimensional models of forest areas, follows the J2EE standard, and constructs a two- and three-dimensional integrated forest area spatial display platform by using charts, graphs, maps, and other visualization elements. The collected forest fire video monitoring base station location coordinates, and the spatial location coordinates sent back by drones and forest rangers in real time are uniformly displayed on the two- and three-dimensional integrated forest area spatial display platform in different icon styles, and the monitoring videos sent back by the monitoring base station and drones, the location information of the monitoring and early warning of the positioning base station, the location of the forest ranger and the patrol trajectory, and the reported events are viewed in real time.

[0054] The following is a detailed description of each part.

[0055] 1. Drone inspection

[0056] Drone inspection uses unmanned aircraft / unmanned airships equipped with infrared and / or visible light cameras to monitor forest fires and transmit the monitoring images back to the service management platform in real time via 4G / 5G mobile Internet;

[0057] The service management platform monitors forest fires through dual-layer fire identification, and extracts the fire contours based on the edge computing algorithm to calculate the fire area. It determines the spread of the fire based on the changes in the extracted fire contours, monitors whether a forest fire has occurred, and the development of the fire after it has occurred.

[0058] The two-level fire and smoke recognition, the first level is based on the image fire and smoke recognition embedded in the video monitoring equipment of the video monitoring base station to monitor the forest fire; the second level is based on the fire and smoke recognition of the monitoring video stream to perform secondary recognition to monitor the forest fire;

[0059] The edge computing algorithm is the processing and analysis running on the edge monitoring device. On the one hand, the edge monitoring device responds to and processes the monitoring data in real time; on the other hand, the processed data can be transmitted back to the service management platform for further analysis and storage.

[0060] Furthermore, a drone ground station is also set up. The drone ground station is a relay station connecting the drone inspection and service management platform, realizing the connection between the service management platform and the drone ground station; establishing a transmission channel for the return of inspection videos and flight trajectory information, and real-time viewing of information including the number of aircraft carrying out the mission, affiliated units, flight trajectories, and current locations; and providing the drone flight trajectory and return video to the service management platform.

[0061] The following is a detailed explanation of drone inspection and live broadcast.

[0062] 1.1. Live broadcast transmission on site, which can transmit and display the live scene shot by drone in real time;

[0063] Streaming is a data transmission method that divides audio and video data into small data blocks. Each block has its own identifier and length information, and is transmitted one by one in the form of a stream. This transmission method has been widely used in the network field to improve transmission efficiency and reduce latency. Online live broadcast is also one of the typical applications of streaming technology.

[0064] The live broadcast of drone gold hunting can transmit the live video to the client device in real time, achieving the effect of real-time viewing.

[0065] In addition, streaming technology can also adaptively adjust the video bit rate according to the network environment, improving the stability of live video and realizing live broadcast backhaul via drones.

[0066] Specifically, the live broadcast is transmitted as follows:

[0067] 1) After the drone passes the authentication, it uses streaming technology to transmit the video images recorded on site back to the streaming media server; at the same time, the flight information is synchronously transmitted back.

[0068] 2) The server backend calculates the number of live streams, network speed and other information to calculate the number of people online, live broadcast rate and live broadcast duration.

[0069] 3) The live broadcast inspection module of the drone management system requests the server to list the live broadcasts currently being broadcast, and displays them on the front end together with the number of online users and live broadcast rate for each live broadcast item calculated in the background.

[0070] 4) Users with viewing permissions can stream and watch the live broadcast.

[0071] 1.2 Streaming Media Service Authentication Mechanism

[0072] (1) A new drone account can be created in the drone management system, and a corresponding sign authentication code will be randomly generated.

[0073] (2) The drone account is used to log in to the drone's operating tablet (mobile tablet). The mobile tablet can control the drone and automatically detect the real-time video images sent back by the drone. The mobile tablet encapsulates the video image data in a streaming request, which contains the drone account's sign authentication code. The mobile tablet backend service sends the streaming request to the streaming server in real time.

[0074] (3) After receiving the push request, the streaming server will extract the drone account sign authentication code from the request. The database will be queried based on the authentication code to verify whether the authentication code matches. If the authentication code matches, the streaming server will actively accept the push request and convert the video image data into the required format for storage or transmission to the viewing client such as a Web application.

[0075] (4) To avoid repeated streaming by the drone account, the streaming server will check whether the drone account is already streaming. If so, the new streaming request will be rejected.

[0076] The streaming media push authentication mechanism effectively prevents malicious occupation and malicious attacks on the server by third parties. Through authentication, only users who have passed the push verification can push the stream, and multiple users cannot use the same account to push the stream at the same time, ensuring the security and effective push of the server.

[0077] 3. Track video synchronous display

[0078] (1) Use streaming technology to transmit the video images recorded on site back to the streaming media server; at the same time, the flight information is synchronously transmitted back.

[0079] (2) When viewing live broadcasts or playbacks, the background extracts the drone streaming time and the flight time in the returned flight information and performs overlapping operations to synchronize the flight information and the drone position at the same time.

[0080] (3) The flight coordinates are plotted as the drone trajectory route, and the current position of the drone and the flight information are synchronously displayed on the small map.

[0081] The flight information such as the current coordinates of the UAV, longitude and latitude, altitude, azimuth, etc., as well as the real-time image of the UAV can be synchronously displayed on the management system. Management personnel can view it through the live inspection or inspection playback function, identify and confirm the fire situation and fire risk, and accurately locate it.

[0082] The real-time feedback from drones can enable managers to understand the real-time situation of forest reserves more quickly and comprehensively, which helps to detect forest fires early, obtain fire information in a timely manner, and quickly deploy fire prevention forces.

[0083] Forest resources are mostly concentrated in vast mountainous areas with poor transportation conditions and difficulty for people and vehicles to reach the area. Compared with other methods, drones have the advantages of low cost, easy deployment, convenience and speed, and are an effective supplement to forest rangers' patrols.

[0084] 2. Forest ranger inspection

[0085] 2.1 Personnel real-time positioning and trajectory recording process

[0086] (1) Read the data stream of the forest ranger's handheld patrol terminal and extract the current GPS coordinates. Filter the data according to the scope of the monitoring area and the coordinates of the previous point to eliminate coordinate anomalies caused by GPS signal problems or electromagnetic interference, thereby improving positioning accuracy.

[0087] (2) The filtered GPS coordinates are fed back to the system for location display and recorded in the trajectory.

[0088] (3) The movement status is obtained from the acceleration sensor of the patrol terminal, and combined with whether the distance between the current position and the previous position is less than the set threshold, a comprehensive judgment is made as to whether the ranger is moving or resting; the timing starts when it is determined that the ranger is resting, and if the continuous rest time exceeds the set threshold, the ranger is given a stay alarm.

[0089] (4) After the patrol is completed, the generated trajectory point set is optimized by using an iterative adaptive point algorithm to reduce the number of trajectory points while ensuring data accuracy and complete trajectory lines.

[0090] (5) The optimized trajectory is transmitted back to the data center through the network.

[0091] 2.2 Event multimedia information collection and transmission process

[0092] (1) The ranger patrol terminal collects and stores complete event information (photos, videos, voice, coordinates, date, and text description) and transmits it back to the data center server via the network.

[0093] (2) Deploy the message middleware service on the data center server, monitor the fixed port, and put the event, location, trajectory and other information sent back by different patrol terminals into the message queue.

[0094] (3) The message service schedules the message queue and writes the data in the queue to the database in time priority order for the management system to query and count, and removes the data entries that have been written to the database from the queue.

[0095] (4) The message service continues to scan the message queue to read the next data entry.

[0096] 3.3 Automated attendance calculation and statistical process based on spatial analysis technology

[0097] (1) In the management system, a grid responsibility area (surface element graphic) can be drawn for each forest ranger on the remote sensing image.

[0098] (2) The system automatically pushes the drawn grid responsibility area to the ranger patrol terminal, which can display and locate the ranger's area position on the remote sensing image and navigate to the area position.

[0099] (3) After the patrol is completed, the system automatically overlays and analyzes the patrol trajectory of the day (line element graphic, Polyline) and the responsible area graphic (surface element graphic, Polygon), and intercepts the part of the trajectory that is within the responsible area.

[0100] (4) Calculate the length of the intercepted trajectory within the responsible area to determine whether its length is greater than or equal to the prescribed daily attendance mileage, so as to determine whether the personnel's patrol performance on that day has met the standards, and automatically display the results on the terminal screen and push them to the data center.

[0101] 3.4 Real-time positioning and trajectory recording of personnel

[0102] It can monitor the real-time location of forest rangers and record their trajectories, and render and analyze patrol trajectories at a lower data storage cost and faster rendering speed without any lag.

[0103] Since the real-time location and movement trajectory of personnel can be viewed in the management system and their attendance can be supervised, the problem of forest rangers falsifying information and deceiving higher-level management departments can be fundamentally eliminated.

[0104] The use of filtering algorithms to process positioning coordinates can effectively avoid the problem of inaccurate positioning of forest rangers caused by GPS signal obstruction or electromagnetic interference.

[0105] The trajectory point set generated by the patrol process is optimized by iterative adaptive point algorithm. The number of trajectory points is reduced while ensuring data accuracy and complete trajectory lines. This can improve the rendering speed of trajectory data and lay the foundation for the background to use patrol trajectories to analyze patrol blind spots.

[0106] 3.5 Event multimedia information collection and transmission

[0107] Information on forest resource damage at the patrol site (such as fire risk, illegal logging and poaching, deforestation, illegal land occupation, geological disasters, etc.) can be transmitted back to the data center in real time in rich multimedia formats. The message queue scheduling service can store data with minimal delay (ms level) and display it on the management platform. Management decision-makers can obtain rich and detailed information on the site at the first time, providing a basis for scientific decision-making.

[0108] (1) In addition to commonly used text description information, the event data collection module also supports multiple forms of data such as photos, videos, voice, latitude and longitude coordinates, date and time, and any combination of these types of data, and supports real-time transmission of these data to the data center, greatly enriching the reference basis for management decision makers to conduct any query and decision analysis.

[0109] (2) The message service can manage the queues of events, locations, trajectories, and other information sent back by different rangers. It can process at least 5,000 records simultaneously with a delay of less than 1 second, ensuring real-time event response.

[0110] 4. Intelligent assistance for decision-making and command

[0111] The service management platform combines the distribution of forest settlements, roads, water systems, fire-fighting teams, and forest and grass resource distribution maps. Through the shortest path analysis, nearest facility analysis, buffer zone analysis, isolation zone setting, field of view analysis, overlay analysis, and loss assessment analysis, it obtains the distribution of settlements around the disaster site, the nearest water point, the nearest fire-fighting team, the best rescue route, and disaster loss assessment, providing decision-making assistance for the rapid and standardized formulation of decision-making plans.

[0112] Among them, the viewshed analysis can set the best installation location of the video surveillance station according to the viewshed range of the observation point; the nearest facility analysis, shortest path analysis, buffer analysis, and isolation belt analysis can analyze the resource distribution status and provide feasible routes for forest fire fighting and dispatching;

[0113] Disaster loss assessment can automatically analyze the affected area according to the fire scope, extract statistical information such as forest area, tree species, stock volume, protection level, etc. in the affected area, and display it visually.

[0114] Specifically, the analysis in intelligent assistance for decision-making and command is as follows:

[0115] (1) The shortest path analysis is to select the starting and ending coordinates on the map interface or import the coordinates of the uploaded event as the starting and ending coordinates. The shortest distance between two points can be analyzed by the shortest distance and road priority methods.

[0116] (2) The nearest facility analysis is to select a point on the map interface or import the coordinates of an uploaded event, set the search radius, and select the shortest distance or highway priority to analyze the nearest roads, rivers, settlements, troop deployments and other facility information.

[0117] (3) Buffer zone analysis is to select points on the map interface or import the coordinate points of uploaded events, set the buffer radius, level, and interval distance, and then analyze information such as forest area, number of management stations, number of watchtowers, and number of settlements within the buffer zone.

[0118] (4) Viewshed analysis is to select the observation point coordinates on the map interface, set the observation point offset and the field of view, and analyze the field of view that can be observed by the current observation point within the field of view.

[0119] (5) Overlay analysis is to draw graphics or import graphics from outside, and select the overlay method of cutting, including or intersecting to analyze the small class information overlaid with the graphics, and the overlay results can be statistically summarized and exported.

[0120] (6) The isolation zone is set by selecting points on the map interface or importing the coordinate points of uploaded events. After manually drawing the isolation zone on the map, the length of the isolation zone, the area of ​​the isolation zone, the area of ​​forest land within the isolation zone, and the distance from the isolation zone to the incident site can be analyzed.

[0121] (7) Loss assessment analysis is to draw the scope of the loss assessment area or the external import scope, and obtain the resource patches within the assessment area through spatial overlay relationships. The area, land type, forest protection level, number of trees, stocking, and dominant tree species of all the resource patches obtained can be used to calculate the affected area, forest stand damage degree, and tree volume loss.

[0122] The specific steps for each measure are given below.

[0123] See also Figure 1 The system shown in the figure implements the flow chart of real-time positioning and track record management of forest rangers, including the following steps:

[0124] (1) Step S101 obtains the GPS location coordinates provided by the forest ranger patrol terminal hardware device.

[0125] (2) Step S102 performs filtering on the acquired coordinates. If the coordinate value of the point is determined to be abnormal, the coordinate value is discarded and the hardware device coordinates are read continuously; if there is no abnormality after filtering, the process proceeds to step S103.

[0126] (3) Step S103 displays the filtered coordinate position on the map interface of the forest ranger's handheld mobile patrol terminal and transmits it back to the data center server in real time. The administrator can locate the forest ranger's current position in the desktop patrol management system.

[0127] (4) Step S104 inserts the coordinates into the trajectory point set to generate a trajectory line.

[0128] (5) Step S105 determines whether the current state is a patrol movement state or a resting state based on the relationship between the current position and the previous position and the ranger's movement information obtained from the acceleration sensor.

[0129] (6) Step S106 is to determine whether the dwell time exceeds the warning threshold based on Step S105. If it exceeds the warning threshold, an audible and visual alarm is sounded to issue a dwell warning.

[0130] (7) Step S107 is to use an iterative adaptive point algorithm to perform thinning optimization on the trajectory point set at the end of the patrol, so as to minimize the amount of trajectory data while ensuring data accuracy and complete trajectory lines.

[0131] (8) Step S108 is to transmit the thinned trajectory data back to the data center. The management personnel can overlay the trajectory on the map in the patrol management system for display and analysis to confirm the patrol effect of the forest ranger and analyze the patrol blind spots, and adjust the patrol routes of the personnel according to the analysis results.

[0132] See also Figure 2 The event multimedia information collection and return example implementation flow chart shown includes the following steps:

[0133] (1) Step S201 is to collect and store multimedia information of the event, including event location coordinates, date and time, and one or more combinations of photos, videos, voice, and text descriptions.

[0134] (2) Step S202 is event feedback, where the system transmits the collected event location coordinates, date and time, and multimedia information such as photos and videos to the data center server.

[0135] (3) Steps S203 and S204 are event monitoring and storage. The message queue service deployed on the data center server monitors the location, trajectory, and event information sent back by each ranger and adds them to the message queue for sorting. The scheduling service reads the complete records from the queue in chronological order and stores them in the database.

[0136] (4) Step S205 is for the management personnel to browse and view the events in the management system and make scheduling decisions based on the analysis results.

[0137] See also Figure 3The automated attendance calculation and statistics example implementation flow chart shown includes the following steps:

[0138] (1) Steps S301 and S302 are completed by the management personnel at the management end. The management personnel use the forest area satellite image and topographic map as the base map to draw the responsibility area in the management system, and issue the responsibility area to specific forest rangers, binding the specific forest rangers with the responsibility area.

[0139] (2) Step S303: After the ranger logs in to the patrol terminal, the patrol terminal system automatically polls to obtain the latest assigned area of ​​responsibility for the ranger (if there is no latest assigned area, the previously assigned area will still be used), and the area will be displayed on the map, so that the range of the ranger can be clearly seen.

[0140] (3) Step S304 is to start patrol track recording.

[0141] (4) Steps S305 and S306 are the end of the patrol. The system performs a superposition analysis on the patrol trajectory and the responsible area to determine whether the patrol trajectory falls within the responsible area. If it falls within the area, it is considered valid. The valid trajectory that falls within the area is extracted through the intersection clipping algorithm, and the trajectory mileage Li and the duration Ti of the valid trajectory are calculated.

[0142] (5) Step S307 is to accumulate the effective track mileage Li of all the extracted effective tracks on the day to obtain the total effective track mileage L of the day, and transmit the total effective track mileage L to the attendance determination module for comparison with the mileage threshold specified for compliance (the threshold is built-in to the module and can be adjusted by the developer, the same below), and the module determines whether the total track mileage L of the attendance on the day meets the standard; similarly, the duration Ti of all the extracted effective tracks on the day is accumulated to obtain the total duration T of the effective tracks on the day, and transmit the total effective track duration T to the attendance determination module for comparison with the time threshold specified for compliance, and the module determines whether the total track duration T of the attendance on the day meets the standard. If either the total track duration or the total track mileage is satisfied, it is determined that the forest ranger’s attendance on the day meets the standard.

[0143] (6) Step S308 is to transmit the result of whether the standard is met to the data center.

[0144] (7) Step S309: The message service running on the server adds the monitored attendance information to the message queue and finally stores it in the database, so that the management personnel can view and analyze it.

[0145] (8) Step S308 is that the attendance statistics module of the patrol management system can pull the forest ranger's attendance information from the database, generate an attendance report according to the monthly / quarterly / yearly layout, and display the number of days for each forest ranger to meet the attendance standard, the number of days not meeting the standard, the number of days not present, as well as the total track mileage and patrol time.

[0146] The above embodiments are preferred examples for implementing the present invention, and the present invention is not limited to the above embodiments. Any non-essential additions and substitutions made by those skilled in the art based on the technical features of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. An air-ground-integrated intelligent forestry supervision system, characterized in that: It includes a monitoring terminal and a service management platform. The monitoring terminal transmits the monitored information back to the service management platform in real time through the network. The service management platform realizes dynamic control of forest and grass resources in three-dimensional scenes, including daily monitoring, display, and auxiliary decision-making. The monitoring terminals include remote sensing satellites, drone inspections, forest ranger handheld patrol terminals, and video monitoring and early warning base stations; The service management platform obtains high-resolution remote sensing images of forest areas provided by satellite remote sensing, digital elevation models converted from remote sensing images or ground measurements, and oblique photography models generated based on drone aerial photography data, and generates real-scene three-dimensional models in combination with three-dimensional data collected by laser scanning; it follows the J2EE standard and builds a two- and three-dimensional integrated forest area space display platform by using charts, graphics, maps and visualization elements; The spatial coordinates collected from video surveillance early warning base stations, drone inspections and forest rangers’ handheld patrol terminals are displayed in real time in the form of icons on the two- and three-dimensional integrated forest area spatial display platform. View the surveillance videos sent back by video surveillance base stations and drones in real time, and check the ranger's location and patrol trajectory as well as reported events.

2. The air-ground-integrated intelligent forestry supervision system according to claim 1 is characterized in that: Drone inspection uses unmanned aircraft / unmanned airships equipped with infrared and / or visible light cameras to monitor forest fires and transmit the monitoring images back to the service management platform in real time via 4G / 5G mobile Internet; The service management platform monitors forest fires through dual-layer fire identification, and extracts the fire contours based on the edge computing algorithm to calculate the fire area. It determines the spread of the fire based on the changes in the extracted fire contours, monitors whether a forest fire has occurred, and the development of the fire after it has occurred. The two-level fire and smoke recognition, the first level is based on the image fire and smoke recognition embedded in the video monitoring equipment of the video monitoring base station to monitor the forest fire; the second level is based on the fire and smoke recognition of the monitoring video stream to perform secondary recognition to monitor the forest fire; The edge computing algorithm is the processing and analysis running on the edge monitoring device. On the one hand, the edge monitoring device responds to and processes the monitoring data in real time; on the other hand, the processed data can be transmitted back to the service management platform for further analysis and storage.

3. The air-ground-integrated intelligent forestry supervision system according to claim 1 or 2, characterized in that: There is also a drone ground station, which serves as a relay station for drone inspection and service management platform, to connect the service management platform with the drone ground station. It establishes a transmission channel for the return of inspection videos and flight trajectory information, and allows real-time viewing of information including the number of aircraft carrying out the mission, affiliated units, flight trajectory, and current location. It also provides the drone flight trajectory and return video to the service management platform.

4. The air-ground-integrated intelligent forestry supervision system as claimed in claim 3 is characterized in that: During the drone inspection, the live footage captured by the drone is transmitted back in real time and displayed live through the streaming media push authentication mechanism: After the drone inspection passes the authentication, the video images recorded on site are transmitted back by streaming: the audio and video data are divided into small data blocks, each with its own identification and length information, and transmitted to the service management platform one by one in the form of streams; at the same time, the flight information is synchronously transmitted back; users with permission can pull the stream to watch the live broadcast; The live broadcast will also include the flight information including the current coordinates of the drone, including latitude and longitude, altitude, and azimuth, as well as the drone's live images, which will be displayed in real time on the service management platform. Management personnel can view the live inspection or inspection playback function to identify and confirm the fire situation and fire danger, and accurately locate the fire; The service management platform can also calculate the number of live streams, network speed, and count the number of people online, live broadcast rate, and live broadcast duration.

5. The air-ground-integrated intelligent forestry supervision system as claimed in claim 4 is characterized in that: The streaming media service authentication mechanism is: 1) A new drone account can be created in the service management platform, and a corresponding sign authentication code will be randomly generated; 2) The drone account is used to log in to the drone's operating tablet and automatically detect the real-time video images sent back by the drone; the operating tablet encapsulates the video image data in a streaming request, which contains the drone account's sign authentication code; the operating tablet backend service sends the streaming request to the streaming media server in real time; 3) After receiving the push request, the streaming media server extracts the drone account sign authentication code from the push request, queries the database based on the authentication code, and verifies whether the authentication code matches; if the authentication code matches, the streaming media server will actively accept the push request and convert the video image data into the required format for storage or transmission to the viewing client; 4) To avoid repeated streaming by the drone account, the streaming media server will check whether the drone account is already streaming; if so, it will reject the new streaming request.

6. The air-ground-integrated intelligent forestry supervision system according to claim 1, characterized in that: The service management platform can view the patrol personnel's location and historical patrol tracks in real time, analyze and process the on-site information of incidents such as text, voice, photos, and videos reported by patrol personnel; Patrol personnel and patrol targets are assigned to specific areas and mountains to implement grid management.

7. The air-ground-integrated intelligent forestry supervision system according to claim 1, characterized in that: The auxiliary decision-making service management platform combines the distribution of forest settlements, roads, water systems, fire-fighting teams and forest and grass resource distribution maps, and obtains the distribution of settlements around the disaster site, the nearest water point, the nearest fire-fighting team, the best rescue route, and disaster loss assessment through shortest path analysis, nearest facility analysis, buffer zone analysis, isolation zone setting, field of view analysis, overlay analysis, and loss assessment analysis, so as to provide decision-making assistance for the rapid and standardized formulation of decision-making plans.

8. The air-ground-integrated intelligent forestry supervision system as claimed in claim 7 is characterized in that: The shortest path analysis is performed by selecting the starting point and the end point coordinates on the map interface, or importing the coordinates of the uploaded event as the starting point and the end point coordinates, and analyzing the shortest distance between two points by the shortest distance and the road priority; The nearest facility analysis is to select a point on the map interface or import the coordinates of the uploaded event, set the search radius, and select the shortest distance and highway priority to analyze the nearest roads, rivers, settlements and firefighting force deployment; The buffer zone analysis is to select points on the map interface or import the coordinate points of uploaded events, set the buffer radius, level and interval distance, and then analyze the forest area, number of management stations, number of watchtowers and number of settlements within the buffer zone; The visual field analysis is to select the coordinates of the observation point on the map interface, set the offset of the observation point and the visual field range to analyze the visual field range that can be observed by the current observation point within the visual field range; The overlay analysis is to analyze the small class information overlaid with the graphics by drawing graphics or importing graphics from outside, selecting the overlay method of cutting, including or intersecting, and the overlay results can be statistically summarized and exported; The isolation belt is drawn manually on the map by selecting a point on the map interface or importing the coordinates of the uploaded event, setting the width of the isolation belt, and analyzing the length of the isolation belt, the area of ​​the isolation belt, the area of ​​the forest land in the isolation belt, and the distance between the isolation belt and the incident site; The loss assessment analysis is carried out by drawing the scope of the loss assessment area or the external import scope, and obtaining the resource patches within the assessment area through spatial overlay relationships. The disaster-stricken area, forest stand damage degree, and timber volume loss can be statistically calculated through the attributes of all the resource patches obtained, including area, land type, forest protection level, number of trees, stocking, and dominant tree species.

Images