An environmental inspection system based on UAV multi-sensor fusion

Abstract

The invention relates to the technical field of forestry environment inspection, and discloses an environment inspection system based on unmanned aerial vehicle (UAV) multi-sensor fusion, comprising a UAV, a sensor fusion processing module, a path planning module and an emergency module; a positioning module obtains the position information of the UAV according to GPS positioning, and a laser radar acquisition module obtains the crown characteristics to define the crown characteristic information; the sensor fusion processing module obtains whether an emergency event occurs, and if an emergency event occurs, the positioning module and the laser radar acquisition module work synchronously to obtain the location of the event; the path planning module is configured with a conventional path planning strategy and an emergency path planning strategy, and when an emergency event occurs, the emergency path planning strategy is executed to generate an emergency inspection path, and the conventional path planning strategy is executed to generate a conventional inspection path at other times; when the sensor fusion processing module obtains that an emergency event occurs, the emergency module predicts the development of a fire according to a preset prediction model and generates a scheduling strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of forestry environment inspection, and in particular to an environment inspection system based on multi-sensor fusion of unmanned aerial vehicles. Background Art

[0002] The so-called Multi-sensor Information Fusion (MSIF) is an information processing process that uses computer technology to automatically analyze and synthesize information and data from multiple sensors or multiple sources under certain criteria to complete the required decision-making and estimation. In forestry monitoring, drone inspections are a common inspection method. Using drones to conduct resource surveys on forestry resources can not only improve the speed and accuracy of resource surveys, but also obtain multi-level forestry data, thereby improving the convenience and accuracy of forestry monitoring. Summary of the invention

[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an environmental inspection system based on unmanned aerial vehicle multi-sensor fusion, which quickly processes and integrates data through a sensor fusion processing module to determine whether an emergency event has occurred, and uses the emergency module to generate a scheduling strategy to facilitate rapid response and timely handling of emergency events.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] An environmental inspection system based on UAV multi-sensor fusion, including

[0006] The drone is equipped with a positioning module, a laser radar acquisition module, an image acquisition module, an infrared thermal imaging module, and a smoke detection module. The positioning module acquires the location information of the drone according to GPS positioning, and the laser radar acquisition module acquires the crown features to define the crown feature information;

[0007] A sensor fusion processing module, wherein the sensor fusion processing module communicates with the image acquisition module, the infrared thermal imaging module, and the smoke detection module to obtain whether an emergency event occurs, wherein the emergency event includes a fire. If an emergency event occurs, the positioning module and the laser radar acquisition module work synchronously to obtain the location of the event;

[0008] A path planning module, wherein the path planning module is configured with a conventional path planning strategy and an emergency path planning strategy. When an emergency event occurs, the emergency path planning strategy is executed to generate an emergency inspection path, and the conventional path planning strategy is executed at other times to generate a conventional inspection path;

[0009] The emergency module predicts the development of the fire according to a preset prediction model and generates a dispatch strategy when the sensor fusion processing module obtains the occurrence of an emergency event.

[0010] Furthermore, the environmental inspection system includes a data center, which is equipped with an environmental database. The environmental database records inspection location information and corresponding daily inspection crown feature information. When the drone executes a regular inspection route, the positioning module and the lidar acquisition module work synchronously. The positioning module continuously acquires location information and transmits it to the data center, and the lidar acquisition module continuously acquires crown feature information and transmits it to the data center. The data center determines whether the location information and the crown feature information within the same time period conform to a continuous correspondence relationship. If so, the location information is used as the inspection location information, and the crown feature information is recorded in the environmental database as daily inspection crown feature information.

[0011] Furthermore, the steps for obtaining the location of the event are as follows:

[0012] The positioning module obtains the location where the emergency event occurs to define the preliminary positioning information.

[0013] Taking the preliminary positioning information as the center, obtaining a preliminary positioning range,

[0014] The laser radar acquisition module obtains surrounding tree crown feature information, and the surrounding tree crown feature information specifically refers to the tree crown features around the place where the emergency event occurs. The corresponding daily inspection tree crown feature information is matched in the environmental database with the preliminary positioning range as an index. The surrounding tree crown feature information is compared with the daily inspection tree crown feature information. If the similarity is higher than a preset similarity threshold, the corresponding daily inspection tree crown feature information is used as the actual tree crown feature information. According to the actual tree crown feature information, the corresponding inspection location information is matched in the environmental database as the place where the event occurred.

[0015] Furthermore, historical forestry fire data, including temperature, humidity, wind direction, wind speed, vegetation coverage, fire intensity and its spread, are used as a training set to train the prediction model, and the current temperature, current humidity, current wind direction, current wind speed, current vegetation coverage and current fire intensity are obtained and input into the prediction model to obtain the predicted fire development. The predicted fire development includes the predicted time, the corresponding predicted fire range, the predicted spread direction and the predicted spread speed.

[0016] Furthermore, the scheduling strategy includes:

[0017] Obtain the locations of surrounding firefighting points to calculate the firefighting dispatch time, which is specifically the time required for each firefighting point to respond and arrive at the location of the incident;

[0018] Obtaining and dispatching fire distribution according to the predicted fire development and generating a plurality of intervention locations, wherein the intervention locations are located in the predicted spreading direction, and the distribution density of the intervention locations is greater in an area where the predicted fire spreading speed is greater;

[0019] A plurality of dispatch information is generated according to the intervention location, and the dispatch information reflects the corresponding dispatch relationship between the surrounding fire fighting points and the intervention location.

[0020] Furthermore, the emergency path planning strategy includes:

[0021] Obtain the predicted fire development situation, generate several simulation times and corresponding simulated fire ranges at equal time intervals according to the predicted time and the corresponding predicted fire range, and generate an emergency inspection path with the current position of the UAV as the starting point, so that the UAV is located on the boundary of the corresponding simulated fire range at the simulated time.

[0022] Furthermore, the location of the incident, vegetation coverage, and fire time are recorded as historical fire data. The conventional inspection route planning strategy includes:

[0023] The time is obtained as the current season data, and a number of risk locations are generated on the simulated forestry map according to the season data and the historical fire data.

[0024] A regular inspection route is generated, where the regular inspection route covers each of the risk locations.

[0025] Furthermore, the data center is configured with a database update strategy, and the database update strategy is configured with a planned update time. When the planned update time arrives, the drone re-collects the pending crown feature information according to the preset inspection update path, and compares the pending crown feature information with the daily inspection crown feature information for similarity. If the crown similarity is lower than the preset crown similarity threshold, it searches for whether there are crown-related features. If there are related features, the pending crown feature information is written into the environmental database as the updated daily inspection crown feature information.

[0026] Furthermore, the crown feature information includes crown height features and crown width features. The steps for acquiring the crown height features include: the lidar acquisition module acquires lidar data, extracts the three-dimensional laser point cloud of the forest area, projects it and generates a corresponding point cloud image, and performs edge segmentation based on the point cloud image to obtain crown height features and crown width features.

[0027] Furthermore, the drone conducts inspections according to the emergency inspection route to obtain actual fire spread conditions, and uses the actual fire spread conditions to correct the prediction model.

[0028] Beneficial effects of the present invention:

[0029] 1. The present invention uses a sensor fusion processing module to obtain patrol data of the forest area using multiple detection modules such as a positioning module, a laser radar acquisition module, an image acquisition module, an infrared thermal imaging module, and a smoke detection module, and then determines whether an emergency event occurs. When an emergency event such as a fire is determined to occur, the path planning module is used to quickly generate an emergency patrol path, and the emergency module generates a scheduling strategy, which is convenient for quickly responding to sudden dangerous situations, improving the response and scheduling speed, and facilitating rapid control of dangerous situations;

[0030] 2. The present invention obtains the position of the drone through the positioning module, and then accurately locates the location of the incident in combination with GPS positioning and tree crown feature information, thereby avoiding the problem of GPS positioning errors under emergency conditions, facilitating emergency dispatch and improving the accuracy of the prediction of subsequent fire development. The predicted fire development is calculated and generated based on historical forestry fire data, with high accuracy, which is convenient for early prevention and control and reduces the spread of fire.

[0031] 3. The present invention facilitates the acquisition of the actual spread of the fire by setting up an emergency inspection path, and compares the actual spread with the simulated fire range to facilitate the correction of the prediction model, thereby improving the accuracy of the prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of the system architecture of the present invention;

[0033] Figure 2 It is a flow chart of the step of obtaining the location of the event in the present invention;

[0034] Figure 3 It is a schematic diagram of the process structure of the scheduling strategy;

[0035] Figure 4 It is a schematic diagram of the process structure of the emergency path planning strategy. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a component centered. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a component centered. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a component centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0039] like Figure 1 As shown, an environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles in this embodiment includes a unmanned aerial vehicle, a sensor fusion processing module, a path planning module, an emergency module and a data center. In this embodiment, the unmanned aerial vehicle and the sensor fusion processing module, the path planning module, the emergency module and the data center are connected through 5G, low-latency communication protocols or offline transmission, which effectively improves the data transmission efficiency, reduces the delay and packet loss rate, and ensures the stability and real-time performance of data transmission.

[0040] The UAV is equipped with a positioning module, a lidar acquisition module, an image acquisition module, an infrared thermal imaging module, and a smoke detection module. The positioning module obtains the location information of the UAV based on GPS positioning, and the lidar acquisition module obtains the crown features to define them as crown feature information.

[0041] The sensor fusion processing module communicates with the image acquisition module, infrared thermal imaging module, and smoke detection module to determine whether an emergency event has occurred, including fire. Fires can be determined by obtaining thermal imaging images through an infrared thermal imager, and the smoke sensor can be used to improve the accuracy of the judgment. The image acquisition module is convenient for capturing forest area image information and transmitting it back. If an emergency event occurs, the positioning module and the lidar acquisition module work synchronously to obtain the location of the event.

[0042] See also Figure 2 , the steps to obtain the location of the event are as follows:

[0043] The positioning module obtains the location of the emergency event as the definition of preliminary positioning information, obtains the preliminary positioning range with the preliminary positioning information as the center, and the laser radar acquisition module obtains the surrounding crown feature information, which is specifically the crown feature around the emergency event location. The corresponding daily inspection crown feature information is matched in the environmental database with the preliminary positioning range as the index, and the surrounding crown feature information is compared with the daily inspection crown feature information. If the similarity is higher than the preset similarity threshold, the corresponding daily inspection crown feature information is used as the actual crown feature information, and the corresponding inspection location information is matched in the environmental database according to the actual crown feature information as the location of the event. If the similarity is not higher than the preset similarity threshold, the preliminary positioning range is expanded. Considering that positioning errors are prone to occur when using GPS for positioning, which in turn affects subsequent operations, the present invention obtains the preliminary positioning information of the drone through the positioning module, and then accurately locates the location of the event in combination with the crown feature information, avoiding the problem of GPS positioning errors under emergency conditions, facilitating emergency dispatch and improving the accuracy of prediction of subsequent fire development. In addition, the setting of the preliminary positioning range reduces the scope of feature comparison. The crown feature information includes crown height features, crown width features and the associated features between the above features, which reduces the amount of data comparison and improves the processing speed.

[0044] The crown feature information includes crown height features and crown width features. The steps for obtaining the crown height features include: the laser radar acquisition module obtains the laser radar data, extracts the three-dimensional laser point cloud of the forest area, projects it and generates the corresponding point cloud image, and performs edge segmentation according to the point cloud image to obtain the crown height features and crown width features.

[0045] Furthermore, the environmental inspection system includes a data center, which is equipped with an environmental database. The environmental database records the inspection location information and the corresponding daily inspection crown feature information. When the drone executes a regular inspection route, the positioning module and the laser radar acquisition module work synchronously. The positioning module continuously obtains the location information and transmits it to the data center. The laser radar acquisition module continuously obtains the crown feature information and transmits it to the data center. The data center determines whether the location information and the crown feature information in the same period meet the continuous correspondence relationship. If they do, the location information is used as the inspection location information, and the crown feature information is recorded as the daily inspection crown feature information in the environmental database. Considering the continuity of the crown feature information, when collecting the crown features, the correlation can be used to improve the accuracy of the data.

[0046] Furthermore, the data center is configured with a database update strategy, and the database update strategy is configured with a planned update time. When the planned update time is reached, the drone re-collects the pending crown feature information according to the preset inspection update path. The database update strategy mainly takes into account that the trees in the forest area will change to a certain extent over time, such as the growth and death of trees. These factors will affect the crown feature information, so it is necessary to re-collect data at the preset planned update time. The pending crown feature information is compared with the daily inspection crown feature information for similarity. If the crown similarity is lower than the preset crown similarity threshold, it is checked whether there is a crown correlation feature. If there is a correlation feature, the pending crown feature information is written into the environmental database as the updated daily inspection crown feature information. The judgment of the crown correlation feature is mainly based on the fact that there are trees with a longer age, and their growth change rate is smaller, so there will be correlation features. It can also be that the growth change rules between similar trees are similar, and there is a high correlation between their crown feature information.

[0047] Furthermore, historical forest fire data, including temperature, humidity, wind direction, wind speed, vegetation coverage, fire intensity and its spread, are used as training sets to train a prediction model, and the current temperature, current humidity, current wind direction, current wind speed, current vegetation coverage and current fire intensity are obtained and input into the prediction model to obtain the predicted fire development. The predicted fire development includes the predicted time, the corresponding predicted fire range, the predicted spread direction and the predicted spread speed. The predicted fire development is calculated and generated based on historical forest fire data, with high accuracy, which is convenient for early prevention and control to reduce the spread of fire. The drone conducts inspections based on the emergency inspection route to obtain the actual spread of the fire, and uses the actual spread of the fire to correct the prediction model.

[0048] The path planning module is configured with conventional path planning strategies and emergency path planning strategies. When an emergency event occurs, the emergency path planning strategy is executed to generate an emergency inspection path, and the conventional path planning strategy is executed to generate a conventional inspection path at other times. When the sensor fusion processing module obtains the occurrence of an emergency event, the emergency module predicts the development of the fire according to the preset prediction model and generates a scheduling strategy.

[0049] See also Figure 3 ,Furthermore, the scheduling strategy includes:

[0050] Obtain the locations of surrounding fire stations to calculate the fire dispatch time. The fire dispatch time is the time required for each fire station to respond and arrive at the location of the incident.

[0051] Obtain and dispatch fire distribution according to the predicted fire development and generate several intervention locations. The intervention locations are located in the predicted spread direction, and the distribution density of the intervention locations is greater in the area where the predicted fire spread speed is greater;

[0052] A number of dispatch information are generated according to the intervention location, and the dispatch information reflects the corresponding dispatch relationship between the surrounding fire-fighting points and the intervention location.

[0053] See also Figure 4 ,Furthermore, the emergency path planning strategy includes:

[0054] Obtain the predicted fire development situation, generate several simulation times and corresponding simulated fire ranges at equal time intervals according to the predicted time and the corresponding predicted fire range, and generate an emergency inspection path with the current position of the drone as the starting point, so that the drone is located on the boundary of the corresponding simulated fire range at the simulated time.

[0055] Furthermore, the location of the incident, vegetation coverage, and fire time are recorded as historical fire data. The conventional inspection route planning strategy includes

[0056] The time is taken as the current season data, and several risk locations are generated on the simulated forestry map according to the seasonal data and historical fire data.

[0057] Generate regular inspection routes that cover various risk locations.

[0058] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles, characterized by: include The drone is equipped with a positioning module, a laser radar acquisition module, an image acquisition module, an infrared thermal imaging module, and a smoke detection module. The positioning module acquires the location information of the drone according to GPS positioning, and the laser radar acquisition module acquires the crown features to define the crown feature information; A sensor fusion processing module, wherein the sensor fusion processing module communicates with the image acquisition module, the infrared thermal imaging module, and the smoke detection module to obtain whether an emergency event occurs, wherein the emergency event includes a fire. If an emergency event occurs, the positioning module and the laser radar acquisition module work synchronously to obtain the location of the event; A path planning module, wherein the path planning module is configured with a conventional path planning strategy and an emergency path planning strategy. When an emergency event occurs, the emergency path planning strategy is executed to generate an emergency inspection path, and the conventional path planning strategy is executed at other times to generate a conventional inspection path; An emergency module, when the sensor fusion processing module obtains the occurrence of an emergency event, the emergency module predicts the development of the fire according to a preset prediction model and generates a dispatch strategy; The environmental inspection system includes a data center, the data center is configured with an environmental database, the environmental database records inspection location information and corresponding daily inspection crown feature information, when the drone executes a regular inspection route, the positioning module and the laser radar acquisition module work synchronously, the positioning module continuously obtains location information and transmits it to the data center, the laser radar acquisition module continuously obtains crown feature information and transmits it to the data center, the data center determines whether the location information and the crown feature information in the same time period conform to a continuous correspondence relationship, if so, the location information is used as the inspection location information, and the crown feature information is recorded in the environmental database as daily inspection crown feature information; The steps for obtaining the location of the event are as follows: The positioning module obtains the location where the emergency event occurs to define the preliminary positioning information. Taking the preliminary positioning information as the center, obtaining a preliminary positioning range, The laser radar acquisition module obtains surrounding tree crown feature information, the surrounding tree crown feature information specifically refers to tree crown features around the location where the emergency event occurs, matches the corresponding daily inspection tree crown feature information in the environmental database with the preliminary positioning range as an index, performs feature comparison between the surrounding tree crown feature information and the daily inspection tree crown feature information, and if the similarity is higher than a preset similarity threshold, uses the corresponding daily inspection tree crown feature information as actual tree crown feature information, and matches the corresponding inspection location information in the environmental database as the location where the event occurred based on the actual tree crown feature information; The prediction model is trained by using historical forest fire data, including temperature, humidity, wind direction, wind speed, vegetation coverage, fire intensity and its spread, as a training set, obtaining the current temperature, current humidity, current wind direction, current wind speed, current vegetation coverage and current fire intensity and inputting them into the prediction model to obtain the predicted fire development, wherein the predicted fire development includes the predicted time, the corresponding predicted fire range, the predicted spread direction and the predicted spread speed The generation step of the scheduling strategy includes: Obtain the locations of surrounding firefighting points to calculate the firefighting dispatch time, which is specifically the time required for each firefighting point to respond and arrive at the location of the incident; Obtaining and dispatching fire distribution according to the predicted fire development and generating a plurality of intervention locations, wherein the intervention locations are located in the predicted spreading direction, and the distribution density of the intervention locations is greater in an area where the predicted fire spreading speed is greater; Generate a number of dispatch information according to the intervention location, wherein the dispatch information reflects the corresponding dispatch relationship between the surrounding firefighting points and the intervention location; The emergency path planning strategy includes: Obtain the predicted fire development situation, generate several simulation times and corresponding simulated fire ranges at equal time intervals according to the predicted time and the corresponding predicted fire range, and generate an emergency inspection path with the current position of the UAV as the starting point, so that the UAV is located on the boundary of the corresponding simulated fire range at the simulated time.

2. The environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles according to claim 1 is characterized by: The location of the incident, vegetation coverage, and fire time are recorded as historical fire data. The conventional inspection route planning strategy includes: The time is obtained as the current season data, and a number of risk locations are generated on the simulated forestry map according to the season data and the historical fire data. A regular inspection route is generated, where the regular inspection route covers each of the risk locations.

3. The environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles according to claim 1 is characterized by: The data center is configured with a database update strategy, and the database update strategy is configured with a planned update time. When the planned update time arrives, the drone re-collects the pending tree crown feature information according to the preset inspection update path, and compares the pending tree crown feature information with the daily inspection tree crown feature information for similarity. If the similarity is lower than the preset similarity threshold, it checks whether there are crown-related features. If there are related features, the pending tree crown feature information is written into the environmental database as the updated daily inspection tree crown feature information.

4. The environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles according to claim 1 is characterized by: The crown feature information includes crown height features and crown width features. The steps for acquiring the crown height features include: the laser radar acquisition module acquires laser radar data, extracts the three-dimensional laser point cloud of the forest area, projects it and generates a corresponding point cloud image, and performs edge segmentation according to the point cloud image to obtain the crown height features and crown width features.

5. The environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles according to claim 1 is characterized by: The drone conducts inspections according to the emergency inspection route to obtain actual fire spread conditions, and uses the actual fire spread conditions to correct the prediction model.

Images