Emergency safety monitoring method and device based on drone

Abstract

The present application discloses an emergency safety monitoring method and device based on drones, which relates to the field of drone monitoring technology. The method includes: receiving the load data collected by each crew member in the drone group when performing a cruise mission on a water space on a preset cruise path, fusing the image data, thermal imaging data, positioning data and radar data to obtain a three-dimensional dynamic model of the water space; identifying the three-dimensional dynamic model, determining the identification object of the water space and the motion state of the identification object; when the motion state is an emergency intervention state, switching the current cruise mission to an emergency mission, and when the remaining energy cannot meet the requirements of the emergency mission, generating an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission. Through the above method, real-time monitoring of the target water area based on drones is realized, accidents can be accurately and timely predicted, sudden accidents can be monitored, and emergency missions can be quickly responded to.

Description

Technical Field

[0001] The present application relates to the field of drone monitoring technology, and in particular to a drone-based emergency safety monitoring method and device. Background Art

[0002] Shipping is an important part of the transportation system. With the development of commercial trade, the number of ships on shipping lines is increasing. In the process of shipping, ship collisions, ship strandings, ship sinkings, etc. often occur. At present, in order to supervise the waters, radar (RADAR, radio detection and ranging), vessel traffic management system (VTS, vessel traffic service), closed-circuit television (CCTV, Closed Circuit Television cinema), ship automatic identification system (AIS, Automatic Identification System) and other systems are usually used to monitor the waters. However, the above schemes can only monitor large ships and those with or with identification equipment installed or turned on. It is impossible to accurately and timely understand the driving status of ships on the waters. At the same time, due to the limited observation distance, it can only monitor simple water environments. For complex waters, patrol ships are also needed to cruise. It is impossible to quickly and accurately monitor complex waters in real time and deal with emergency safety matters in a timely manner.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide an emergency safety monitoring method and device based on drones, aiming to solve the technical problem that the existing technology is unable to monitor the driving status of ships in complex waters and handle emergencies.

[0005] To achieve the above objectives, the present application provides an emergency safety monitoring method based on a drone, the method comprising:

[0006] Receiving load data collected by each crew member of the UAV group when performing a cruise mission on a water space on a preset cruise path, the load data including image data, thermal imaging data, positioning data and radar data;

[0007] Fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space;

[0008] Identify the three-dimensional dynamic model to determine the identification object in the water space and the motion state of the identification object;

[0009] When the motion state is an emergency intervention state, the current cruise mission is switched to an emergency mission. When the remaining energy cannot meet the requirements of the emergency mission, an emergency response area and an emergency response signal are generated according to the motion state, so that the emergency drone group performs the emergency mission.

[0010] In one embodiment, the step of fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space includes:

[0011] Determine characteristic image data and characteristic thermal imaging data in the image data and the thermal imaging data respectively, wherein the characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position;

[0012] Determine the pixel deviation of the characteristic image data and the characteristic thermal imaging data, use the pixel deviation as correction data, perform position offset correction on the image data and the thermal imaging data, and obtain water area visual data;

[0013] Determine the motion properties of the object in the water space according to the positioning data and the radar data to obtain water space motion data;

[0014] Modeling is performed based on the water space to obtain an initial water model of the water space, and the water visual data and the water motion data are added to the initial water model to obtain a three-dimensional dynamic model.

[0015] In one embodiment, the step of determining the pixel deviation of the characteristic image data and the characteristic thermal imaging data, using the pixel deviation as correction data, performing position offset correction on the image data and the thermal imaging data, and obtaining the water area visual data comprises:

[0016] Determining the marking positions of the characteristic image data and the characteristic thermal imaging data;

[0017] Respectively projecting the characteristic image data and the characteristic thermal imaging data onto a reference plane to obtain a first projection position and a second projection position;

[0018] Determine the pixel coordinates of the first projection position and the second projection position respectively, and obtain a first pixel coordinate and a second pixel coordinate;

[0019] Taking the first pixel coordinate as a reference point, determining a deviation direction and a deviation distance between the second pixel coordinate and the first pixel coordinate;

[0020] Obtaining correction data according to the deviation direction and the deviation distance;

[0021] The second pixel coordinates are position-corrected based on the correction data so that the first pixel coordinates and the second pixel coordinates coincide with each other, thereby obtaining water area visual data.

[0022] In one embodiment, the step of identifying the three-dimensional dynamic model and determining the identification object in the water space and the motion state of the identification object includes:

[0023] Screening a relatively high temperature area in the thermal imaging data, determining an overlap position between the relatively high temperature area and the image data, performing target recognition on the overlap position, and determining an identification object of the water space;

[0024] Determine the position of the identified object in the three-dimensional dynamic model and the positioning data of the current drone;

[0025] Recording a first change trend of the positioning data and a second change trend of the radar data, and determining a movement direction and a movement speed of the identified object according to the first change trend and the second change trend;

[0026] The motion state of the identified object is obtained according to the motion direction and the motion speed.

[0027] In one embodiment, when the motion state is an emergency intervention state, switching the current cruise mission to an emergency mission, and when the remaining energy cannot meet the requirements of the emergency mission, generating an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission, the steps include:

[0028] When the motion state is an emergency intervention state, the current cruise mission is switched to an emergency mission, and the remaining energy is determined to predict the minimum emergency power value;

[0029] When the remaining energy is less than the minimum emergency power value, the remaining energy cannot meet the emergency task, and the motion trajectory corresponding to the identification is predicted according to the motion state;

[0030] An emergency area is generated according to the motion trajectory, and an emergency response signal is generated according to the emergency area and the motion trajectory, so that the emergency drone group performs the emergency task.

[0031] In one embodiment, before the step of receiving the load data collected by each crew member in the unmanned aerial vehicle group when performing a cruising mission on a water space on a preset cruising path, the step further includes:

[0032] Determine the water area division and airspace division of the water area space, and generate an electronic fence according to the airspace division and the no-fly zone of the water area space;

[0033] Determine the concerned location in the water zone, and generate an initial cruise path based on the emergency monitoring point and the concerned location based on the energy priority principle;

[0034] The initial cruising path is constrained according to the electronic fence to obtain a preset cruising path.

[0035] In one embodiment, the drone-based emergency safety monitoring method further includes:

[0036] Determine the communication address of the target UAV, and generate a communication code according to the communication address;

[0037] Generate a video return signal based on the communication code and the video return instruction;

[0038] Sending a video feedback signal to a crew member of the drone group, and receiving a drone view image fed back by the crew member of the drone group based on the video feedback signal;

[0039] The drone's perspective image is projected onto the emergency central control screen.

[0040] In one embodiment, the drone-based emergency safety monitoring method further includes:

[0041] Determine the communication address of the target UAV, and generate a communication code according to the communication address;

[0042] Sending a manual intervention signal to the target drone based on the communication code, and receiving response information of a receipt after the target drone successfully verifies the manual intervention signal;

[0043] After receiving the response information, establishing a control channel based on the communication code;

[0044] The target UAV is manually controlled based on data transmission with the target UAV through the control channel.

[0045] In one embodiment, after receiving the response information, the step of establishing a control channel based on the communication code includes:

[0046] After receiving the response information, performing hash mapping with the communication code to obtain communication verification information;

[0047] Sending the communication verification information to the target drone, so that the target drone monitors the communication verification information;

[0048] The communication verification information is additionally compiled with the transmission data to obtain exclusive communication information;

[0049] A control channel is established according to the dedicated communication information.

[0050] In addition, to achieve the above purpose, the present application also proposes an emergency safety monitoring device based on a drone, and the emergency safety monitoring device based on a drone includes:

[0051] A data perception module is used to receive the load data collected by each crew member of the UAV group when performing a cruise mission on a water space on a preset cruise path, wherein the load data includes image data, thermal imaging data, positioning data and radar data;

[0052] A spatial modeling module, used for fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space;

[0053] A motion fitting module, used to identify the three-dimensional dynamic model, determine the identified object in the water space and the motion state of the identified object;

[0054] The emergency processing module is used to switch the current cruise mission to an emergency mission when the motion state is an emergency intervention state, and when the remaining energy cannot meet the requirements of the emergency mission, generate an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission.

[0055] In addition, to achieve the above-mentioned purpose, the present application also proposes a drone-based emergency safety monitoring device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the drone-based emergency safety monitoring method as described above.

[0056] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the emergency safety monitoring method based on the drone as described above are implemented.

[0057] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the emergency safety monitoring method based on a drone as described above.

[0058] The present application provides an emergency safety monitoring method based on drones, which receives the load data collected by each crew member of the drone group when performing a cruise mission on a water space on a preset cruise path, fuses the image data, thermal imaging data, positioning data and radar data, and obtains a three-dimensional dynamic model of the water space; identifies the three-dimensional dynamic model, determines the identification object of the water space and the motion state of the identification object; when the motion state is an emergency intervention state, switches the current cruise mission to an emergency mission, and when the remaining energy cannot meet the requirements of the emergency mission, generates an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission. Through the above method, real-time monitoring of the target water area based on drones is realized, which can accurately and timely predict accidents and monitor sudden accidents, and quickly respond to emergency tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 This is a flow chart of Example 1 of the emergency safety monitoring method based on drones of this application;

[0062] Figure 2 A schematic diagram of an initial cruising path of an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0063] Figure 3 A schematic diagram of the emergency radius of a monitoring point according to an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0064] Figure 4 A simplified schematic diagram of pixel deviation of an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0065] Figure 5 A schematic diagram of image data and thermal imaging data of an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0066] Figure 6 A schematic diagram of image data and thermal imaging data of an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0067] Figure 7This is a schematic diagram of motion recognition according to an embodiment of an emergency safety monitoring method based on a drone of the present application;

[0068] Figure 8 This is a schematic diagram of the module structure of an emergency safety monitoring device based on a drone according to an embodiment of the present application;

[0069] Fig. 9 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the emergency safety monitoring method based on a drone in an embodiment of the present application.

[0070] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0072] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0073] The main solution of the embodiment of the present application is: receiving the load data collected by each crew member in the unmanned aerial vehicle group when performing a cruise mission on a water space on a preset cruise path, wherein the load data includes image data, thermal imaging data, positioning data and radar data;

[0074] Fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space;

[0075] Identify the three-dimensional dynamic model to determine the identification object in the water space and the motion state of the identification object;

[0076] When the motion state is an emergency intervention state, the current cruise mission is switched to an emergency mission. When the remaining energy cannot meet the requirements of the emergency mission, an emergency response area and an emergency response signal are generated according to the motion state, so that the emergency drone group performs the emergency mission.

[0077] At present, water areas are monitored through radar (RADAR, radio detection and ranging), vessel traffic management system (VTS, vessel traffic service), closed-circuit television (CCTV, Closed Circuit Television cinema), ship automatic identification system (AIS, Automatic Identification System), etc. However, the above solutions can only monitor large ships and those with identification equipment installed or turned on. It is impossible to accurately and timely understand the driving status of ships in the waters. At the same time, due to the limitation of observation distance, it can only monitor simple water environments. For complex waters, patrol ships are still needed to cruise. It is impossible to quickly and accurately monitor complex waters in real time and deal with emergency safety issues in a timely manner.

[0078] The present application provides a solution, which receives the load data collected by each crew member of the drone group when performing a cruise mission on a water space on a preset cruise path, fuses the image data, thermal imaging data, positioning data and radar data, and obtains a three-dimensional dynamic model of the water space; identifies the three-dimensional dynamic model, determines the identification object of the water space and the motion state of the identification object; when the motion state is an emergency intervention state, switches the current cruise mission to an emergency mission, and when the remaining energy cannot meet the needs of the emergency mission, generates an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission. Through the above method, real-time monitoring of the target water area based on drones is realized, which can accurately and timely predict accidents and monitor sudden accidents, and quickly respond to emergency tasks.

[0079] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions, an emergency safety monitoring device based on a drone, etc., and this embodiment does not specifically limit this. The following takes the emergency safety monitoring device based on a drone as an example to illustrate this embodiment and the following embodiments.

[0080] The present application embodiment provides an emergency safety monitoring method based on a drone, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the emergency safety monitoring method based on drone of the present application.

[0081] In this embodiment, the emergency safety monitoring method based on a drone includes steps S10 to S40:

[0082] Step S10, receiving the load data collected by each crew member in the unmanned aerial vehicle group when performing a cruise mission in the water space on a preset cruise path, wherein the load data includes image data, thermal imaging data, positioning data and radar data.

[0083] It should be noted that the drone group is a formation composed of multiple drones of different types, where the drone group is deployed in emergency monitoring points around the waters. The drones in the drone group can be in a collaborative working relationship. Each drone includes at least a camera, a ranging radar, an infrared sensor, a positioning device and other payload equipment, which are used to collect visible light image data, thermal imaging data, the drone's positioning data and the drone's ranging radar data.

[0084] It should be understood that the cruise path is the flight path of the UAV when performing an inspection mission in the water area, wherein the cruise path is three-dimensional data, which includes the flight altitude of the UAV in addition to the flight route.

[0085] In the specific implementation, each crew member in the drone group can fly according to the preset cruising path and turn on the drone's payload equipment to monitor the water space below. During the cruising process, the drone can obtain visible light image data and thermal imaging data in the water in real time, and transmit the visible light image data and thermal imaging data to the background database as image data. At the same time, the positioning data sent back by the drone in real time is also fed back to the drone's positioning data to determine the drone's current location and the drone's position in real time. At the same time, the drone can detect the water space through the ranging radar, determine the obstacles in the water space, including various types of ships and other objects, and determine the distance between the drone and the ship.

[0086] In the drone formation, drones of the same type performing the same mission can be divided into patrol teams, and one of the drones can be selected as the patrol team leader, and the other drones can be selected as patrol team members. In order to facilitate the execution of drone tasks, the patrol team members can upload the collected payload data to the patrol team leader, and the patrol team leader will forward the payload data collected by each patrol team member.

[0087] Alternatively, the emergency safety monitoring equipment can establish data connection channels with each drone to receive various types of payload data collected by each drone. Each connection channel can transmit data through an independent data channel or in the form of broadcast and encrypted communication.

[0088] In a feasible implementation manner, before the step of receiving the load data collected by each crew member in the drone group when performing a cruise mission on the water space on a preset cruise path, it also includes:

[0089] Determine the water area division and airspace division of the water area space, and generate an electronic fence according to the airspace division and the no-fly zone of the water area space;

[0090] Determine the concerned location in the water zone, and generate an initial cruise path based on the emergency monitoring point and the concerned location based on the energy priority principle;

[0091] The initial cruising path is constrained according to the electronic fence to obtain a preset cruising path.

[0092] It should be noted that water zoning refers to the result of dividing waters according to geographical or economic factors, where water zoning can at least include exclusive economic zone, contiguous zone, territorial waters, territorial baseline zone, port channel, anchorage waters, wharf waters, inland waters, etc. In specific practical applications, bridge areas, ferry areas, construction areas, anchorage areas, warning areas, and shallow water areas can be used as regulatory objects, and specific zoning can be adapted to actual conditions.

[0093] It should be understood that airspace zoning can be understood as a three-dimensional space, and a certain amount of flight space can be applied for drones according to local aircraft management regulations. Among them, there may be no-fly zones in the water space, and airspace zoning naturally does not include no-fly zones. The airspace zoning defines the maximum range for the plane flight of drones and also limits the maximum flight altitude for drones.

[0094] It can be understood that the electronic fence is a virtual fence. The electronic fence is used to limit the flight area of ​​the drone. In the specific implementation, the longitude, latitude and altitude information can be used as the boundary line of the electronic fence to limit the area enclosed by the electronic fence and the current positioning data of the drone.

[0095] In the specific implementation, the water area can be divided into multiple areas according to the geographical information and hydrological characteristics of the current water monitoring area, and the air area can be divided according to aviation management regulations, flight safety considerations, etc., to ensure that the drone operates at a legal and safe height and range. The water space can be used as the initial fence range, and the boundary line of the water space can determine the boundary information of longitude and latitude. The initial fence range corresponding to the water space is determined by the longitude and latitude of the boundary, and the initial fence range is added with height information to determine the initial air fence corresponding to the initial fence range. The initial air fence is merged with the no-fly zone of the water space, and the area corresponding to the no-fly zone in the initial air fence is removed to obtain the air area where the drone can fly. Determine the focus position in the water partition, where the focus position is each position in the water partition, such as accident-prone areas. Considering the endurance and efficiency of the drone, path planning should minimize energy consumption. At the same time, it is also necessary to ensure that all focus positions can be effectively covered and that emergency monitoring points can be quickly reached. Based on the principle of energy priority, the initial cruise path can be generated based on emergency monitoring points and focus positions. Reference Figure 2 , Figure 2 Schematic diagram of the initial cruising path. A is a one-way cruising along the main channel, B is a two-way cruising along the main channel, C is a multi-directional cruising along the main channel, D is a multi-directional cruising along the main channel, E is a lateral cruising perpendicular to the main channel, and F is a mixed cruising. Figure 3 The schematic diagram of the emergency radius of the monitoring point is shown in Figure 1. The initial cruise path is adjusted in combination with the electronic fence, and the initial cruise path is constrained to ensure that the drone always flies within the allowed area, and the preset cruise path is obtained.

[0096] When determining the cruise path, the cruise cost needs to be taken into account to improve the cruise efficiency. Therefore, when ensuring the cruise cost, the total cruise distance of the drone can be the shortest as the target declaration process cruise path. According to the formula:

[0097]

[0098] Among them, S represents the total distance of drone cruising, M is the total number of drones cruising in the water space, Z represents the number of drones participating in the water space cruising, and N represents the number of monitoring points. It is a decision variable with a value of 1 or 0, where 1 means the cruise path is feasible and 0 means the cruise path is not feasible. Represents the distance between the i-th cruising area and the j-th cruising area.

[0099] When making constraints, you can:

[0100]

[0101] It means that each group of drones approaches the cruising point only once.

[0102]

[0103] It means that each group of drones leaves the cruising point only once.

[0104]

[0105] It means that every group of task points is valid if all the tasks are traversed through this loop, where: and Represents the sequence of areas to be cruised.

[0106]

[0107] represents the maximum range constraint of a single UAV, k is the cruise coefficient, and its value range is (0, 1], Represents the maximum cruising time. Represents the average cruising speed.

[0108]

[0109] It represents the range constraints among multiple UAVs, and uses standard deviation to coordinate and optimize the range of multiple UAVs.

[0110] Step S20, fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space.

[0111] It should be noted that the three-dimensional dynamic model is able to model the space above the water space, which may include water information and various objects in the airspace, such as ships, drones, etc.

[0112] In a specific implementation, the image data is the image information collected by the visible light camera, and the thermal imaging data is the image formed by the infrared data in the detection area collected by the infrared sensor, reflecting the thermal imaging data of the detection area. The image data and the thermal imaging data are fused and processed to obtain image data containing temperature information, and the image data is used as the environmental information of the three-dimensional dynamic model to describe the current water environment. The positioning data is the current location of the drone, which also includes the altitude information of the drone. The radar data is the radar ranging information of the drone for detectable objects in the environment, which is used to describe the distance information between the drone and other objects. The environmental information, positioning data and radar data are fused to obtain a three-dimensional dynamic model of the water space, which can reflect the real-time position of each object in the water space in real time.

[0113] In a feasible implementation manner, the step of fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space includes:

[0114] Determine characteristic image data and characteristic thermal imaging data in the image data and the thermal imaging data respectively, wherein the characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position;

[0115] Determine the pixel deviation of the characteristic image data and the characteristic thermal imaging data, use the pixel deviation as correction data, perform position offset correction on the image data and the thermal imaging data, and obtain water area visual data;

[0116] Determine the motion properties of the object in the water space according to the positioning data and the radar data to obtain water space motion data;

[0117] Modeling is performed based on the water space to obtain an initial water model of the water space, and the water visual data and the water motion data are added to the initial water model to obtain a three-dimensional dynamic model.

[0118] It should be noted that since the camera for collecting image data and the sensor for collecting thermal imaging data are not installed at the same position, there will be a certain distance deviation between the image data and thermal imaging data collected in the same direction. The distance deviation is positively correlated with the physical deviation between the camera and the sensor, and the imaging magnification, and the pixel deviation refers to the pixel deviation value in the image data and thermal imaging data. In this embodiment, the image data can be used as a reference, the upper side of the pixel in the image data can be used as the positive direction, and the right side of the pixel in the image data can be used as the positive direction.

[0119] In a specific implementation, the characteristic image data and characteristic thermal imaging feature data in the image data and the thermal imaging data are determined respectively. The characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position. The characteristic image data and the characteristic thermal imaging data are forward projected to determine the projection positions A and B of the characteristic points. Figure 4 , Figure 4 Calculate the pixel deviation of two projection positions A and B. . Determine the marking position of the characteristic image data and the characteristic thermal imaging data; project the characteristic image data and the characteristic thermal imaging data to the reference plane respectively to obtain the first projection position and the second projection position; determine the pixel coordinates of the first projection position and the second projection position respectively to obtain the first pixel coordinate and the second pixel coordinate; take the first pixel coordinate as a reference point to determine the deviation direction and deviation distance between the second pixel coordinate and the first pixel coordinate; obtain correction data according to the deviation direction and deviation distance; perform position correction on the second pixel coordinate based on the correction data to make the first pixel coordinate and the second pixel coordinate coincide, and obtain water area visual data. Determine the current pixel deviation as the correction data, perform position offset correction on the image data and the thermal imaging data, and the correction data is , the water area visual data is obtained, and the positions of projection position A and projection position B in the water area visual data coincide, that is, the feature data in the image data and thermal imaging data are completely integrated. Then, according to the positioning data and radar data, the motion properties of objects in the water area space can be determined, including the direction and speed of motion. Then, the water area space is modeled to obtain the initial water area model of the water area space, and the water area visual data and water area motion data are added to the initial water area model to obtain a three-dimensional dynamic model.

[0120] In addition, refer to Figure 5 , Figure 5 The figure is a schematic diagram of image data and thermal imaging data. When the water environment is a visible environment, an image with image data and thermal imaging data can be obtained, where the left side is image data and the right side is thermal imaging data. When both the image data and the thermal imaging data can identify the characteristics of the object, the two feature data can be fused. Figure 6 When the schematic diagram of image data and thermal imaging data is shown, the left side is the image data in foggy weather, and the right side is the thermal imaging data, so the thermal imaging data can be used as the feature data.

[0121] Step S30, identifying the three-dimensional dynamic model to determine the identified object in the water space and the motion state of the identified object.

[0122] It should be noted that the identification object is a ship in the water space or a person on the water surface, and the motion state of the identification object includes a moving state and a stationary state. In addition, the motion state includes a moving direction and a moving speed.

[0123] In a specific implementation, the three-dimensional dynamic model is identified, and the identification object is determined mainly by thermal imaging data and supplemented by image data. It can be known that a ship will generate heat when it is running, while the temperature of the water surface is relatively low. Therefore, the temperature difference between the ship and the water surface is large, so most ships can be identified by thermal imaging data. Or when there is a collision between ships, the crew falling into the water can also be identified. Therefore, the heat source in the three-dimensional dynamic model can be detected, the heat source can be determined as the identification object, and the identification object can be continuously monitored. The relatively high temperature area in the thermal imaging data is screened, and the overlapping position of the relatively high temperature area and the image data is determined. The target is identified at the overlapping position to determine the identification object in the water space; the position of the identification object in the three-dimensional dynamic model and the positioning data of the current drone are determined; the first change trend of the positioning data and the second change trend of the radar data are recorded, and the movement direction and movement speed of the identification object are determined according to the first change trend and the second change trend; the movement state of the identification object is obtained according to the movement direction and the movement speed. Reference Figure 7 , Figure 7 The figure is a motion recognition diagram. The figure shows the position of the UAV and the position of the ship at two moments. The flight speed of the UAV can be determined according to the payload data of the UAV. The UAV can determine the radar deflection angle at two moments and the distance between the UAV and the ship according to the radar data at two moments. The motion state of the ship can be determined by the deflection angle and the distance between the UAV and the ship.

[0124] Step S40, when the motion state is an emergency intervention state, switch the current cruise mission to an emergency mission, and when the remaining energy cannot meet the requirements of the emergency mission, generate an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission.

[0125] It should be noted that the emergency intervention state refers to when a ship collision occurs, or when a ship collision is predicted, or when a person falls into the water, etc., it can be determined as an emergency intervention state. Emergency tasks include task content and task location, as well as the degree of urgency.

[0126] When the motion state is the emergency intervention state, the current cruise mission can be switched to the emergency mission. When the remaining energy can meet the current emergency mission requirements, the current emergency mission can be executed. If the remaining energy cannot meet the mission requirements of the emergency mission, the emergency mission can be predicted, and the prediction scale is determined by the distance between the takeover drone and the emergency mission location. At the same time, when it is determined that the motion state is the emergency intervention state, the current cruise mission is switched to the emergency mission, and the remaining power is determined, and the minimum emergency power value is predicted. When the remaining power is less than the minimum emergency power value, the remaining power cannot meet the emergency mission. The corresponding motion trajectory is identified according to the motion state prediction, and an emergency area is generated according to the motion trajectory. An emergency response signal is generated according to the emergency area and the motion trajectory, so that the emergency drone group performs the emergency mission.

[0127] In a feasible implementation manner, the drone-based emergency safety monitoring method further includes:

[0128] Determine the communication address of the target UAV, and generate a communication code according to the communication address;

[0129] Generate a video return signal based on the communication code and the video return instruction;

[0130] Sending a video feedback signal to a crew member of the drone group, and receiving a drone view image fed back by the crew member of the drone group based on the video feedback signal;

[0131] The drone's perspective image is projected onto the emergency central control screen.

[0132] In a specific implementation, the drone emergency safety monitoring method can also allow the selection of any one of the cruising drones to obtain its video image. At this time, the communication address of the target drone can be determined, and a communication code can be generated based on the communication address. The communication code is specific and realizes the safe transmission of the returned video data. After obtaining the communication code, the video return signal and the communication code can be combined to generate a video return signal, which is sent to the target drone, and the crew members receiving the target drone forward the video signal based on the video return signal, feedback the drone's perspective image, and project the drone's perspective image to the emergency central control screen.

[0133] In a feasible implementation manner, the drone-based emergency safety monitoring method further includes:

[0134] Determine the communication address of the target UAV, and generate a communication code according to the communication address;

[0135] Sending a manual intervention signal to the target drone based on the communication code, and receiving response information of a receipt after the target drone successfully verifies the manual intervention signal;

[0136] After receiving the response information, establishing a control channel based on the communication code;

[0137] The target UAV is manually controlled based on data transmission with the target UAV through the control channel.

[0138] In a specific implementation, when it is necessary to manually take over a drone, the communication address of the target drone can be determined, a communication code can be generated, and since a takeover operation needs to be performed, a takeover request needs to be sent to the drone, and manual control can be performed only after the drone is successfully verified. At this time, a manual intervention signal can be sent to the target drone according to the communication code, wherein the manual intervention signal is a signal that needs to be verified, and after the target drone successfully verifies the manual intervention signal, a response message of a receipt is received, and after receiving the response message, the communication code is hash mapped to obtain communication verification information, and the communication verification information is sent to the target drone, so that the target drone monitors the communication verification information, and the communication verification information is additionally compiled with the transmission data to obtain exclusive communication information, and a control channel is established according to the exclusive communication information, and then data is transmitted with the target drone based on the control channel, and the target drone is manually controlled.

[0139] This embodiment provides an emergency safety monitoring method based on drones, which receives the load data collected by each crew member of the drone group when performing a cruise mission on a water space on a preset cruise path, fuses the image data, thermal imaging data, positioning data and radar data, and obtains a three-dimensional dynamic model of the water space; identifies the three-dimensional dynamic model, determines the identification object of the water space and the motion state of the identification object; when the motion state is an emergency intervention state, switches the current cruise mission to an emergency mission, and when the remaining energy cannot meet the emergency mission requirements, generates an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission. Through the above method, real-time monitoring of the target water area based on drones is realized, which can accurately and timely predict accidents and monitor sudden accidents, and quickly respond to emergency tasks.

[0140] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the emergency safety monitoring method based on drones of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0141] This application also provides an emergency safety monitoring device based on drones, please refer to Figure 8 , the emergency safety monitoring devices based on drones include:

[0142] The data sensing module 10 is used to receive the load data collected by each crew member of the unmanned aerial vehicle group when performing a cruise mission on a water space on a preset cruise path, wherein the load data includes image data, thermal imaging data, positioning data and radar data;

[0143] A space modeling module 20, for fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space;

[0144] A motion fitting module 30 is used to identify the three-dimensional dynamic model, determine the identified object in the water space and the motion state of the identified object;

[0145] The emergency processing module 40 is used to switch the current cruise mission to an emergency mission when the motion state is an emergency intervention state, and when the remaining energy cannot meet the requirements of the emergency mission, generate an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission.

[0146] In a feasible implementation manner, the spatial modeling module 20 is also used to respectively determine the characteristic image data and the characteristic thermal imaging data in the image data and the thermal imaging data, wherein the characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position; determine the pixel deviation of the characteristic image data and the characteristic thermal imaging data, and use the pixel deviation as correction data to perform camera offset correction on the image data and the thermal imaging data to obtain water area visual data; determine the motion properties of objects in the water area space according to the positioning data and the radar data to obtain water area motion data; perform modeling based on the water area space to obtain an initial water area model of the water area space, and add the water area visual data and the water area motion data to the initial water area model to obtain a three-dimensional dynamic model.

[0147] In a feasible implementation manner, the spatial modeling module 20 is also used to determine the marking positions of the characteristic image data and the characteristic thermal imaging data; project the characteristic image data and the characteristic thermal imaging data onto a reference plane respectively to obtain a first projection position and a second projection position; determine the pixel coordinates of the first projection position and the second projection position respectively to obtain a first pixel coordinate and a second pixel coordinate; use the first pixel coordinate as a reference point to determine a deviation direction and a deviation distance between the second pixel coordinate and the first pixel coordinate; obtain correction data according to the deviation direction and the deviation distance; perform position correction on the second pixel coordinate based on the correction data so that the first pixel coordinate and the second pixel coordinate coincide, and obtain water area visual data.

[0148] In a feasible implementation, the motion fitting module 30 is also used to screen the relatively high temperature area in the thermal imaging data, determine the overlapping position of the relatively high temperature area and the image data, perform target recognition on the overlapping position, and determine the identification object of the water space; determine the position of the identification object in the three-dimensional dynamic model and the positioning data of the current UAV; record the first change trend of the positioning data and the second change trend of the radar data, and determine the movement direction and movement speed of the identification object according to the first change trend and the second change trend; obtain the movement state of the identification object according to the movement direction and the movement speed.

[0149] In a feasible implementation, the emergency processing module 40 is also used to switch the current cruise mission to an emergency mission when the motion state is an emergency intervention state, determine the remaining energy, and predict the minimum emergency power value; when the remaining energy is less than the minimum emergency power value, the remaining energy cannot meet the emergency task, and the motion trajectory corresponding to the identification is predicted according to the motion state; an emergency area is generated according to the motion trajectory, and an emergency response signal is generated according to the emergency area and the motion trajectory, so that the emergency drone group performs the emergency mission.

[0150] In a feasible implementation, the data perception module 10 is also used to determine the water area partitions and airspace partitions of the water space, and generate electronic fences based on the airspace partitions and the no-fly zones of the water space; determine the focus locations in the water area partitions, and based on the energy priority principle, generate an initial cruise path based on the emergency monitoring points and the focus locations; constrain the initial cruise path based on the electronic fences to obtain a preset cruise path.

[0151] In a feasible implementation, the monitoring feedback module 50 is used to determine the communication address of the target drone, generate a communication code based on the communication address; generate a video feedback signal based on the communication code and the video feedback instruction; send the video feedback signal to the crew members of the drone group, receive the drone perspective image fed back by the crew members of the drone group based on the video feedback signal; and project the drone perspective image to the emergency central control screen.

[0152] In a feasible implementation, the manual takeover module 60 is used to determine the communication address of the target UAV, generate a communication code according to the communication address; send a manual intervention signal to the target UAV based on the communication code, and receive a response message received after the target UAV successfully verifies the manual intervention signal; after receiving the response message, establish a control channel based on the communication code; perform data transmission with the target UAV based on the control channel, and manually control the target UAV.

[0153] In a feasible implementation, the manual takeover module 60 is also used to perform hash mapping with the communication code after receiving the response information to obtain communication verification information; send the communication verification information to the target drone so that the target drone monitors the communication verification information; additionally compile the communication verification information and the transmission data to obtain exclusive communication information; and establish a control channel based on the exclusive communication information.

[0154] The emergency safety monitoring device based on drones provided in the present application adopts the emergency safety monitoring method based on drones in the above-mentioned embodiments, which can solve the technical problem of being unable to monitor the driving status of ships in complex waters and handle emergencies. Compared with the prior art, the beneficial effects of the emergency safety monitoring device based on drones provided in the present application are the same as the beneficial effects of the emergency safety monitoring method based on drones provided in the above-mentioned embodiments, and the other technical features of the emergency safety monitoring device based on drones are the same as the features disclosed in the above-mentioned embodiment methods, which will not be repeated here.

[0155] The present application provides an emergency safety monitoring device based on a drone, and the emergency safety monitoring device based on a drone includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the emergency safety monitoring method based on the drone in the above-mentioned embodiment one.

[0156] Reference below Fig. 9 , which shows a schematic diagram of the structure of an emergency safety monitoring device based on a drone suitable for implementing the embodiment of the present application. The emergency safety monitoring device based on a drone in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The drone-based emergency safety monitoring equipment shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0157] like Fig. 9As shown, the emergency safety monitoring equipment based on the drone may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the emergency safety monitoring equipment based on the drone are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the emergency safety monitoring device based on the drone to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an emergency safety monitoring device based on the drone with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0158] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0159] The emergency safety monitoring device based on drones provided by the present application adopts the emergency safety monitoring method based on drones in the above-mentioned embodiments, and can solve the technical problems of emergency safety monitoring based on drones. Compared with the prior art, the beneficial effects of the emergency safety monitoring device based on drones provided by the present application are the same as the beneficial effects of the emergency safety monitoring method based on drones provided by the above-mentioned embodiments, and the other technical features of the emergency safety monitoring device based on drones are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0160] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0161] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0162] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the emergency safety monitoring method based on a drone in the above-mentioned embodiment.

[0163] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0164] The above-mentioned computer-readable storage medium may be included in the emergency safety monitoring device based on the drone; or it may exist independently without being assembled into the emergency safety monitoring device based on the drone.

[0165] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the emergency safety monitoring device based on the drone, the emergency safety monitoring device based on the drone: receives the load data collected by each crew member in the drone group when performing a cruise mission on the water space on a preset cruise path, and the load data includes image data, thermal imaging data, positioning data and radar data; fuses the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space; identifies the three-dimensional dynamic model to determine the identification object of the water space and the motion state of the identification object; when the motion state is an emergency intervention state, switches the current cruise mission to an emergency mission, and when the remaining energy cannot meet the requirements of the emergency mission, generates an emergency response area and an emergency response signal according to the motion state, so that the emergency drone group performs the emergency mission.

[0166] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0167] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0169] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned emergency safety monitoring method based on drones, and can solve the technical problems of emergency safety monitoring based on drones. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the emergency safety monitoring method based on drones provided in the above-mentioned embodiments, and will not be elaborated here.

[0170] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned emergency safety monitoring method based on a drone.

[0171] The computer program product provided in this application can solve the technical problem of emergency safety monitoring based on drones. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the emergency safety monitoring method based on drones provided in the above embodiment, which will not be repeated here.

[0172] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An emergency safety monitoring method based on drones, characterized in that: The emergency safety monitoring method based on drone includes: Receiving load data collected by each crew member of the UAV group when performing a cruise mission on a water space on a preset cruise path, the load data including image data, thermal imaging data, positioning data and radar data; Fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space; Identify the three-dimensional dynamic model to determine the identification object in the water space and the motion state of the identification object; When the motion state is an emergency intervention state, the current cruise mission is switched to an emergency mission. When the remaining energy cannot meet the requirements of the emergency mission, an emergency response area and an emergency response signal are generated according to the motion state, so that the emergency drone group performs the emergency mission. The step of fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space includes: Determine characteristic image data and characteristic thermal imaging data in the image data and the thermal imaging data respectively, wherein the characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position; Determine the pixel deviation of the characteristic image data and the characteristic thermal imaging data, use the pixel deviation as correction data, perform position offset correction on the image data and the thermal imaging data, and obtain water area visual data; Determine the motion properties of the object in the water space according to the positioning data and the radar data to obtain water space motion data; Modeling is performed based on the water space to obtain an initial water model of the water space, and the water visual data and the water motion data are added to the initial water model to obtain a three-dimensional dynamic model.

2. The method according to claim 1, characterized in that The step of determining the pixel deviation of the characteristic image data and the characteristic thermal imaging data, using the pixel deviation as correction data, performing position offset correction on the image data and the thermal imaging data, and obtaining the water area visual data comprises: Determining the marking positions of the characteristic image data and the characteristic thermal imaging data; Respectively projecting the characteristic image data and the characteristic thermal imaging data onto a reference plane to obtain a first projection position and a second projection position; Determine the pixel coordinates of the first projection position and the second projection position respectively to obtain a first pixel coordinate and a second pixel coordinate; Taking the first pixel coordinate as a reference point, determining a deviation direction and a deviation distance between the second pixel coordinate and the first pixel coordinate; Obtaining correction data according to the deviation direction and the deviation distance; The second pixel coordinates are position-corrected based on the correction data so that the first pixel coordinates and the second pixel coordinates coincide with each other, thereby obtaining water area visual data.

3. The method according to claim 1, characterized in that The step of identifying the three-dimensional dynamic model and determining the identification object in the water space and the motion state of the identification object comprises: Screening a relatively high temperature area in the thermal imaging data, determining an overlap position between the relatively high temperature area and the image data, performing target recognition on the overlap position, and determining an identification object of the water space; Determine the position of the identified object in the three-dimensional dynamic model and the positioning data of the current drone; Recording a first change trend of the positioning data and a second change trend of the radar data, and determining a movement direction and a movement speed of the identified object according to the first change trend and the second change trend; The motion state of the identified object is obtained according to the motion direction and the motion speed.

4. The method according to claim 1, characterized in that The steps of switching the current cruise mission to an emergency mission when the motion state is an emergency intervention state, and generating an emergency response area and an emergency response signal according to the motion state so that the emergency drone group performs the emergency mission when the remaining energy cannot meet the requirements of the emergency mission include: When the motion state is an emergency intervention state, the current cruise mission is switched to an emergency mission, and the remaining energy is determined to predict the minimum emergency power value; When the remaining energy is less than the minimum emergency power value, the remaining energy cannot meet the emergency task, and the motion trajectory corresponding to the identified object is predicted according to the motion state; An emergency area is generated according to the motion trajectory, and an emergency response signal is generated according to the emergency area and the motion trajectory, so that the emergency drone group performs the emergency task.

5. The method according to claim 1, characterized in that Before the step of receiving the load data collected by each crew member in the unmanned aerial vehicle group when performing a cruise mission on the water space on the preset cruise path, the method further includes: Determine the water area division and airspace division of the water area space, and generate an electronic fence according to the airspace division and the no-fly zone of the water area space; Determine the concerned location in the water zone, and generate an initial cruise path based on the emergency monitoring point and the concerned location based on the energy priority principle; The initial cruising path is constrained according to the electronic fence to obtain a preset cruising path.

6. The method according to any one of claims 1 to 5, characterized in that The emergency safety monitoring method based on drone also includes: Determine the communication address of the target UAV, and generate a communication code according to the communication address; Generate a video return signal based on the communication code and the video return instruction; Sending a video feedback signal to a crew member of the drone group, and receiving a drone view image fed back by the crew member of the drone group based on the video feedback signal; The drone's perspective image is projected onto the emergency central control screen.

7. The method according to any one of claims 1 to 5, characterized in that The emergency safety monitoring method based on drone also includes: Determine the communication address of the target UAV, and generate a communication code according to the communication address; Sending a manual intervention signal to the target drone based on the communication code, and receiving response information of a receipt after the target drone successfully verifies the manual intervention signal; After receiving the response information, establishing a control channel based on the communication code; The target UAV is manually controlled based on data transmission with the target UAV through the control channel.

8. The method according to claim 7, characterized in that After receiving the response information, the step of establishing a control channel based on the communication code comprises: After receiving the response information, performing hash mapping with the communication code to obtain communication verification information; Sending the communication verification information to the target drone, so that the target drone monitors the communication verification information; The communication verification information is additionally compiled with the transmission data to obtain exclusive communication information; A control channel is established according to the dedicated communication information.

9. An emergency safety monitoring device based on a drone, characterized in that: The emergency safety monitoring device based on drone includes: A data perception module is used to receive the load data collected by each crew member of the UAV group when performing a cruise mission on a water space on a preset cruise path, wherein the load data includes image data, thermal imaging data, positioning data and radar data; A spatial modeling module, used for fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space; A motion fitting module, used to identify the three-dimensional dynamic model, determine the identified object in the water space and the motion state of the identified object; An emergency processing module, used for switching the current cruise mission to an emergency mission when the motion state is an emergency intervention state, and generating an emergency response area and an emergency response signal according to the motion state to enable the emergency drone group to perform the emergency mission when the remaining energy cannot meet the requirements of the emergency mission; The step of fusing the image data, the thermal imaging data, the positioning data and the radar data to obtain a three-dimensional dynamic model of the water space includes: Determine characteristic image data and characteristic thermal imaging data in the image data and the thermal imaging data respectively, wherein the characteristic image data and the characteristic thermal imaging data are characteristic data of the same entity and the same position; Determine the pixel deviation of the characteristic image data and the characteristic thermal imaging data, use the pixel deviation as correction data, perform position offset correction on the image data and the thermal imaging data, and obtain water area visual data; Determine the motion properties of the object in the water space according to the positioning data and the radar data to obtain water space motion data; Modeling is performed based on the water space to obtain an initial water model of the water space, and the water visual data and the water motion data are added to the initial water model to obtain a three-dimensional dynamic model.

Images