A method, device, storage medium and electronic device for combined navigation of unmanned aerial vehicles

Abstract

The present specification discloses a method, device, storage medium and electronic device for combined navigation of unmanned aerial vehicles. When a first target area is detected, the main control unit in the combined navigation system of the unmanned aerial vehicle determines the first position of the unmanned aerial vehicle through an inertial measurement unit and a visual sensor, determines a first navigation path from the first position to the first target area, and controls the unmanned aerial vehicle to fly according to the first navigation path. When the unmanned aerial vehicle is controlled to fly according to the first navigation path, the flight altitude of the unmanned aerial vehicle is monitored, and when the flight altitude is a preset altitude, the second position of the unmanned aerial vehicle is determined. Environmental perception data of the environment in which the unmanned aerial vehicle is located is collected through a visual sensor, and olfactory perception data of the environment in which the unmanned aerial vehicle is located is collected through an olfactory sensor. According to the olfactory perception data, a second target area is determined. Based on the environmental perception data, a second navigation path from the second position to the second target area is determined, and the unmanned aerial vehicle is controlled to fly according to the second navigation path.

Description

Technical Field

[0001] The present application relates to the field of unmanned aerial vehicle navigation, and in particular to a method, device, storage medium and electronic device for combined navigation of unmanned aerial vehicles. Background Art

[0002] With the development of science and technology, drone navigation technology has developed rapidly and is widely used in various fields, such as aerial photography, logistics distribution, search and rescue, etc. Among them, combined navigation technology refers to a method of combining two or more navigation and positioning methods for navigation.

[0003] At present, the most commonly used integrated navigation method in various fields uses the Global Navigation Satellite System (GNSS) and other navigation and positioning methods. However, the radio signals of GNSS are susceptible to interference or deception, and the satellite payload, transmission channel and ground station are vulnerable to attack, which can easily lead to the inability of the integrated navigation system to provide navigation services. Therefore, it is an important issue to realize the integrated navigation of UAVs without GNSS.

[0004] Based on this, this application specification provides a method for combined navigation of unmanned aerial vehicles. Summary of the invention

[0005] The present specification provides a method, device, storage medium and electronic device for integrated navigation of unmanned aerial vehicles to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] The present specification provides a method for integrated navigation of a UAV, the method being applied to a main control unit in an integrated navigation system of a UAV, the integrated navigation system of the UAV further comprising: an inertial measurement unit, a visual sensor and an olfactory sensor; the method comprising:

[0008] When the first target area is detected, the position of the drone is determined by the inertial measurement unit and the visual sensor, and is used as the first position;

[0009] Determining a first navigation path for the UAV from the first location to the first target area;

[0010] Controlling the UAV to fly according to the first navigation path;

[0011] When controlling the UAV to fly according to the first navigation path, monitoring the flight altitude of the UAV;

[0012] When the flight altitude is monitored to be a preset altitude, the location of the drone is determined and used as a second location; and environmental perception data of the environment in which the drone is located is collected through the visual sensor, and olfactory perception data of the environment in which the drone is located is collected through the olfactory sensor;

[0013] Determine a second target area according to the olfactory perception data; wherein the range of the second target area is not greater than the range of the first target area;

[0014] Determining a second navigation path of the UAV from the second location to the second target area based on the environmental perception data;

[0015] Based on the second navigation path, the drone is controlled.

[0016] Optionally, determining the location of the drone includes:

[0017] Acquire the attitude information of the drone through the inertial measurement unit, and acquire a first environment image of the environment in which the drone is located through the visual sensor;

[0018] The location of the UAV is determined based on the first environment image and the posture information using scene matching technology.

[0019] Optionally, determining the second target area specifically includes:

[0020] Inputting the olfactory perception data into a pre-deployed smoke plume model to obtain a spatial concentration distribution of the target gas in the environment where the UAV is located output by the smoke plume model;

[0021] A second target area is determined according to the spatial concentration distribution of the target gas.

[0022] Optionally, determining the second target area specifically includes:

[0023] In the spatial concentration distribution, determining a maximum concentration value of the target gas, and determining a region corresponding to the target gas when the concentration value of the target gas is the maximum concentration value;

[0024] The area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value is used as the second target area.

[0025] Optionally, the visual sensor includes a first visual sensor and a second visual sensor;

[0026] Determining the position of the drone by using the inertial measurement unit and the visual sensor as the first position specifically includes:

[0027] Determine the position of the drone by using the inertial measurement unit and the first visual sensor, and use the position as the first position;

[0028] Collecting environmental perception data of the environment in which the drone is located by the visual sensor specifically includes:

[0029] The environmental perception data of the environment in which the drone is located is collected through the second visual sensor.

[0030] Optionally, determining a second navigation path of the UAV from the second position to the second target area specifically includes:

[0031] Determining obstacles in the environment of the drone based on the environmental perception data;

[0032] Based on the determined obstacles in the environment of the drone, a second navigation path of the drone from the second position to the second target area is determined.

[0033] Optionally, detecting the first target area specifically includes:

[0034] Collecting a second environment image through the visual sensor;

[0035] The second environment image is detected, and when it is detected that the second environment image includes a target feature, a first target area is determined based on an area corresponding to the second environment image.

[0036] This specification provides a device for integrated navigation of unmanned aerial vehicles, the device is located in the main control unit of the integrated navigation system of unmanned aerial vehicles, the integrated navigation system of unmanned aerial vehicles also includes: an inertial measurement unit, a visual sensor and an olfactory sensor; the device specifically includes:

[0037] A detection module, configured to determine the position of the drone by using the inertial measurement unit and the visual sensor when the first target area is detected, and use the position as the first position;

[0038] A first path determination module, configured to determine a first navigation path of the UAV from the first position to the first target area;

[0039] A first control module, used to control the UAV to fly according to the first navigation path;

[0040] an altitude monitoring module, configured to monitor the flight altitude of the drone when controlling the drone to fly according to the first navigation path;

[0041] A data acquisition module, for determining the position of the drone as a second position when the flight altitude is monitored to be a preset altitude; and for acquiring environmental perception data of the environment in which the drone is located through the visual sensor, and for acquiring olfactory perception data of the environment in which the drone is located through the olfactory sensor;

[0042] A region determination module, configured to determine a second target region according to the olfactory perception data; wherein the range of the second target region is not greater than the range of the first target region;

[0043] A second path determination module, configured to determine a second navigation path of the UAV from the second position to the second target area based on the environmental perception data;

[0044] A second control module is used to control the UAV based on the second navigation path.

[0045] Optionally, the detection module is specifically used to obtain the posture information of the drone through the inertial measurement unit, and to obtain a first environmental image of the environment in which the drone is located through the visual sensor; and to determine the position of the drone based on the first environmental image and the posture information using scene matching technology.

[0046] Optionally, the area determination module is specifically used to input the olfactory perception data into a pre-deployed smoke plume model to obtain the spatial concentration distribution of the target gas in the environment of the UAV output by the smoke plume model; and determine the second target area based on the spatial concentration distribution of the target gas.

[0047] Optionally, the area determination module is specifically used to determine the maximum concentration value of the target gas in the spatial concentration distribution, and determine the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value; and use the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value as the second target area.

[0048] Optionally, the visual sensor includes a first visual sensor and a second visual sensor;

[0049] The detection module is specifically used to determine the position of the drone through the inertial measurement unit and the first visual sensor, and use the position as the first position;

[0050] The data acquisition module is specifically used to collect environmental perception data of the environment in which the drone is located through the second visual sensor.

[0051] Optionally, the second path determination module is specifically used to determine obstacles in the environment of the drone based on the environmental perception data; and determine a second navigation path of the drone from the second position to the second target area based on the determined obstacles in the environment of the drone.

[0052] Optionally, the detection module is specifically used to collect a second environmental image through the visual sensor; detect the second environmental image, and when it is detected that the second environmental image includes target features, determine the first target area based on the area corresponding to the second environmental image.

[0053] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method of drone combined navigation is implemented.

[0054] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for combined navigation of unmanned aerial vehicles when executing the program.

[0055] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0056] It can be seen from the method of UAV integrated navigation provided in this specification that when the presence of a first target area in the environment of the UAV is detected, the main control unit in the UAV integrated navigation system can first determine the first position of the UAV through the inertial measurement unit and the visual sensor, and determine the first navigation path of the UAV from the first position to the first target area, so as to control the UAV to fly according to the first navigation path. In the process of controlling the UAV to fly according to the first navigation path, the main control unit can monitor the flight altitude of the UAV, and when the flight altitude of the UAV is monitored to be a preset altitude, determine the second position of the UAV. And collect environmental perception data of the environment of the UAV through the visual sensor, and collect olfactory perception data of the environment of the UAV through the olfactory sensor. Then, according to the olfactory perception data, determine the second target area, and the range of the second target area is not greater than the range of the first target area. Finally, based on the environmental perception data, determine the second navigation path of the UAV from the second position to the second target area, so as to control the UAV to fly according to the second navigation path.

[0057] It can be seen from the above method that when this method performs UAV combined navigation, the UAV is first navigated and the flight altitude is monitored through the inertial measurement unit and the visual sensor. When the UAV flight altitude is monitored to be a preset altitude, the subsequent UAV navigation is performed through the olfactory sensor and the visual sensor. This method enables the UAV to perform autonomous navigation in a variety of environments. While achieving precise positioning of the UAV navigation, it improves the stability of navigation, which is different from the method of relying on GNSS for UAV navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0059] Figure 1 A flowchart of a method for integrated navigation of unmanned aerial vehicles provided in this specification;

[0060] Figure 2 A schematic diagram of a UAV integrated navigation system provided in this specification;

[0061] Figure 3 A flow chart of a method for fire detection provided in this manual;

[0062] Figure 4 A schematic diagram of drone navigation based on environmental perception data and olfactory perception data provided in this manual;

[0063] Figure 5 A schematic diagram of a method for determining the collection step length of olfactory sensing data provided in this specification;

[0064] Figure 6 A schematic diagram of a device for integrated navigation of unmanned aerial vehicles provided in this manual;

[0065] Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the specification.

[0067] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0068] like Figure 1 As shown, Figure 1 A flowchart of a method for integrated navigation of a drone provided in this specification may specifically include the following steps:

[0069] S100: When a first target area is detected, the position of the drone is determined by the inertial measurement unit and the visual sensor, and is used as the first position.

[0070] S102: Determine a first navigation path of the drone from the first position to the first target area.

[0071] S104: Control the UAV to fly according to the first navigation path.

[0072] like Figure 2 As shown, it is a schematic diagram of a UAV integrated navigation system provided in this specification. It can be seen that the system includes: a main control unit, an inertial measurement unit, a visual sensor, and an olfactory sensor. In this specification, the inertial measurement unit is used to provide the attitude information of the UAV, such as speed, heading, etc., and the inertial measurement unit can be multiple, the visual sensor is used to collect visual perception data, and the olfactory sensor is used to collect olfactory perception data, so as to assist the main control unit in controlling the UAV.

[0073] The method of UAV combined navigation provided in this manual can be applied to different scenarios, such as personnel search and rescue, fire detection, and gas leakage source detection, etc. For the sake of ease of explanation, the following description takes the scene of fire detection as an example.

[0074] In this specification, when the main control device detects the first target area, the position of the drone can be determined through the inertial measurement unit and the visual sensor, and used as the first position.

[0075] In the aforementioned fire detection scenario, the first target area may be the fire area. In order to realize the fire detection of the drone, the main control unit controls the drone to a high-altitude position and detects the first target area at high altitude, that is, the area where the fire exists, and obtains a rough fire area based on the detection results. Based on the inertial measurement unit and the visual sensor, the drone is controlled to enter the rough fire area, and then based on the olfactory sensor and the visual sensor, the source of the fire is located, that is, the precise fire area is located, and the source of the fire is sent to the server so that relevant personnel can control the fire situation in the fire area. Figure 3As shown, it is a flow chart of a method for fire detection provided in this specification. It can be seen that in this specification, the main control unit controls the flight altitude of the drone to be a first preset altitude, and detects in real time whether there is a first target area, that is, a fire area, in the current environment where the drone is located. When the existence of the first target area is detected, the inertial measurement unit and the visual sensor are started to assist the main control unit in controlling the drone to fly to the first target area. During the flight, the flight altitude of the drone can be monitored in real time. When the flight altitude of the drone is monitored to be a second preset altitude, the olfactory sensor and the visual sensor are started to assist the main control unit in navigating the drone. Among them, the first preset altitude is not less than the second preset altitude.

[0076] When the main control unit detects the first target area, it can collect images of the environment in which the drone is located through a visual sensor, that is, collect the second environment image. And a detection algorithm or a detection model, etc. can be used to detect the second environment image to determine whether there is a target feature in the second environment image. The target feature can be predetermined based on a specific scene, such as the aforementioned fire discovery scene, and the target feature can be a color feature, a temperature feature, a smoke feature, etc. When the main control unit detects that the second environment image includes a target feature, it means that there is a fire in the area corresponding to the second environment image, so the main control unit can determine the first target area based on the area corresponding to the second environment image, such as the area corresponding to the second environment image can be used as the first target area, or a designated area in the area corresponding to the second environment image can be used as the first target area, etc., and this specification does not limit it. In addition, when the main control unit detects the first target area, the specific method used is also not limited, such as the aforementioned visual sensor, of course, it can also be used through a temperature sensor, a smoke sensor, etc. This specification does not limit what the first target area is specifically, and it can be determined based on specific scenarios. For example, in a fire discovery scenario, the first target area is the area including the fire site. For example, in a personnel search and rescue scenario, the first target area is the area including the location of the personnel.

[0077] The main control device can obtain the attitude information of the drone through the inertial measurement unit, and can obtain the first environment image of the environment in which the drone is located through the visual sensor. Then, based on the first environment image and the attitude information of the drone, the scene matching technology is used to determine the current location of the drone, and this is used as the first location. Based on the first environment information and the attitude information of the drone, the scene matching algorithm is used to determine the location of the drone, which is a relatively mature technology at present, and the details will not be repeated.

[0078] Based on the above, the main control device detects the first target area and determines the first position of the drone when it detects the first target area, then the main control device can determine the first navigation path of the drone from the first position to the first target area. How the main control device performs path planning and determines the first navigation path of the drone from the first position to the first target area can be achieved by using a grid map method, a graph search algorithm, etc., and this manual does not make specific restrictions.

[0079] S106: When controlling the UAV to fly according to the first navigation path, monitoring the flight altitude of the UAV.

[0080] S108: When the flight altitude is monitored to be a preset altitude, the location of the UAV is determined and used as the second location; and environmental perception data of the environment in which the UAV is located is collected through the visual sensor, and olfactory perception data of the environment in which the UAV is located is collected through the olfactory sensor.

[0081] In one or more embodiments of this specification, as described above, when the main control device detects the first target area, the flight altitude of the drone is controlled to be the first preset altitude. When the drone is controlled to fly to the first target area, or when the drone is controlled to fly according to the first navigation path, the flight altitude of the drone can be monitored in real time. When the flight altitude of the drone is monitored to be the second preset altitude, the main control device determines the current position of the drone and uses it as the second position. This specification does not elaborate on how to determine the current position of the drone. For example, the current second position of the drone can be determined based on the determined first navigation path, or the current second position of the drone can be determined based on an inertial measurement unit and a visual sensor, and so on.

[0082] Since the drone needs to accurately locate the second target area after descending to a certain height, this manual does not limit what the second target area is specifically. It can be determined based on specific scenarios. For example, in a fire detection scenario, the second target area is the area where the source of the fire is located. Since specific gases are produced when a fire occurs, the second target area can also be the area where the source of the target gas is located. This manual does not specifically limit the specific target gas. In the aforementioned fire detection scenario, the target gas can be carbon dioxide, carbon monoxide, etc. In the dangerous gas leakage source detection scenario, the second target area is the area where the source of the dangerous gas leakage is located. In this manual, the main control device can locate the source of the target gas, and the area where the source of the target gas is located is the second target area.

[0083] Therefore, the main control device can enable visual sensors and olfactory sensors. The visual sensors can collect environmental perception data of the drone's environment, and the olfactory sensors can collect olfactory perception data of the drone's environment. Of course, the olfactory perception data is olfactory perception data for the target gas, and then the flight of the drone can be controlled through the environmental perception data and olfactory perception data.

[0084] S110: Determine a second target area according to the olfactory perception data; wherein the range of the second target area is not larger than the range of the first target area.

[0085] S112: Determine a second navigation path of the UAV from the second position to the second target area based on the environmental perception data.

[0086] S114: Control the drone based on the second navigation path.

[0087] like Figure 4 As shown, it is a schematic diagram of a drone navigation based on environmental perception data and olfactory perception data provided in the specification of this application. In this specification, the plume model can be pre-deployed in the main control device, and then the main control device can input the obtained olfactory perception data into the plume model, and the plume model can output the spatial concentration distribution of the target gas in the environment where the drone is located. Then the second target area is determined according to the obtained spatial concentration distribution of the target gas. Specifically, the main control device can determine the maximum concentration value of the target gas in the spatial concentration distribution of the target gas, and determine the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration. Then the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value can be used as the second target area, or the area corresponding to the target gas within the specified concentration range in the spatial concentration distribution of the target gas can be used as the second target area, and so on. This specification is not limited specifically, as long as the range of the second target area is not greater than the range of the first target area. In addition, the main control device can determine the second navigation path of the drone from the second position to the second target area based on the environmental perception data. Environmental perception data is used to determine obstacles in the environment where the drone is located, so that the main control device can plan a safe navigation path. Specifically, the path planning can be carried out using the grid map method, graph search algorithm, etc. as described above, and this manual does not make specific restrictions.

[0088] Then, the main control device can control the drone to fly according to the second navigation path.

[0089] based on Figure 1The above-mentioned UAV combined navigation method provided in the present specification is provided through a UAV combined navigation system including a main control unit, an inertial measurement unit, a visual sensor and an olfactory sensor. The main control unit first navigates the UAV and monitors the flight altitude through the inertial measurement unit and the visual sensor. When the UAV flight altitude is monitored to be a preset altitude, the olfactory sensor and the visual sensor are used for subsequent UAV navigation. This method enables the UAV to perform autonomous navigation in a variety of environments, gets rid of dependence on GNSS, and improves the stability of navigation while achieving accurate positioning of the source of the target gas.

[0090] In addition, in the present specification, the visual sensor may include a first visual sensor and a second visual sensor, and the resolution, magnification and lighting conditions of the first visual sensor are better than those of the second visual sensor, that is, the performance of the first visual sensor is better than that of the second visual sensor.

[0091] In the above step S100, when the main control device determines the position of the drone through the inertial measurement unit and the visual sensor and takes it as the first position, the main control device can determine the position of the drone through the inertial measurement unit and the first visual sensor and take it as the first position. In the above step S108, when the main control device collects environmental perception data of the environment in which the drone is located through the visual sensor, the main control device can collect environmental perception data of the environment in which the drone is located through the second visual sensor.

[0092] Furthermore, this specification also provides a flow chart of a method for collecting olfactory sensor data, such as Figure 5 The figure shows a flow chart of a method for collecting olfactory sensing data provided in this specification.

[0093] S500: Obtain the concentration of the target gas corresponding to several odor collection points in the environment where the drone is located through the olfactory sensor; and determine the wind speed, the diffusion coefficient of the target gas, and the release rate of the target gas in the environment where the drone is located.

[0094] S502: According to the concentration of the target gas corresponding to the several odor collection points, the wind speed, the diffusion coefficient, the release rate, and based on a preset objective function, a nonlinear least squares algorithm is used to determine the predicted odor source of the target gas and the location of the odor source.

[0095] S504: Determine a first collection step length according to the determined location of the predicted odor source and the second location.

[0096] S506: Determine an actual collection step length according to the first collection step length and a preset second collection step length for collecting olfactory sensor data, and control the drone to fly toward the predicted odor source according to the actual collection step length to collect olfactory perception data next time.

[0097] Based on the above, it can be known that the drone is currently located in the second position. The main control device can determine several odor collection points in the environment where the drone is located. For each odor collection point, the olfactory sensor is used to obtain the concentration of the target gas corresponding to the odor collection point. In addition, the main control device can determine the wind speed, diffusion coefficient of the target gas, and release rate of the target gas when the drone is in the second position. Then, the least squares algorithm can be used to determine the predicted odor source of the target gas based on the wind speed, diffusion coefficient of the target gas, release rate of the target gas, and concentration of the target gas corresponding to several odor collection points in the environment where the drone is located.

[0098] In this specification, the relationship between the location of the predicted odor source and the wind speed, the diffusion coefficient of the target gas, and the release rate of the target gas can be expressed by the following formula:

[0099] ,

[0100] .

[0101] in, To predict the location of the odor source, is the location of the odor collection point, is the preset objective function, Indicates that at the odor collection point The concentration of the target gas, is the wind speed, is the distance from the sensor to the odor source, is the release rate, is the diffusion coefficient, express and The distance between Indicates the number of odor collection points.

[0102] Based on the above formula, the least square method is used to determine the minimum value of S, thereby obtaining the predicted location of the odor source. It should be noted that there are two or more odor collection points.

[0103] Therefore, the main control device can determine the distance between the location of the odor source and the second location, which is the first collection step. When the first collection step is smaller than the second collection step, the first collection step is used as the actual collection step, and the drone is controlled to fly according to the actual collection step to collect the next olfactory perception data. For example: the preset collection step of olfactory sensing data is 5m, that is, the main control device controls the drone to collect olfactory perception data every 5m of flight. Assuming that the second collection step is 4m, that is, the distance from the predicted location of the odor source to the location of the drone is 4m, the actual collection step is 4m, so that the drone can be controlled to move 4m in the direction of the predicted odor source, and collect olfactory perception data again after moving 4m.

[0104] The above method for determining the collection step length of olfactory perception data improves the accuracy of the determined odor source.

[0105] Based on the method for combined navigation of unmanned aerial vehicles described above, the embodiment of this specification also provides a schematic diagram of a device for combined navigation of unmanned aerial vehicles, such as Figure 6 shown.

[0106] Figure 6 A schematic diagram of a device for drone integrated navigation provided in an embodiment of this specification, wherein the device is located in a main control unit of a drone integrated navigation system, wherein the drone integrated navigation system further includes: an inertial measurement unit, a visual sensor, and an olfactory sensor; the device specifically includes:

[0107] The detection module 600 is used to determine the position of the drone by using the inertial measurement unit and the visual sensor when the first target area is detected, and use the position as the first position;

[0108] A first path determination module 602 is used to determine a first navigation path of the UAV from the first position to the first target area;

[0109] A first control module 604, used to control the UAV to fly according to the first navigation path;

[0110] an altitude monitoring module 606, configured to monitor the flight altitude of the drone when controlling the drone to fly according to the first navigation path;

[0111] The data acquisition module 608 is used to determine the position of the drone as the second position when the flight altitude is monitored to be a preset altitude; and to collect environmental perception data of the environment in which the drone is located through the visual sensor, and to collect olfactory perception data of the environment in which the drone is located through the olfactory sensor;

[0112] The region determination module 610 is used to determine a second target region according to the olfactory perception data; wherein the range of the second target region is not greater than the range of the first target region;

[0113] A second path determination module 612, configured to determine a second navigation path of the UAV from the second position to the second target area based on the environmental perception data;

[0114] The second control module 614 is used to control the UAV based on the second navigation path.

[0115] Optionally, the detection module 600 is specifically used to obtain the posture information of the drone through the inertial measurement unit, and to obtain a first environmental image of the environment in which the drone is located through the visual sensor; and to determine the position of the drone based on the first environmental image and the posture information using scene matching technology.

[0116] Optionally, the area determination module 610 is specifically used to input the olfactory perception data into a pre-deployed smoke plume model to obtain the spatial concentration distribution of the target gas in the environment of the UAV output by the smoke plume model; and determine the second target area based on the spatial concentration distribution of the target gas.

[0117] Optionally, the area determination module 610 is specifically used to determine the maximum concentration value of the target gas in the spatial concentration distribution, and determine the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value; and use the area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value as the second target area.

[0118] Optionally, the visual sensor includes a first visual sensor and a second visual sensor;

[0119] The detection module 600 is specifically used to determine the position of the drone through the inertial measurement unit and the first visual sensor, and use it as the first position;

[0120] The data acquisition module 608 is specifically used to collect environmental perception data of the environment in which the drone is located through the second visual sensor.

[0121] Optionally, the second path determination module 612 is specifically used to determine obstacles in the environment of the drone based on the environmental perception data; and determine a second navigation path of the drone from the second position to the second target area based on the determined obstacles in the environment of the drone.

[0122] Optionally, the detection module 600 is specifically used to collect a second environment image through the visual sensor; detect the second environment image, and when it is detected that the second environment image includes target features, determine the first target area based on the area corresponding to the second environment image.

[0123] The embodiments of this specification also provide a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the method of drone combined navigation described above.

[0124] Based on the method of UAV combined navigation described above, this specification embodiment also proposes Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the method of UAV integrated navigation described above.

[0125] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0126] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0127] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0128] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0129] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0130] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0136] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0140] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0141] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for integrated navigation of unmanned aerial vehicles, characterized in that: The method is applied to a main control unit in a drone integrated navigation system, wherein the drone integrated navigation system further comprises: an inertial measurement unit, a visual sensor and an olfactory sensor; the drone integrated navigation system is used for fire detection; the method comprises: An image of the environment in which the drone is located is collected by a visual sensor as a second environment image, and the second environment image is detected to determine whether there is a target feature in the second environment image, wherein the target feature is a color feature, a temperature feature, and a smoke feature; when it is detected that the second environment image includes the target feature, a first target area is determined based on an area corresponding to the second environment image, wherein the first target area is a fire area; a position of the drone is determined by the inertial measurement unit and the visual sensor as the first position; Determining a first navigation path for the UAV from the first location to the first target area; Controlling the UAV to fly according to the first navigation path; When controlling the UAV to fly according to the first navigation path, monitoring the flight altitude of the UAV; When the flight altitude is monitored to be a preset altitude, the location of the drone is determined and used as a second location; and environmental perception data of the environment in which the drone is located is collected through the visual sensor, and olfactory perception data of the environment in which the drone is located is collected through the olfactory sensor; the environmental perception data is used to determine obstacles in the environment in which the drone is located; Determine a second target area according to the olfactory perception data; wherein the range of the second target area is not greater than the range of the first target area, the second target area is the area where the source of the target gas is located, and the target gas is carbon dioxide or carbon monoxide; Determining a second navigation path of the UAV from the second location to the second target area based on the environmental perception data; Based on the second navigation path, the drone is controlled.

2. The method according to claim 1, characterized in that Determine the location of the drone, including: Acquire the attitude information of the drone through the inertial measurement unit, and acquire a first environment image of the environment in which the drone is located through the visual sensor; The location of the UAV is determined based on the first environment image and the posture information using scene matching technology.

3. The method according to claim 1, characterized in that Determine the second target area, including: Inputting the olfactory perception data into a pre-deployed smoke plume model to obtain a spatial concentration distribution of the target gas in the environment where the UAV is located output by the smoke plume model; A second target area is determined according to the spatial concentration distribution of the target gas.

4. The method according to claim 3, characterized in that Determine the second target area, including: In the spatial concentration distribution, determining a maximum concentration value of the target gas, and determining a region corresponding to the target gas when the concentration value of the target gas is the maximum concentration value; The area corresponding to the target gas when the concentration value of the target gas is the maximum concentration value is used as the second target area.

5. The method according to claim 1, characterized in that The visual sensor includes a first visual sensor and a second visual sensor; Determining the position of the drone by using the inertial measurement unit and the visual sensor as the first position specifically includes: Determine the position of the drone by using the inertial measurement unit and the first visual sensor, and use the position as the first position; Collecting environmental perception data of the environment in which the drone is located by the visual sensor specifically includes: The environmental perception data of the environment in which the drone is located is collected through the second visual sensor.

6. The method according to claim 1, characterized in that Determining a second navigation path of the UAV from the second position to the second target area specifically includes: Determining obstacles in the environment of the drone based on the environmental perception data; Based on the determined obstacles in the environment of the drone, a second navigation path of the drone from the second position to the second target area is determined.

7. The method according to claim 1, characterized in that Detecting the first target area specifically includes: Collecting a second environment image through the visual sensor; The second environment image is detected, and when it is detected that the second environment image includes a target feature, a first target area is determined based on an area corresponding to the second environment image.

8. A device for integrated navigation of unmanned aerial vehicles, characterized in that: The device is located in the main control unit of the UAV integrated navigation system, and the UAV integrated navigation system also includes: an inertial measurement unit, a visual sensor and an olfactory sensor; the UAV integrated navigation system is used for fire detection; the device specifically includes: The detection module is used to collect an image of the environment in which the drone is located through a visual sensor as a second environment image, detect the second environment image, and determine whether there is a target feature in the second environment image, wherein the target feature is a color feature, a temperature feature, and a smoke feature; when it is detected that the second environment image includes the target feature, a first target area is determined based on an area corresponding to the second environment image, wherein the first target area is a fire area; and the position of the drone is determined through the inertial measurement unit and the visual sensor as the first position; A first path determination module, configured to determine a first navigation path of the UAV from the first position to the first target area; A first control module, used to control the UAV to fly according to the first navigation path; an altitude monitoring module, configured to monitor the flight altitude of the drone when controlling the drone to fly according to the first navigation path; A data acquisition module, for determining the position of the drone as a second position when the flight altitude is monitored to be a preset altitude; and for collecting environmental perception data of the environment in which the drone is located through the visual sensor, and for collecting olfactory perception data of the environment in which the drone is located through the olfactory sensor; the environmental perception data is used to determine obstacles in the environment in which the drone is located; A region determination module, configured to determine a second target region according to the olfactory perception data; wherein the range of the second target region is not greater than the range of the first target region, the second target region is the region where the source of the target gas is located, and the target gas is carbon dioxide or carbon monoxide; A second path determination module, configured to determine a second navigation path of the UAV from the second position to the second target area based on the environmental perception data; A second control module is used to control the UAV based on the second navigation path.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images