Unmanned vehicles without satellite navigation systems

Abstract

An unmanned vehicle operating system without a satellite navigation system includes a vehicle-side control unit, a vehicle-side global positioning signal receiving unit, a calculation unit, an inertial measurement unit, an image acquisition unit, and a database. The vehicle-side global positioning signal receiving unit is signal-connected to the vehicle-side control unit; the calculation unit is signal-connected to the vehicle-side control unit; the inertial measurement unit is signal-connected to the calculation unit; the image acquisition unit is signal-connected to the calculation unit; and the database is signal-connected to the calculation unit. When the unmanned vehicle cannot receive satellite positioning signals due to external factors, the calculation unit calculates a correction signal by pairing images. The vehicle-side control unit then calculates subsequent control signals based on the correction signal, enabling the unmanned vehicle to complete its mission.

Description

Technical Field

[0001] The present invention relates to a mobile device and method for unmanned vehicles without satellite navigation systems, and more particularly to a mobile device that calculates correction signals through inertial sensing elements and paired images to further derive control signals. Prior Technology

[0002] The Global Positioning System (GPS), GLONASS, BeiDou Navigation Satellite System (BDS), and Galileo, which are commonly heard of today, can all be considered a type of Global Navigation Satellite System (GNSS).

[0003] Existing drones (or unmanned vehicles) mainly use GPS or GNSS, requiring four or more satellites for precise positioning. However, in many external situations, such as the modern battlefield, long-distance wireless satellite signals are easily interfered with, potentially preventing drones from completing their missions because they cannot locate or calculate their own flight status.

[0004] In summary, how to enable drones to locate themselves and calculate their flight status without a global satellite system has become a pressing issue for the industry to address through improvement and innovation. Summary of the Invention

[0005] In view of the shortcomings of the prior art, we have been eager to improve and innovate it. After years of research and experimentation, we have finally successfully developed the mobile device of the unmanned vehicle without satellite navigation system of the present invention.

[0006] This invention provides a mobile device for unmanned vehicles without a satellite navigation system, comprising a vehicle-side control unit, a vehicle-side global positioning signal receiving unit, a calculation unit, an inertial measurement unit, an image acquisition unit, and a database; wherein, the vehicle-side global positioning signal receiving unit is signal-connected to the vehicle-side control unit; the calculation unit is signal-connected to the vehicle-side control unit; the inertial measurement unit is signal-connected to the calculation unit; the image acquisition unit is signal-connected to the calculation unit; and the database is signal-connected to the calculation unit.

[0007] The calculation unit includes an inertial measurement calculation module and a satellite image calculation module.

[0008] Among them, the inertial measurement calculation module is a neural network or a neural network; the satellite image calculation module has a feature extraction algorithm.

[0009] The inertial measurement unit is a combination of an accelerometer and a gyroscope; the image acquisition unit is a camera or a video camera.

[0010] Among them, the mobile device of the unmanned vehicle without a satellite navigation system includes a barometer and a magnetic needle, and the barometer and magnetic needle signal connection calculation unit.

[0011] Among them, the unmanned vehicle without a satellite navigation system includes a power source, which is electrically connected to the vehicle's control unit.

[0012] This invention also provides a method for unmanned vehicles to move without a satellite navigation system, the steps of which include: Step S01, Change navigation mode: When the vehicle-side global satellite positioning signal receiving unit does not receive a positioning signal, the vehicle-side control unit activates the calculation unit to replace the vehicle-side global satellite positioning signal receiving unit; Step S02, Read measurement values: The inertial measurement unit outputs the angular velocity and acceleration measurement values ​​of the unmanned vehicle to the calculation unit; Step S03, generating navigation signal: The calculation unit calculates the local movement amount using the measurement values. The local movement amount is added to the absolute starting latitude and longitude coordinates when the unmanned vehicle begins to operate to form a navigation signal; Step S04, Image Capture: The image capture unit captures the image and provides it to the calculation unit; Step S05, Feature point extraction: The calculation unit extracts and marks the feature points of the image; Step S06, Extract satellite image feature points: The calculation unit extracts and marks the feature points of the satellite images based on the satellite images in the database; Step S07, Feature Point Pairing: The calculation unit pairs the feature points of the image with feature points with those of the satellite image with feature points; Step S08, Overlapping Feature Points: The calculation unit matches the feature points of the satellite image with the feature points of the image, and projects the image onto the satellite image so that the corresponding feature points of the two coincide, thereby determining the position of the unmanned vehicle in the satellite image; Step S09, Flight Status Correction: The calculation unit uses satellite imagery to calculate the longitude and latitude coordinates of the unmanned vehicle when it captures the image, and combines this with the timestamp of the image captured by the image capturing unit. Then, using the local displacement amount from step S03, the calculation unit calculates a correction signal. The correction signal is transmitted to the vehicle-end control unit, which calculates the subsequent control signal based on the correction signal. The control signal causes the unmanned vehicle to correct its flight status.

[0013] In step S03, the calculation unit uses a neural network or a neural network. The training method for the neural network or neural network is as follows: the barometer provides the calculation unit with the altitude of the unmanned vehicle; the magnetic needle provides the calculation unit with the heading angle of the unmanned vehicle; the neural network or neural network forms a loss function based on the measurement values, the aforementioned altitude and heading angle, and the corresponding real labels, and uses the Adam optimizer to find the minimum point in the loss function.

[0014] Among them, the absolute starting latitude and longitude coordinates are either manually entered values ​​or the positioning signals received by the vehicle-end global satellite positioning signal receiving unit when the unmanned vehicle begins to operate.

[0015] In summary, the present invention provides an unmanned vehicle operating device without a satellite navigation system. When the unmanned vehicle is in operation and fails to receive a satellite (GPS or GNSS) positioning signal, the calculation unit calculates a correction signal by pairing the image captured by the image capturing unit with satellite images in the database. The vehicle-end control unit then calculates the subsequent control signal based on the correction signal, enabling the unmanned vehicle to complete its mission. Simple Explanation of the Diagram

[0016] Figure 1 is a schematic diagram of a mobile device for an unmanned vehicle without a satellite navigation system according to the present invention; Figure 2 is a flowchart illustrating a method for unmanned vehicles to move without a satellite navigation system according to the present invention. Implementation

[0017] Please refer to Figure 1, which is a schematic diagram of a mobile device for an unmanned vehicle without a satellite navigation system according to the present invention. As shown in the figure, the mobile device for the unmanned vehicle without a satellite navigation system includes a power source 10, a vehicle-end control unit 11, a vehicle-end global positioning signal receiving unit 12, an inertial measurement unit 13, an image acquisition unit 14, a calculation unit 15, and a database 16. The power source 10, the vehicle-end control unit 11, the vehicle-end global positioning signal receiving unit 12, the inertial measurement unit 13, the image acquisition unit 14, the calculation unit 15, and the database 16 are installed inside the unmanned vehicle; the unmanned vehicle can be aerial, land-based, surface-based, or underwater, but is not limited thereto. In this invention, an aerial unmanned vehicle is used as an example, but this is not a limitation.

[0018] The vehicle-side control unit 11 is signal-connected to the vehicle-side global satellite positioning signal receiving unit 12, or the vehicle-side control unit 11 accepts manually input values ​​as absolute starting latitude and longitude coordinates.

[0019] The inertial measurement unit 13 is connected to the calculation unit 15; the inertial measurement unit 13 is a device for measuring the three-axis attitude angles (or angular rates) and acceleration of an object; the inertial measurement unit 13 is a combination of an accelerometer and a gyroscope.

[0020] The image capturing unit 14 is connected to the calculation unit 15; the image capturing unit 14 is a camera or camcorder.

[0021] The calculation unit 15 is signal-connected to the inertial measurement unit 13 and the vehicle-end control unit 11; the calculation unit 15 has an inertial measurement calculation module 150 and a satellite image calculation module 151; the inertial measurement calculation module 150 is signal-connected to the satellite image calculation module 151, and the inertial measurement calculation module 150 can be a neural network or a neural network; the satellite image calculation module 151 has a feature extraction algorithm.

[0022] The unmanned vehicle without a satellite navigation system includes a barometer and a magnetometer (not shown in the figure). The barometer and magnetometer are located inside the unmanned vehicle, and the barometer and magnetometer signals are connected to the calculation unit 15. The barometer is used to calculate the altitude (flight altitude) of the unmanned vehicle, and the magnetometer is used to obtain the heading angle of the unmanned vehicle.

[0023] Database 16 is connected to computing unit 15; Database 16 contains satellite images.

[0024] The power source 10 is electrically connected to the vehicle-end control unit 11, so as to provide electrical energy to the vehicle-end global satellite positioning signal receiving unit 12, inertial measurement unit 13, image acquisition unit 14, calculation unit 15 and database 16 through the vehicle-end control unit 11.

[0025] When the unmanned vehicle is in operation (flight state), if the vehicle-side global satellite positioning signal receiving unit 12 does not receive a positioning signal from the satellite, the vehicle-side control unit 11 activates the calculation unit 15 to replace the vehicle-side global satellite positioning signal receiving unit 12.

[0026] The inertial measurement unit 13 outputs the measured values ​​of angular velocity and acceleration to the inertial measurement calculation module 150.

[0027] The image capturing unit 14 continuously captures images at an angle perpendicular to the ground and provides the captured images to the calculation unit 15.

[0028] The satellite image processing module 151 extracts and marks the feature points in the image captured by the image capturing unit 14, such as points on the outline of a road, points on the corners of a building, and points on the corners of the water-land interface, which are sufficiently representative features.

[0029] The satellite image processing module 151 extracts and marks the feature points in the satellite images based on the satellite images in the database 16; the images with feature points are paired with the satellite images with feature points, and each pairing generates a score representing the degree of matching.

[0030] After matching the feature points of the satellite image with the image, the calculation unit 15 projects the image onto the satellite image so that the corresponding feature points of the two coincide. At this time, the position of the unmanned vehicle in the satellite image can be determined.

[0031] Please refer to Figure 2, which is a flowchart illustrating a method for moving an unmanned vehicle without a satellite navigation system according to the present invention. As shown in the figure, the method for moving an unmanned vehicle without a satellite navigation system includes the following steps: Step S00: Determine whether satellite positioning is received: Does the vehicle-side global satellite positioning signal receiving unit 12 receive positioning signals from satellites? If no, proceed to step S01; if yes, proceed to step S00A. Step S00A, Continuous Flight: The vehicle-side global satellite positioning signal receiving unit 12 continuously receives positioning signals from satellites and transmits the positioning signals to the vehicle-side control unit 11. The vehicle-side control unit 11 calculates subsequent control signals based on the positioning signals, and the unmanned vehicle completes the mission according to the control signals. Step S01, Change navigation mode: The vehicle-side global satellite positioning signal receiving unit 12 does not receive a positioning signal from the satellite, and the vehicle-side control unit 11 activates the calculation unit 15 so that the calculation unit 15 replaces the vehicle-side global satellite positioning signal receiving unit 12; Step S02, Read measurement values: During the operation of the unmanned vehicle, the inertial measurement unit 13 outputs the measurement values ​​of angular velocity and acceleration to the inertial measurement calculation module 150; Step S03, generating navigation signals: The neural network (or neural network) of the inertial measurement calculation module 150 directly reads the measurement values ​​from step S02 to calculate the local movement from the start of the unmanned vehicle to the time of calculation; In this step S03, the neural network (or neural network) needs to be trained to calculate the navigation signal. The training method is as follows: the barometer provides the altitude of the unmanned vehicle to the calculation unit 15; the magnetic needle provides the heading angle of the unmanned vehicle to the calculation unit 15; the neural network (or neural network) of the inertial measurement calculation module 150 uses the measurement values ​​described in step S02 and the Adam Optimizer to find the minimum point in the loss function. Further explanation: The neural network (or neural network) of the inertial measurement calculation module 150, based on the measurement values ​​described in step S02, uses the aforementioned altitude and heading angle, as well as acceleration and angular velocity, to derive an inferred relative change. This change is then calculated relative to the true label (the true label is a precise observation obtained by measuring the location of the unmanned vehicle using external instruments; this is calculated from GPS (or GNSS) coordinates to determine the corresponding positional change in three-dimensional space). The difference between the inferred value and the true label is then calculated, and a loss function is used to calculate this difference. The Adam Optimizer is then used to adjust the model's learning rate and find the minimum point of the loss function. Step S04, Image Capture: The image capture unit 14 continuously captures images (videos or photographs) at an angle perpendicular to the ground; the images captured by the image capture unit 14 are provided to the calculation unit 15; Step S05, extract feature points: The satellite image processing module 151 extracts and marks the feature points in the image captured by the image capturing unit 14, such as points on the outline of a road, points on the corners of a building, and points on the corners of the water-land interface, which are sufficiently representative features; Step S06, extract satellite image feature points: The satellite image processing module 151 extracts and marks the feature points in the satellite images based on the satellite images in the database 16; In this step S06, the database 16 needs to establish relevant satellite images in advance, storing satellite images of the unmanned vehicle's mission range in the database 16 for query and comparison. Considering that the altitude of the unmanned vehicle's location will affect the scale of the satellite images, as well as the changes in the terrain in different seasons, satellite images collected at multiple scales and in multiple seasons will be stored in the database 16. The satellite images of the mission range will be divided into multiple images, rather than using a single map to cover the entire area. The four corners of the satellite images stored in the database 16 are marked with longitude and latitude coordinates, which are key information for the unmanned vehicle's positioning in subsequent steps. Step S07, Feature Point Pairing: The calculation unit 15 performs feature point pairing between the image with feature points from step S05 and the satellite image with feature points from step S06. Each pairing generates a score representing the degree of matching. If the satellite image with the highest matching score meets the minimum standard for the number of feature point pairings, the position of the unmanned vehicle is inferred to be within the range of this satellite image. Otherwise, if the matching fails, the matching process is repeated until a successful match is achieved. The minimum standard for the number of pairings effectively filters out results with low matching degrees. The number of pairings varies depending on the application scenario. In scenarios requiring high precision, this standard can be increased to strictly filter the pairing results. Step S08, Overlapping Feature Points: After matching the feature points of the satellite image with those of the image, the calculation unit 15 projects the image onto the satellite image so that the corresponding feature points of the two coincide. At this point, the position of the unmanned vehicle in the satellite image can be determined. Step S09, Flight Status Correction: After the satellite image processing module 151 uses satellite images to calculate the longitude and latitude coordinates of the unmanned vehicle when it captures the image, and combines them with the timestamp of the image captured by the image capturing unit 14, and then uses the local motion amount in S03, the processing unit 15 calculates the correction signal. The correction signal is transmitted to the vehicle-end control unit 11. The vehicle-end control unit 11 calculates the subsequent control signal based on the correction signal. The control signal is used to correct the flight status of the unmanned vehicle.

[0032] The steps S01 to S03, S04 and S05, and S06 described above can be performed simultaneously or sequentially.

[0033] In summary, when the vehicle-mounted global satellite positioning signal receiving unit 12 fails to receive a positioning signal, the vehicle-mounted control unit 11 activates the calculation unit 15. The calculation unit 15 uses the measurement values ​​of the inertial measurement unit 13, the images captured by the image capturing unit 14, and the satellite images stored in the database 16 to calculate a correction signal. The vehicle-mounted control unit 11 then calculates the subsequent control signal based on the correction signal, enabling the unmanned vehicle to complete its mission according to the control signal.

[0034] 10: Power Source 11: Vehicle-side control unit 12: Vehicle-side Global Positioning Signal Receiving Unit 13: Inertial Measurement Unit 14: Image Capture Unit 15: Computational Unit 150: Inertial Measurement Calculation Module 151: Satellite Image Calculation Module 16: Database S00-S09: Steps

Claims

1. A mobile device for unmanned vehicles without a satellite navigation system, comprising: a vehicle-end control unit; a vehicle-end global positioning signal receiving unit, signal-connected to the vehicle-end control unit; a calculation unit, signal-connected to the vehicle-end control unit; the calculation unit having an inertial measurement calculation module and a satellite image calculation module, the inertial measurement calculation module being a neural network-like network or a neural network; the satellite image calculation module having a feature extraction algorithm; an inertial measurement unit, signal-connected to the calculation unit; an image capturing unit, signal-connected to the calculation unit; and a database, signal-connected to the calculation unit.

2. The unmanned vehicle without a satellite navigation system as described in claim 1, wherein the inertial measurement unit is a combination of an accelerometer and a gyroscope; and the image acquisition unit is a camera or a video camera.

3. The unmanned vehicle without a satellite navigation system as described in claim 1 further includes a barometer and a magnetic needle, the barometer and the magnetic needle being signal-connected to the calculation unit.

4. The unmanned vehicle without a satellite navigation system as described in claim 1 further includes a power source electrically connected to the vehicle-end control unit.

Images