Aircraft online real-time mapping processing method, system, electronic device and readable storage medium

Abstract

Embodiments of the present application disclose an aircraft online real-time mapping processing method, system, electronic device and readable storage medium, which are applied to the technical field of data processing of aircraft images and comprise the following steps: a ground station controls an aircraft to calculate position information and attitude information, and collects a vertical video of a survey area in real time, and transmits the above data to the ground station in real time; the ground station extracts key frame images from the real-time vertical video, and sends the key frame images to a cloud server after time alignment and matching with the position information and the attitude information; the cloud server calls a preset processing algorithm, realizes fast absolute orientation processing of the key frame images, generates a single image with geographical coordinates, and returns the single image after absolute orientation to a client of the ground station, and performs real-time superimposed display on a network map of the client. The method, system, electronic device and readable storage medium can process images collected by the aircraft online in real time, and improve processing efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing of aircraft images, and particularly relates to an aircraft online real-time mapping processing method and system, an electronic device and a readable storage medium. BACKGROUND

[0002] Unmanned aerial vehicles have been widely used in various fields. The latest progress in unmanned aerial vehicle technology, control technology, large data volume and large bandwidth data transmission technology, high reliability and high performance processing chip technology, and the improvement of cloud cheap computing power make the online splicing technology based on unmanned aerial vehicle streaming media images feasible and practical.

[0003] The image splicing algorithm of a general micro unmanned aerial vehicle is to reconstruct the pose (position and attitude) of the image by using a computer vision motion recovery structure algorithm, and then to perform absolute orientation and splicing processing by using the pose information provided by the high-precision reconstruction algorithm and the photogrammetry geometric principle. The disadvantage is that the algorithm is complex and time-consuming, and all sensor data must be collected at one time before offline centralized processing, and the mapping efficiency is low. SUMMARY

[0004] The application provides an aircraft online real-time mapping processing method, system, electronic device and readable storage medium, which aims to integrate the advantages of existing aircraft, memory, large bandwidth data transmission and cloud server, realize online real-time processing of images, and improve the low efficiency of the existing aircraft image mapping method.

[0005] The application embodiment provides an aircraft online real-time mapping processing method, which comprises the following steps:

[0006] The ground station controls the aircraft to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data, and collect real-time orthographic videos of a survey area, and the position information, the attitude information and the collected real-time orthographic videos are real-time delivered to the ground station;

[0007] The ground station extracts key frame images from the real-time orthographic videos, and performs time alignment and matching with the position information and the attitude information, and sends the aligned and matched position information, attitude information and key frame images to a cloud server;

[0008] The cloud server calls a preset processing algorithm according to the matched position information, attitude information and key frame image, and a received puzzle processing instruction sent by the client of the ground station, realizes fast absolute orientation processing of the key frame image, generates a single image with geographical coordinates, and returns the single image after absolute orientation to the client of the ground station in a network path mapping manner, and performs real-time superimposed display with the network map of the client.

[0009] The embodiment of the application further provides a flight vehicle online real-time mapping processing method, comprising:

[0010] The flight vehicle calculates position information according to acquired positioning data, calculates attitude information according to acquired inertial measurement data, and collects real-time orthographic video of a survey area, and the position information, the attitude information and the collected real-time orthographic video are real-time delivered to the ground station;

[0011] Key frame images are extracted from the real-time orthographic video, and are time-aligned and matched with the position information and the attitude information, and the aligned and matched position information, attitude information and key frame images are sent to a cloud server, so that the cloud server calls a preset processing algorithm according to the aligned and matched position information, attitude information and key frame images, and a received puzzle processing instruction sent by the client of the ground station, realizes fast absolute orientation processing of the key frame image, generates a single image with geographical coordinates, and returns the single image after absolute orientation to the client of the ground station in a network path mapping manner;

[0012] The single image is real-time superimposed displayed in the network map of the client.

[0013] The embodiment of the application further provides a flight vehicle online real-time mapping processing system, comprising a ground station, a flight vehicle and a cloud server.

[0014] The flight vehicle is used for calculating position information according to acquired positioning data, calculating attitude information according to acquired inertial measurement data, and collecting real-time orthographic video of a survey area under the control of the ground station;

[0015] The ground station is used for controlling the flight vehicle to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data, and collect real-time orthographic video of a survey area, and real-time delivering the position information, the attitude information and the collected real-time orthographic video to the ground station;

[0016] The ground station is further configured to extract a key frame image from the real-time orthographic video, and perform time alignment matching with the position information and the attitude information, and send the aligned and matched position information, attitude information and key frame image to the cloud server.

[0017] The cloud server is configured to call a preset processing algorithm according to the aligned and matched position information, attitude information and key frame image, and a received puzzle processing instruction sent by a client of the ground station, to realize fast absolute orientation processing of the key frame image, to generate a single image with geographic coordinates, and to return the absolutely oriented single image to the client of the ground station through network path mapping, and to perform real-time superimposed display with a network map of the client.

[0018] The embodiment of the application further provides an electronic device, and the aerial vehicle includes a memory and a processor; the memory stores executable program code; the processor coupled with the memory invokes the executable program code stored in the memory to implement the online real-time mapping processing method.

[0019] The embodiment of the application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the aerial vehicle online real-time mapping processing method.

[0020] The aerial vehicle fly-by-puzzle method provided by the embodiment of the application calculates position information according to positioning data, calculates attitude information according to inertial measurement data, and collects real-time orthographic video of a survey area, and sends the above information to a ground station in real time, the ground station extracts a key frame image from the real-time orthographic video, and sends the key frame image to a cloud server after time alignment matching with the position information and the attitude information, the cloud server calls a preset processing algorithm to realize fast absolute orientation processing of the key frame image, generates a single image with geographic coordinates, and returns the single image to a client of the ground station through network path mapping, and performs real-time superimposed display with a network map of the client, the calculation method is simple, and the advantages of existing aerial vehicles, memories, large bandwidth data transmission, cloud computing power and the like are integrated, the disadvantage that all sensor data must be collected at one time before being processed offline is overcome, and online real-time mapping processing of the aerial vehicle is realized without offline centralized processing, which not only improves the mapping efficiency of aerial vehicle images, but also improves real-time application, is time-saving and low-cost, and breaks through the traditional aerial survey image processing mode. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only show some embodiments of the present application.

[0022] Figure 1 is a schematic diagram of an application scenario of an aircraft online real-time mapping processing method provided by an embodiment of the present application.

[0023] Figure 2 is a schematic diagram of an implementation process of an aircraft online real-time mapping processing method provided by an embodiment of the present application.

[0024] Figure 3 is Figure 2 is a schematic diagram of a detailed step flow of step S102 in the aircraft online real-time mapping processing method.

[0025] Figure 4 is a schematic diagram of an implementation process of an aircraft online real-time mapping processing method provided by another embodiment of the present application.

[0026] Figure 5 is a schematic diagram of a structure of an aircraft online real-time mapping processing system provided by an embodiment of the present application.

[0027] Figure 6 is a schematic diagram of a structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0029] Referring to Figure 1 , Figure 1The application scenario schematic diagram of the aircraft online real-time mapping processing method provided by the embodiment of the present application is shown in the figure. In the embodiment, the application scenario of the aircraft online real-time mapping processing method can include an aircraft 10, a virtual reference station (Virtual Reference Station) 20, a satellite 30, a ground station 40, and a cloud server 50. In the embodiment, the aircraft 10 includes various unmanned aerial vehicles, preferably micro, light, and small unmanned aerial vehicles, and is provided with a camera 11, an inertial measurement unit (IMU) 12, and a positioning unit 13. The positioning unit 13 can specifically include various positioning systems. Real-time kinematic (RTK) technology or the like can be used to obtain the coordinates of the aircraft 10 in the coordinate system by communicating between the aircraft 10, the virtual reference station 20, and the satellite 30, and then send the collected coordinates, attitude, and video data to the ground station 40. The ground station 40 pre-processes the data, uploads the pre-processing results to the cloud server 50 for processing, which can improve the processing speed and realize the puzzle processing of the aircraft 10 during flight by mutual transmission of data with the ground station 40 and the cloud server 50. For details, refer to the detailed description of each embodiment below.

[0030] In the embodiment, the camera 11 and the inertial measurement unit (IMU) 12 are fixedly connected to the aircraft. The IMU is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object. Generally, an IMU includes three single-axis accelerometers and three single-axis gyroscopes. The IMU in the embodiment can also include other numbers of other types of accelerometers and gyroscopes, without specific limitation. The ground station 40 can be a PC terminal or a mobile phone terminal with a built-in client. The client can be a browser or an APP (application) or other client.

[0031] Referring to Figure 2 , Figure 2 The flowchart of the aircraft online real-time mapping processing method provided by the embodiment of the present application is shown in the figure. The method is suitable for processing images collected by an aircraft. The aircraft is provided with at least a camera, an inertial measurement unit, and a positioning unit. The execution subject of the method is a ground station and a cloud server, as shown in Figure 2 The method mainly includes the following steps:

[0032] S101, the ground station controls the aircraft to calculate position information according to the obtained positioning data, calculate attitude information according to the obtained inertial measurement data, and collect real-time orthographic video of the survey area, and then real-time issues the position information, the attitude information, and the collected real-time orthographic video to the ground station;

[0033] Specifically, the ground station controls the aircraft by collecting real-time orthographic video Vp of the survey area, collecting positioning data such as Global Navigation Satellite System (GNSS) positioning data through a positioning unit, obtaining inertial measurement data through an inertial measurement unit, which can include acceleration, angular velocity and other data that can represent the flight state of the aircraft, and calculating the position information of the unmanned aerial vehicle according to the positioning data, which can be a position sequence Rp, calculating the attitude information according to the inertial measurement data, which can be a starting attitude sequence Op, and transmitting the above information collected and calculated in real time to the ground station through the data communication link between the aircraft and the ground station.

[0034] In this step, the acquisition and real-time calculation of the position and orientation system (POS) data composed of the inertial measurement unit and the positioning unit or the inertial measurement data can be performed.

[0035] In this embodiment, the observation data of the aircraft (i.e., the mobile station) and the calculation configuration information can be used to create a network calculation task, perform virtual reference station data (equivalent to positioning observation data of the ground station) calculation, and based on the calculated virtual reference station and mobile station data, use Real Time Kinematic (RTK) technology to calculate the trajectory of the camera of the aircraft during flight (i.e., Rp), and use IMU integration technology to calculate the high-precision attitude information of the camera (i.e., Op), which continuously records real-time orthographic video data of the survey area (i.e., Vp) during the flight of the aircraft. Since the position of the virtual reference station is determined by the single point positioning solution of the mobile station receiver (which can be a mobile phone with an App or a magic box), the baseline formed by the mobile station and the virtual reference station is usually only a few to tens of meters, and the mobile station and the virtual reference station perform carrier phase difference correction to realize real-time RTK.

[0036] S102, the ground station extracts key frame images from the real-time orthographic video, and time aligns and matches the position information and the attitude information, and sends the aligned and matched position information, attitude information and key frame images to a cloud server;

[0037] The ground station synchronizes the data in time, so as to align the position sequence Rp, the attitude sequence Op and the video data Vp.

[0038] Specifically, referring to Figure 3 , step S102 can be further refined as follows:

[0039] S1021, frame extraction is performed every N seconds on the received real-time orthographic video data to extract the key frame images;

[0040] Every N seconds, a frame is extracted from the real-time orthographic video as the key frame image, and the value of N can be set, which needs to meet the condition that N should guarantee that the adjacent two frames of data extracted have an overlap. In actual application, the value of N can be set according to the user's self-defined operation, or can also be automatically configured by the ground station according to the above condition.

[0041] S1022, align and match the calculated position information, attitude information and extracted key frame image through the synchronized time, complete time synchronization;

[0042] S1023, upload the aligned position information, attitude information and key frame image to the cloud server through the communication link between the ground station and the cloud server.

[0043] S103, the cloud server calls the preset processing algorithm according to the aligned and matched position information, attitude information and key frame image, and the received puzzle processing instruction sent by the client of the ground station, realizes the fast absolute orientation processing of the key frame image, and generates a single image with geographical coordinates;

[0044] The client in the ground station sends the puzzle instruction to the cloud server, and the cloud server performs puzzle processing according to the aligned and matched position information, attitude information and key frame image received from the ground station, and generates a single image with geographical coordinates.

[0045] Specifically, the cloud server background scheduling program opens a thread to listen to the data uploaded by the ground station in real time, and the background reads, identifies and filters the position information, attitude information and key frame image information of the data uploaded by the ground station through the preset algorithm. If redundant data (i.e. data with high overlap) is found, the processing of the redundant data is automatically discarded, and the effective data after filtering is matched with the key frame image and processed by the double algorithm based on the collinear equation, so as to realize the fast absolute orientation processing of the image and generate a single image with geographical coordinates. The collinear equation is an existing algorithm, that is, the geometric relationship among the image point, object point and projection center is used for calculation.

[0046] S104, return the single image after absolute orientation to the client in the ground station through the network path mapping mode, and perform real-time superposition display with the network map of the client.

[0047] The client can be a browser client.

[0048] In the embodiment of the present application, the ground station controls the aircraft to calculate position information according to the positioning data, to calculate attitude information according to the inertial measurement data, and to collect real-time orthographic video of the survey area, and to transmit the above information to the ground station in real time. The ground station extracts key frame images from the real-time orthographic video and sends them to the cloud server after time alignment and matching with the position information and the attitude information. The cloud server calls a preset processing algorithm to realize fast absolute orientation processing of the key frame images, generates a single image with geographic coordinates, and returns the single image to the client of the ground station through network path mapping, and performs real-time overlay display with the network map of the client. The calculation method is simple, integrates the advantages of existing aircraft, memory, large bandwidth data transmission, cloud computer and other devices, overcomes the disadvantage that current UAV image jigsaw must collect all sensor data at one time before offline centralized processing, and realizes online real-time mapping processing of the aircraft, which not only improves the mapping efficiency of the aircraft image, but also improves the real-time application, saves time and cost, and breaks through the traditional aerial image processing mode.

[0049] Referring to Figure 4 , Figure 4 is a flowchart of an aircraft online real-time mapping processing method provided by another embodiment of the present application. The execution subject of the method is a ground station. As shown in Figure 4 , the method mainly includes the following steps:

[0050] S201, controlling the aircraft to calculate position information according to the acquired positioning data, to calculate attitude information according to the acquired inertial measurement data, and to collect real-time orthographic video of the survey area, and to transmit the position information, the attitude information and the collected real-time orthographic video to the ground station in real time;

[0051] Specifically, the aircraft is provided with a camera, an inertial measurement unit and a positioning unit.

[0052] The ground station controls the aircraft to collect real-time orthographic video of the survey area through the camera, to collect global navigation satellite system data through the positioning unit, and to collect the inertial measurement data through the inertial measurement unit;

[0053] The ground station controls the aircraft to calculate the attitude information according to the inertial measurement data, and to calculate the position information using real-time dynamic difference technology according to the global navigation satellite system data and the data of the virtual reference station;

[0054] The calculated attitude information, position information and real-time orthographic video are transmitted to the ground station through the data communication link between the aircraft and the ground station.

[0055] S202, extract the key frame image from the real-time orthographic video, and time align and match the position information and the attitude information, and send the aligned and matched position information, attitude information and key frame image to the cloud server;

[0056] Specifically, the received real-time orthographic video data is extracted every N seconds, the key frame image is extracted, the position information, the attitude information and the key frame image are aligned and matched through the synchronized time, the aligned position information, the attitude information and the key frame image are uploaded to the cloud server through the communication link between the ground station and the cloud server, so that the cloud server calls a preset processing algorithm according to the aligned and matched position information, attitude information and key frame image, and a received puzzle processing instruction sent by the client of the ground station, realizes fast absolute orientation processing of the key frame image, generates a single image with geographical coordinates, and returns the absolute oriented single image to the client of the ground station through network path mapping.

[0057] S203, the ground station displays the single image in real time on the network map of the client.

[0058] Other details in the embodiment are described in the foregoing Figure 2 and Figure 3 The description in the embodiments shown in the foregoing.

[0059] In the embodiment of the application, the ground station controls the aircraft to calculate the position information according to the positioning data, calculates the attitude information according to the inertial measurement data, and collects the orthographic video of the survey area in real time, and the above information is sent to the ground station in real time. The ground station extracts the key frame image from the real-time orthographic video, and sends it to the cloud server after time aligning and matching with the position information and the attitude information, so that the cloud server calls a preset processing algorithm to realize fast absolute orientation processing of the key frame image, generates a single image with geographical coordinates, and returns the single image to the client of the ground station through network path mapping, and displays the single image in real time on the network map of the client. The calculation method is simple, and the advantages of existing aircraft, memory, large bandwidth data transmission, cloud computer and other devices are integrated. There is no need to process data offline, so as to realize online real-time mapping processing of the aircraft. Not only can the mapping efficiency of the aircraft image be improved, but also the real-time application can be improved, the time consumption is short, the cost is low, and the traditional aerial survey image processing mode is broken through.

[0060] Referring to Figure 5 , Figure 5 is a structural schematic diagram of the aircraft online real-time mapping processing system provided by the embodiment of the application. Only parts related to the embodiment of the application are shown for convenience of description. Figure 5The example aerial online real-time mapping processing system can be the aforementioned Figure 2 The aerial online real-time mapping processing method provided by the embodiment is executed by a ground station and a cloud server, and the aerial online real-time mapping processing system includes the ground station 40, the aerial vehicle 10, and the cloud server 50 (the reference signs of the embodiment are continued Figure 1 ).

[0061] The aerial vehicle 10 is configured to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data, and collect orthographic videos of a survey area in real time under the control of the ground station 40.

[0062] The ground station 40 is configured to control the aerial vehicle 10 to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data, and collect orthographic videos of a survey area in real time, and to transmit the position information, the attitude information, and the collected real-time orthographic videos to the ground station 40 in real time.

[0063] The ground station 40 is further configured to extract key frame images from the real-time orthographic videos, and to perform time alignment and matching with the position information and the attitude information, and to transmit the aligned and matched position information, attitude information, and key frame images to the cloud server 50.

[0064] The cloud server 50 is configured to perform fast absolute orientation processing on the key frame images according to the aligned and matched position information, attitude information, and key frame images, and a received puzzle processing instruction sent by a client of the ground station 40, to generate a single image with geographic coordinates, and to return the single image after absolute orientation to the client of the ground station 40 in a network path mapping manner, and to perform real-time superimposed display with a network map of the client.

[0065] Further, the aerial vehicle 10 is provided with a camera, an inertial measurement unit, and a positioning unit.

[0066] The ground station 40 is further configured to control the aerial vehicle 10 to collect real-time orthographic videos of a survey area by the camera, to collect global navigation satellite system data by the positioning unit, and to collect the inertial measurement data by the inertial measurement unit, to control the aerial vehicle 10 to calculate the attitude information according to the inertial measurement data, and to calculate the position information using real-time dynamic difference technology according to the global navigation satellite system data and data of a virtual reference station, and to transmit the calculated attitude information, position information, and real-time orthographic videos to the ground station 40 through a data communication link between the aerial vehicle 10 and the ground station 40.

[0067] The ground station 40 is also used for extracting a key frame image from the received real-time orthographic video data every N seconds, aligning and matching the position information, the attitude information and the key frame image through the synchronized time, and uploading the aligned position information, the attitude information and the key frame image to the cloud server 50 through a communication link between the ground station 40 and the cloud server 50.

[0068] The cloud server 50 is also used for opening a thread through a background scheduler, listening to the aligned position information, the attitude information and the key frame image uploaded by the ground station 40 in real time, reading, identifying and redundantly filtering the aligned position information, the attitude information and the key frame image according to the preset processing algorithm, the aligned position information, the attitude information and the key frame image uploaded by the ground station 40, and the puzzle processing instruction issued by the client in the ground station 40, and performing key frame image matching and double algorithm processing based on a collinearity equation on the filtered effective data, to realize fast absolute orientation processing of the image and generate a single image with geographical coordinates.

[0069] For other details of the embodiment, refer to the foregoing Figures 2 to 4 description of the illustrated embodiment.

[0070] In the embodiment, the ground station controls the aircraft to calculate position information according to positioning data, calculate attitude information according to inertial measurement data, and collect orthographic video of a survey area in real time, and the above information is issued to the ground station in real time. The ground station extracts a key frame image from the real-time orthographic video, and sends the key frame image to the cloud server after time alignment and matching with the position information and the attitude information. The cloud server calls a preset processing algorithm to realize fast absolute orientation processing of the key frame image, generate a single image with geographical coordinates, and return the single image to the client of the ground station through a network path mapping manner, and perform real-time superimposed display with a network map of the client. The calculation method is simple, integrates the advantages of existing aircrafts, memories, large-bandwidth data transmission, cloud computers and other devices, does not need to process data offline, realizes real-time mapping processing of the aircraft online, can improve the mapping efficiency of the aircraft image, and can improve real-time application, has short time consumption and low cost, and breaks through the traditional aerial survey image processing mode.

[0071] For Figure 6 the foregoing, the embodiment of the present application also provides an electronic device which can be the foregoing ground station. The electronic device comprises a memory 100 and a processor 200. The memory 100 stores executable program codes. The processor 200 coupled with the memory 100 calls the executable program codes stored in the memory 100 to perform the foregoing Figure 4The memory 100 can be a high-speed random access memory (RAM) or a non-volatile memory such as a disk memory.

[0072] The embodiment of the present application also provides a UAV, which is used for calculating position information according to positioning data obtained by a positioning unit, calculating attitude information according to inertial measurement data obtained by an inertial measurement unit, and collecting a real-time orthographic video of a survey area by a camera under the control of a ground station, and real-time transmitting the position information, the attitude information and the collected real-time orthographic video to the ground station, so as to realize the above-mentioned method in cooperation with the ground station and a cloud server. Figures 2 to 3 The UAV online real-time mapping processing method described in the embodiment.

[0073] Further, the embodiment of the present application also provides a computer readable storage medium, which can be arranged in the ground station, and the computer readable storage medium can be a memory of the ground station. The computer readable storage medium stores a computer program, and the program is executed by a processor to realize the above-mentioned method. Figure 4 The UAV online real-time mapping processing method described in the embodiment.

[0074] Further, the computer readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and various media capable of storing program codes.

[0075] It should be noted that, for the above-mentioned method embodiments, in order to simplify the description, the above-mentioned method embodiments are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0076] In the above-mentioned embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0077] The above is the description of the aircraft online real-time mapping processing method, the aircraft online real-time mapping processing system, the electronic device and the computer readable storage medium provided by the application. For those skilled in the art, according to the idea of the embodiment of the application, the specific implementation and the application range will be changed. In summary, the content of the specification should not be understood as a limitation of the application.

Claims

1. An aircraft on-line real-time mapping processing method, characterized in that, The ground station controls the aircraft to calculate position information according to acquired positioning data, to calculate attitude information according to acquired inertial measurement data, and to collect real-time orthographic video of the survey area, and to transmit the position information, the attitude information and the collected real-time orthographic video to the ground station in real time. The ground station extracts key frame images by frame extraction every N seconds from the real-time orthographic video, and performs time alignment and matching of the extracted key frame images with the position information and the attitude information, and transmits the aligned and matched position information, attitude information and key frame images to the cloud server, wherein the value of N satisfies the condition that the adjacent two frames of extracted data have overlapping degree. The cloud server performs fast absolute orientation processing on the key frame images according to the aligned and matched position information, attitude information and key frame images, and a received puzzle processing instruction sent by the client of the ground station, generates a single image with geographic coordinates, and returns the absolute oriented single image with geographic coordinates to the client of the ground station through network path mapping, and performs real-time superimposed display with the network map of the client. The aircraft is provided with a camera, an inertial measurement unit and a positioning unit, the ground station controls the aircraft to calculate position information according to acquired positioning data, to calculate attitude information according to acquired inertial measurement data, and to collect real-time orthographic video of the survey area, and to transmit the position information, the attitude information and the collected real-time orthographic video of the survey area to the ground station in real time.

2. The method of claim 1, wherein, The ground station controls the aircraft to collect real-time orthographic video of the survey area through the camera, to collect global navigation satellite system data through the positioning unit, and to collect inertial measurement data through the inertial measurement unit. The ground station controls the aircraft to calculate the attitude information according to the inertial measurement data, and to calculate the position information using real-time kinematic differential technology according to the global navigation satellite system data and the data of the virtual reference station. The calculated attitude information, position information and real-time orthographic video are transmitted to the ground station through the data communication link between the aircraft and the ground station. The time alignment and matching of the extracted key frame images with the position information and the attitude information, and the transmission of the aligned and matched position information, attitude information and key frame images to the cloud server include:

3. The method according to claim 1 or 2, characterized in that, The position information, the attitude information and the key frame images are aligned and matched through the synchronized time. The aligned position information, attitude information and key frame images are uploaded to the cloud server through the communication link between the ground station and the cloud server. The cloud server performs fast absolute orientation processing on the key frame images according to the aligned and matched position information, attitude information and key frame images, and a received puzzle processing instruction sent by the client of the ground station, generates a single image with geographic coordinates, and returns the absolute oriented single image with geographic coordinates to the client of the ground station through network path mapping, and performs real-time superimposed display with the network map of the client.

4. The method of claim 3, wherein, ​ The background scheduler of the cloud server opens a thread to listen to the position information, attitude information and key frame image uploaded by the ground station in real time. The cloud server uses the preset processing algorithm to read, identify and filter the position information, attitude information and key frame image uploaded by the ground station according to the position information, attitude information and key frame image uploaded by the ground station, and to perform key frame image matching and double algorithm processing based on the collinear equation on the effective data after filtering, so as to realize fast absolute orientation processing of the image and generate a single image with geographical coordinates.

5. An aircraft on-line real-time mapping processing method, characterized in that, The method is applied to the ground station and includes: The aircraft calculates position information according to acquired positioning data, calculates attitude information according to acquired inertial measurement data, and collects real-time orthographic video of the measurement area, and transmits the position information, attitude information and collected real-time orthographic video to the ground station in real time; The key frame image is extracted by frame extraction every N seconds, and the extracted key frame image is time-aligned and matched with the position information and attitude information, and the aligned and matched position information, attitude information and key frame image are sent to the cloud server, so that the cloud server calls the preset processing algorithm according to the aligned and matched position information, attitude information and key frame image and the received puzzle processing instruction sent by the client of the ground station, realizes fast absolute orientation processing of the key frame image, generates a single image with geographical coordinates, and returns the absolute oriented single image with geographical coordinates to the client of the ground station through network path mapping. The single image is displayed in real time on the network map of the client.

6. The method of claim 5, wherein, The aircraft is provided with a camera, an inertial measurement unit and a positioning unit, and the aircraft calculates position information according to acquired positioning data, calculates attitude information according to acquired inertial measurement data, and collects real-time orthographic video of the measurement area, and transmits the position information, attitude information and collected real-time orthographic video of the measurement area to the ground station in real time. The ground station controls the aircraft to collect real-time orthographic video of the measurement area through the camera, to collect global navigation satellite system data through the positioning unit, and to collect inertial measurement data through the inertial measurement unit. The ground station controls the aircraft to calculate the attitude information according to the inertial measurement data, and to calculate the position information using real-time kinematic differential technology according to the global navigation satellite system data and the data of the virtual reference station. The calculated attitude information, position information and real-time orthographic video are transmitted to the ground station through the data communication link between the aircraft and the ground station.

7. The method according to claim 5 or 6, characterized in that, The extracted key frame image is time-aligned and matched with the position information and the attitude information, and the aligned and matched position information, attitude information and key frame image are sent to a cloud server, which comprises: aligning and matching the position information, the attitude information and the key frame image through the synchronized time; uploading the aligned position information, the attitude information and the key frame image to the cloud server through a communication link between the ground station and the cloud server.

8. An airborne on-line real-time mapping processor system, characterized by comprise: a ground station, an aircraft and a cloud server; the aircraft is configured to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data and collect orthographic video of a survey area in real time under the control of the ground station; the ground station is configured to control the aircraft to calculate position information according to acquired positioning data, calculate attitude information according to acquired inertial measurement data and collect orthographic video of a survey area in real time, and to send the position information, the attitude information and the collected real-time orthographic video to the ground station in real time; the ground station is further configured to extract key frame images by frame extraction of the real-time orthographic video every N seconds, time-align and match the extracted key frame images with the position information and the attitude information, and send the aligned and matched position information, attitude information and key frame images to the cloud server, wherein the value of N satisfies the condition that the adjacent two frames of extracted data should have overlapping degree; the cloud server is configured to call a preset processing algorithm according to the aligned and matched position information, attitude information and key frame image, and a received puzzle processing instruction sent by a client of the ground station, to realize fast absolute orientation processing of the key frame image, to generate a single image with geographic coordinates, and to return the absolute oriented single image with geographic coordinates to the client of the ground station through network path mapping, and to perform real-time superimposed display with a network map of the client.

9. An electronic device, comprising: The electronic device comprises: a memory and a processor; the memory stores executable program code; the processor coupled with the memory invokes the executable program code stored in the memory to execute the aircraft online real-time mapping processing method of any one of claims 5 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the aircraft online real-time mapping processing method of any one of claims 5 to 7.

Images