A preprocessing method based on full waveform data of airborne lidar

Abstract

This invention provides a method for preprocessing full-waveform data from an airborne laser radar, relating to the field of data processing technology. The method comprises the following steps: calculating the number of sampling points based on the operating altitude of the drone and determining a specified data range to be captured; initializing the configuration of an acquisition card; starting a data acquisition loop, waiting for a hardware trigger, and capturing raw data; dynamically capturing a specified range in the data buffer; writing the data to a file and storing the file in segments. When the number of detection records is greater than or equal to N, the current file is closed and a new one is created. The technical solution of the invention overcomes the problems of low efficiency and low real-time performance in the prior art for full-waveform data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a full-waveform data preprocessing method based on an airborne laser radar. Background Art

[0002] The waveform of a full-waveform lidar (LIDAR) refers to the energy of a laser pulse transmitted or received over time. A full-waveform LIDAR system samples the intensity of backscattered echo pulses reflected from contact and interaction with a target at small intervals, digitally quantifies the intensity, and records the intensity of the backscattered pulses. Full-waveform LIDAR samples both the transmitted pulse and the backscattered echo pulse at small intervals. The resulting raw data is a two-dimensional set of data with the sampling points on the horizontal axis and the echo intensity on the vertical axis, capturing virtually the entire backscattered echo waveform.

[0003] Due to the limited flight altitude of airborne lidar, its operating range is generally within 50-100m at low altitude. The water depth measurement range is generally limited to 0-50m, and in most cases, 0-5m in land and shallow waters. Its data is saved as .out files and converted into decimal data using code to display some data graphs. A one-minute data set contains 300,000 rows of data, and each row has 16,384 columns. At present, the amount of full waveform data of airborne lidar is very large, and it can collect 9.6GB of data per minute. Each flight generally lasts 20-30 minutes, and collects data for more than ten minutes. The data volume fluctuates depending on the acquisition card model and the characteristics of the collected ground objects, but the data volume is still very large.

[0004] The sampling rate of the high-speed acquisition card is 10GHz. Taking a set of near-point scanning data as an example, the effective data of the echo waveform is concentrated in a certain part of each waveform acquired, which depends on the distance from the laser to the target. However, each waveform has many invalid signals. If they are not processed during acquisition, a large amount of invalid data will be collected together with the original data, thereby increasing the data volume of the original data.

[0005] Existing methods for extracting effective vibration data based on waveform feature statistics use the waveform characteristics of vibration signals as a basis. By statistically analyzing the waveform characteristics of the collected data, noise peaks are filtered out and the vibration signal is intercepted. This method first collects and stores a large amount of raw full-waveform data, then performs a post-processing operation on the data. Copying large amounts of data increases the computer processing burden and reduces processing efficiency, while the amount of collected data itself is not effectively reduced. Subsequently, effective data is extracted based on the waveform characteristics. This entire process is a post-processing operation after the data is obtained, which reduces real-time performance.

[0006] Therefore, a full-waveform data preprocessing method based on airborne lidar with high processing efficiency and real-time performance is needed. Summary of the Invention

[0007] The main purpose of the present invention is to provide a full-waveform data preprocessing method based on airborne lidar to solve the problems of low efficiency and low real-time performance of full-waveform data processing in the existing technology.

[0008] To achieve the above object, the present invention provides a method for preprocessing full waveform data of an airborne laser radar, which specifically includes the following steps:

[0009] S1, calculates the number of sampling points according to the flight altitude of the UAV during operation and determines the interception of the specified data range.

[0010] S2, initialize the acquisition card configuration.

[0011] S3, start the data acquisition cycle, wait for the hardware trigger time, and collect raw data.

[0012] S4, dynamically intercept the specified range in the data buffer.

[0013] S5, write the file and store the file in segments. When the number of detection records is greater than or equal to N, close the current file and create a new file.

[0014] Furthermore, step S1 specifically includes the following steps:

[0015] S1.1, calculate the distance interval of sampling points :

[0016] ;

[0017] ;

[0018] Among them, the speed of light 3× m / s, is the sampling rate, is the sampling period.

[0019] S1.2, calculate the maximum acquisition distance of the acquisition card :

[0020] ;

[0021] The total sampling time =N× , N is the total number of sampling points.

[0022] S1.3, the scanning angle of the rotating mirror is , actual collection distance ,in, is the flight altitude of the UAV, and the relationship between the actual collection distance and the maximum collection distance is:

[0023] .

[0024] S1.4, the actual number of sampling points is :

[0025] ;

[0026] in, is the actual acquisition distance, is the sampling point distance interval.

[0027] Furthermore, step S2 specifically includes the following steps:

[0028] S2.1, calculate the actual number of sampling points based on the actual UAV working altitude , and according to Set the intercept point parameters.

[0029] S2.2, configure the acquisition card parameters, including: sampling rate, trigger mode and original record length.

[0030] Further, in step S2.1, according to The specific parameters for setting the interception point are: setting the interception starting point START_SAMPLE and the number of sampling points SAMPLES_TO_SAVE for sampling interception.

[0031] Furthermore, step S3 specifically includes: responding to an external trigger signal and collecting complete waveform data.

[0032] Furthermore, step S4 is specifically as follows:

[0033] S4.1, check whether the interception starting point exceeds the total number of sampling points, calculate the theoretical end point, and compare it with the buffer boundary to determine whether the end point is out of bounds. If it is out of bounds, it is automatically adjusted to the valid range.

[0034] Calculate the interception end point end:

[0035] end=min(START_SAMPLE+SAMPLES_TO_SAVE,total_samples);

[0036] Among them, total_samples is the total number of sampling points, and min is the minimum value function.

[0037] If the interception start point exceeds the buffer range, it is reset to 0 and the record is discarded. If the interception end point exceeds the buffer range, it is intercepted to the end of the buffer.

[0038] S4.2: When data spans the Extradata and current buffer, the intercepted range is concatenated and the effective data length of each segment is calculated, including the available length of the Extradata and the length to be supplemented in the current buffer. The data intercepted from the Extradata and the data supplemented from the current buffer are then written in segments, with priority given to intercepting from the Extradata buffer and the remaining part supplemented from the current buffer.

[0039] S4.3, dynamically calculate the actual interception length and determine the final length based on the boundary protection result to ensure that the actual interception length does not exceed SAMPLES_TO_SAVE.

[0040] Furthermore, step S5 is specifically as follows:

[0041] S5.1, use the global counter to count the number of records written to the current file; after accumulating N=300,000 intercepted data, close the current file and generate a new file.

[0042] S5.2, the file name is named by channel and time, and the header file retains the original acquisition naming rule by time;

[0043] The file size is:

[0044] 300000 × SAMPLES_TO_SAVE × 2 bytes.

[0045] The present invention has the following beneficial effects:

[0046] 1. The present invention adopts a pre-processing method rather than a post-processing method, which greatly increases the real-time performance of the acquisition program.

[0047] 2. Directly operate the original data in the cache to avoid dynamic memory allocation and frequent writing and writing, improve processing efficiency, and effectively reduce the overall data size.

[0048] 3. By configuring parameters in advance, the calculation complexity in the algorithm is reduced and the entire process is simplified.

[0049] 4. Supports seamless interception across buffers to ensure data integrity under high-frequency triggering, avoid memory out-of-bounds and invalid access, and improve system robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0051] Figure 1 A flow chart of a method for preprocessing full waveform data of an airborne laser radar according to the present invention is shown.

[0052] Figure 2 The waveform diagram after the data is directly preprocessed is shown.

[0053] Figure 3 The waveform diagram after data preprocessing using the method provided by the present invention is shown. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 The method for preprocessing full waveform data of an airborne laser radar shown in FIG. 1 includes the following steps:

[0057] S1, calculates the number of sampling points according to the flight altitude of the UAV during operation and determines the interception of the specified data range.

[0058] S2, initialize the acquisition card configuration.

[0059] S3, start the data acquisition cycle, wait for the hardware trigger time, and collect raw data.

[0060] S4, dynamically intercept the specified range in the data buffer.

[0061] S5, write the file and store the file in segments. When the number of detection records is greater than or equal to N, close the current file and create a new file.

[0062] Specifically, step S1 includes the following steps:

[0063] S1.1, calculate the distance interval of sampling points :

[0064] ;

[0065] ;

[0066] Among them, the speed of light 3× m / s, is the sampling rate, is the sampling period.

[0067] S1.2, calculate the maximum acquisition distance of the acquisition card :

[0068] ;

[0069] The total sampling time =N× , N is the total number of sampling points.

[0070] S1.3, the scanning angle of the rotating mirror is , actual collection distance ,in, is the flight altitude of the UAV, and the relationship between the actual collection distance and the maximum collection distance is:

[0071] .

[0072] S1.4, the actual number of sampling points is ,

[0073] ;

[0074] in, is the actual acquisition distance, is the sampling point distance interval.

[0075] Specifically, step S2 includes the following steps:

[0076] S2.1, calculate the actual number of sampling points based on the actual UAV working altitude , and according to Set the intercept point parameters.

[0077] S2.2, configure the acquisition card parameters, including basic sampling information such as sampling rate, trigger mode (external trigger), and original record length (16384 points).

[0078] Specifically, in step S2.1, according to The specific parameters for setting the interception point are: setting the interception starting point START_SAMPLE and the number of sampling points SAMPLES_TO_SAVE for sampling interception.

[0079] Specifically, step S3 is as follows: responding to an external trigger signal and collecting complete waveform data.

[0080] Specifically, step S4 is as follows:

[0081] S4.1, boundary protection, checks whether the interception starting point exceeds the total number of sampling points, calculates the theoretical end point, and compares it with the buffer boundary to determine whether the end point is out of bounds. If it is out of bounds, it is automatically adjusted to the valid range;

[0082] Calculate the interception end point end:

[0083] end=min(START_SAMPLE+SAMPLES_TO_SAVE,total_samples);

[0084] Among them, total_samples is the total number of sampling points, and min is the minimum value function.

[0085] If the interception start point exceeds the buffer range, it is reset to 0 and the record is discarded. If the interception end point exceeds the buffer range, it is intercepted to the end of the buffer.

[0086] S4.2, data merging. When data spans Extradata and the current buffer, the intercepted range is spliced and the effective data length of each segment is calculated, including the available length of Extradata and the length to be supplemented in the current buffer. Then, the data intercepted from Extradata and the data supplemented from the current buffer are written in segments, with priority given to intercepting from the Extradata buffer and the remaining part supplemented from the current buffer.

[0087] S4.3, length correction, dynamically calculates the actual intercepted length and determines the final length based on the boundary protection result to ensure that the actual intercepted length does not exceed SAMPLES_TO_SAVE.

[0088] Specifically, step S5 is as follows:

[0089] S5.1, use the global counter (file_record_count) to count the number of records written to the current file;

[0090] After accumulating N=300,000 intercepted data, close the current file and generate a new file.

[0091] S5.2, the file name is named by channel and time (such as dataA-0715-153045123.out), and the header file retains the original acquisition naming rule by time (such as headerA-0715-153045123.out);

[0092] The file size is:

[0093] 300000 × SAMPLES_TO_SAVE × 2 bytes;

[0094] The original trigger timestamp and record length are retained in the header file and are not modified by the interception operation.

[0095] Therefore, the file data volume is reduced from storing 16384 sampling points per line to storing SAMPLES TO SAVE sampling points per line, and the data volume is reduced by 16384 / SAMPLES TO SAVE times.

[0096] Example 2

[0097] The full waveform data was collected using a TELEDYNE SP DEVICES ADQ7DC acquisition card with a sampling frequency of 10 GHz. The ADC sampling rate was set to 16834 Hz in the system. The data, originally collected every minute at 300,000 rows and 16,384 columns, was streamed and saved as a 9.6 GB .out file. Due to the operating conditions of the airborne lidar, valid data is primarily concentrated in a certain portion of the complete sampled waveform (depending on the actual sampling distance). The valid data portion of the complete waveform data was captured by data interception. The specific method was to determine the number of sampling points based on the drone's flight altitude, the relationship between the laser scanning angle and the measured target, and the sampling frequency. The specific formula is as follows:

[0098] Calculate the distance between each sampling point (spatial resolution):

[0099] The time interval between each sampling point is the sampling period: , the laser round trip time is , so the sampling point distance interval is:

[0100] ;

[0101] The speed of light 3× m / s, sampling rate =10GHz, substituting into the formula we get:

[0102] =1.5cm;

[0103] At this sampling rate, the maximum distance the acquisition card can collect data is:

[0104]

[0105] Where the total sampling time = × .

[0106] Because the drone works at the same height during flight and the scanning angle of the rotating mirror is fixed , actual collection distance ,in is the flight altitude of the UAV, and the relationship between the actual collection distance and the maximum collection distance is:

[0107] ;

[0108] The laser scanning angle is =7.5°, that is, when the UAV working altitude is known, the actual collection distance range can be calculated, but it cannot exceed .

[0109] If the number of sampling points is 16384, then =245.76m, that is, all points within the range of 245.76m from the ground point to the laser can be collected. If the UAV-mounted laser radar works at an altitude of 100m, the actual collection distance is =100.86m.

[0110] The working height will not be changed during the operation. The depth of the shallow water area is 5-10m. The effective data of the original waveform is mainly concentrated in the middle part. The sampling point of the radar can be adjusted to 8192 according to the working height of the UAV to reduce the data volume by half. =122.88m, valid data is mainly concentrated in the range of 100±5m, plus environmental factors may cause fluctuations within the range of 5m. According to calculations, the valid waveform is concentrated in the sampling point range of 6666±333. If the influence of drone vibration is considered, the angle offset is 7.5°±2.5°, then the valid data is concentrated in the range of 100.38m-101.54m (that is, the sampling points are concentrated in the range of 6692-6769). The working state of the acquisition card is continuous trigger acquisition, and the frequency is 5000hz.

[0111] The method provided by the present invention first collects data by an acquisition card. Each time the collected data is triggered and stored in a buffer area but has not yet been written to a hard disk, the buffer area data is directly processed. Before flight, a partial interception parameter is manually set, and a portion of the buffer area sampling points 5500 to 8000 is intercepted and cached data is stored in the hard disk, and the remaining portion is discarded. This operation effectively reduces the overall data size of the original data and is more efficient. There is no need to copy the data to increase the processing input and output volume, and the sampling speed will not be reduced. The original data is directly operated to avoid dynamic memory allocation. This preprocessing is completed during the acquisition process, which is conducive to efficient subsequent data processing.

[0112] In order to verify the method provided by the present invention, a comparative experiment was conducted. Under the same collection time, the experimental results are as follows: Figure 2 and Figure 3 shown. Figure 2The original waveform is extracted after direct preprocessing of one row of data in the 9.6 GB original data file, and the acquisition time is the same. Figure 2 The amount of data in the image is very large. After preprocessing, the effective waveforms are mainly concentrated around the sampling point position of 1500, while the other waveforms are invalid information. Figure 3 The data file size is reduced to 600MB. The effective information is the same, and the data volume is reduced by about 16 times. The data of 1024 sampling points between 1024 and 2048 are intercepted. Figure 1 The noise in the front section and the large amount of invalid data in the back section are cut off, and only the valid data components are collected and retained.

[0113] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the scope of protection of the present invention.

Claims

1. A method for preprocessing full waveform data of airborne laser radar, characterized in that: The specific steps include: S1, calculates the number of sampling points according to the flight altitude of the UAV during operation and determines the interception of the specified data range; S2, initialize the acquisition card configuration; S3, start the data acquisition cycle, wait for the hardware trigger time, and collect raw data; S4, dynamically intercepting a specified range in the data buffer; S5, write the file and store it in segments. When the number of detection records is greater than or equal to N, close the current file and create a new one; Step S1 specifically includes the following steps: S1.1, calculate the distance interval of sampling points : ； ； Among them, the speed of light 3× m / s, is the sampling rate, is the sampling period; S1.2, calculate the maximum acquisition distance of the acquisition card : ； The total sampling time =N× , N is the total number of sampling points; S1.3, the scanning angle of the rotating mirror is , actual collection distance ,in, is the flight altitude of the UAV, and the relationship between the actual collection distance and the maximum collection distance is: ； S1.4, the actual number of sampling points is , ； in, is the actual acquisition distance, is the distance interval between sampling points; Step S4 is specifically as follows: S4.1, check whether the interception starting point exceeds the total number of sampling points, calculate the theoretical end point, and compare it with the buffer boundary to determine whether the end point is out of bounds. If it is out of bounds, automatically adjust it to the valid range; Calculate the interception end point end: end=min(START_SAMPLE+SAMPLES_TO_SAVE,total_samples); Among them, total_samples is the total number of sampling points, min is the minimum value function; If the interception start point exceeds the buffer range, it is reset to 0 and the record is discarded. If the interception end point exceeds the buffer range, it is intercepted to the end of the buffer. S4.2, when data spans the Extradata and current buffer, concatenate the intercepted ranges, calculate the effective data length of each segment, including the available Extradata length and the length to be supplemented in the current buffer, and then write the data intercepted from the Extradata and the data supplemented from the current buffer in segments, giving priority to intercepting from the Extradata buffer and supplementing the remaining part from the current buffer; S4.3, dynamically calculate the actual interception length and determine the final length based on the boundary protection result to ensure that the actual interception length does not exceed SAMPLES_TO_SAVE.

2. The method for preprocessing full waveform data of an airborne laser radar according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1, calculate the actual number of sampling points based on the actual UAV working altitude , and according to Set the intercept point parameters; S2.2, configure the acquisition card parameters, including: sampling rate, trigger mode and original record length.

3. The method for preprocessing full waveform data of an airborne laser radar according to claim 2, characterized in that: In step S2.1, The specific parameters for setting the interception point are: setting the interception starting point START_SAMPLE and the number of sampling points SAMPLES_TO_SAVE for sampling interception.

4. The method for preprocessing full waveform data of an airborne laser radar according to claim 1, characterized in that: Step S3 specifically includes: responding to an external trigger signal and collecting complete waveform data.

5. The method for preprocessing full waveform data of an airborne laser radar according to claim 1, characterized in that: Step S5 is specifically as follows: S5.1, use the global counter to count the number of records written to the current file; after accumulating N=300,000 intercepted data, close the current file and generate a new file; S5.2, the file name is named by channel and time, and the header file retains the original acquisition naming rule by time; The file size is: 300000 × SAMPLES_TO_SAVE × 2 bytes.

Images