Aircraft flight data accident investigation inversion method based on video image data

Abstract

This invention discloses a method for inverting aircraft flight data accident investigations based on video image data. The method includes: S1, selecting two consecutive video images P1 and P2 from the video stream data, obtaining the measured latitude, longitude, elevation, and distance from the camera position to reference objects A and B, and obtaining the conversion coefficient between the pixel distance in the video image and the measured distance; S2, constructing a geographic coordinate system with the camera position as the origin and the ground as the coordinate plane, converting and processing the data using the conversion coefficient, and uniformly expressing it on the geographic coordinate system, calculating the geographic coordinate information of aircraft position C and aircraft position C1 respectively, fitting a line to form the inferred trajectory, constructing a flight dataset, storing the inferred trajectory containing geographic coordinate information in the flight dataset, and outputting it. This invention can invert and obtain important flight data such as the trajectory, speed, and altitude of the aircraft involved in the accident based on video stream data, providing accurate and reliable flight data support for aircraft accident investigations.

Description

Technical Field

[0001] This invention relates to the field of flight data inversion and acquisition in aircraft accident investigation, and particularly to a method for aircraft flight data accident investigation and inversion based on video image data. Background Technology

[0002] Aircraft (such as helicopters, light helicopters, or drones) lack flight data recorders (like the flight data recorders on commercial airliners, commonly known as black boxes). Unlike commercial airliners, flight data such as speed, altitude, and trajectory cannot be extracted from flight data recorders during accident investigations. Aircraft accident investigations are typically conducted after a crash, and the lack of crucial flight data like speed, altitude, and trajectory makes post-crash investigations extremely difficult. While ground-based eyewitness cameras record video footage of the aircraft from its abnormal attitude in the air to the crash, current accident investigation methods primarily rely on this video data for superficial or auxiliary analysis. However, the accurate inversion of crucial flight data such as speed, altitude, and trajectory remains a significant challenge, severely restricting the implementation of aircraft accident investigations. Therefore, how to accurately invert and obtain crucial flight data such as speed, altitude, and trajectory is currently a key problem in aircraft accident investigations. Summary of the Invention

[0003] The purpose of this invention is to provide a method for investigating and retrieving aircraft flight data based on video image data. Based on video stream data, it is possible to retrieve important flight data such as the trajectory, speed, and altitude of the aircraft involved in the accident, solving a key problem in aircraft accident investigation. This method provides accurate and reliable flight data support for aircraft accident investigation, facilitating the investigation and improving the capability, efficiency, and credibility of aircraft accident investigation.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A method for investigating and retrieving aircraft flight data based on video image data, the method comprising:

[0006] S1. Obtain video stream data of the same camera position of the aircraft before the accident. Filter the video stream data into two video images, P1 and P2. The lines connecting the camera position and the aircraft position in both video images contain reference objects A and B located on the ground. Obtain the measured latitude, longitude, elevation, and distance from the camera position to reference objects A and B, and obtain the conversion factor between the pixel distance in the video image and the measured distance. The line connecting the camera position Q and the aircraft position C in video image P1 is denoted as line L1, and the line connecting the camera position Q and the aircraft position C1 in video image P2 is denoted as line L2.

[0007] S2. Construct a geographic coordinate system with the camera position as the origin and the ground as the coordinate plane. Convert the straight lines L1 and L2 with the reference objects A and B using conversion factors and then express them uniformly on the geographic coordinate system. Calculate the geographic coordinate information of the aircraft position C and the aircraft position C1 respectively and fit a line to form the predicted trajectory. The geographic coordinate information includes latitude, longitude and elevation. Construct a flight dataset, store the predicted trajectory containing geographic coordinate information in the flight dataset and output it.

[0008] To better achieve the present invention, the present invention also includes the following methods:

[0009] S3. Extract the time interval t between the two consecutive video images P1 and P2, and the distance between the aircraft position C and the aircraft position C1. According to the formula Calculated ground speed Stored in the flight dataset.

[0010] Furthermore, the present invention also includes the following method:

[0011] S4. Extract the time interval t between the two consecutive video images P1 and P2, and the elevation difference between the aircraft position C and the aircraft position C1. According to the formula The rate of decline was calculated. Stored in the flight dataset.

[0012] Preferably, several video images arranged in chronological order are selected from the video stream data to form video image time-series data. The video image time-series data is segmented and organized, wherein the data group before and after the i-th segment consists of the i-th video image and the (i+1)-th video image. The data group before and after the i-th segment is used as the two video images P1 and P2 before and after the i-th segment in method S1 and processed according to method S1 to method S2 to obtain the predicted trajectory of the data group before and after the i-th segment. The predicted trajectories of the data groups before and after all segments are obtained in sequence according to the time series. All predicted trajectories are curve fitted to obtain the fitted predicted trajectories corresponding to all segments and stored in the flight dataset.

[0013] Preferably, the data group before and after the i-th segment is taken as the two video images P1 and P2 before and after the i-th segment in method S1, and the ground velocity of the data group before and after the i-th segment is obtained according to method S3. The ground velocity of all data groups before and after each segment is obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted and inferred trajectory; the data sets before and after the i-th segment are used as two video images P1 and P2 before and after the i-th segment in method S1, and the descent rate of the data sets before and after the i-th segment is obtained according to method S4. The rate of decline of all data groups before and after each segment was obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted inferred trajectory for storage.

[0014] Preferably, the video stream data includes video data of the entire process from the aircraft's flight to its crash, and the last video image in the video image time sequence data is a video image of the aircraft crashing to the ground or above the crash site.

[0015] Preferably, the physical length of line L1 in video image P1 is calculated as follows:

[0016] Obtain the length pixel value of the aircraft in video image P1. Obtain the physical length of the camera's image sensor corresponding to the video stream data. The length of the video image corresponding to the image sensor (in pixels). The focal length captured by the video stream data Obtain the measured physical length of the aircraft. The physical length of line L1 The expression is as follows:

[0017] ;

[0018] Obtain the pixel value representing the distance from the center of the aircraft to the camera position in video image P1. Based on the actual measured length Divide by length pixel value The conversion factor can then be obtained.

[0019] Preferably, the latitude and longitude expression of the aircraft position C in video image P1 is as follows:

[0020]

[0021] ,in , These are the longitude and latitude of the shooting location, respectively. Let C be the angle between the aircraft's position and the ground. , These are the longitude and latitude of the aircraft's position C, respectively. Let L be the physical length of line L1.

[0022] Preferably, if the aircraft position C1 in video image P2 is the crash site, the longitude of the crash site is obtained. with latitude Then the distance between the aircraft position C and the aircraft position C1 , Let L1 be the angle between lines L1 and L2, and its expression is as follows:

[0023] .

[0024] Preferably, the altitude of the aircraft position C in video image P1 The expression is as follows:

[0025] ,in The height of reference object A, The height of the camera at the camera position. Let L1 be the physical length of line L1. The horizontal distance from reference object A to the camera is given.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] This invention can retrieve important flight data such as trajectory, speed, and altitude of an accident aircraft based on video stream data, solving a key problem in aircraft accident investigation. It provides accurate and reliable flight data support for aircraft accident investigation, facilitating the investigation and improving its capabilities, efficiency, and credibility. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a flowchart illustrating the method for investigating and retrieving pre-accident flight data in the embodiment.

[0030] Figure 3 The above is a schematic diagram illustrating the principle in a geographic coordinate system, using video image P1 before the crash and video image P2 after the crash as examples in the embodiment.

[0031] Figure 4 This is a schematic diagram illustrating the calculation principle of the aircraft position C in video image P1 in the embodiment;

[0032] Figure 5 This is a schematic diagram illustrating the extraction of three video images from time-series data in the example embodiment. Detailed Implementation

[0033] The present invention will be further described in detail below with reference to embodiments:

[0034] Example

[0035] like Figure 1 , Figure 2 As shown, a method for investigating and retrieving aircraft flight data based on video image data is described, the method comprising:

[0036] S1. Obtain video stream data of the same camera position of the aircraft before the accident. Filter two video images P1 and P2 from the video stream data. The lines connecting the camera position and the aircraft position in both video images contain reference objects A and B located on the ground. Obtain the measured latitude, longitude, elevation, and distance from the camera position of reference objects A and B (including the measured data of reference object A and reference object B; the measured data of reference object A includes the measured latitude, longitude, elevation, and distance from reference object A to the camera position; the measured data of reference object B includes the measured latitude, longitude, elevation, and distance from reference object B to the camera position). Obtain the conversion factor between pixel distance and measured distance in the video images. The conversion factor between pixel distance and measured distance can be calculated based on any distance, such as the pixel distance from reference object A to the camera position versus the measured distance, or the pixel distance from reference object B to the camera position versus the measured distance, and so on. Alternatively, the distance data of two points in the video image that are easy to calculate the conversion factor between pixel distance and measured distance can be calculated, and the average value can be used as the conversion factor. Measuring the pixel distance between any two points in a video image (such as the aircraft position C in video image P1 and the aircraft position C1 in video image P2) and then using the conversion factor, the actual physical distance between the two points can be obtained. The most important aspect of this invention is the calculation of the distance from aircraft position C to camera position and the distance from aircraft position C1 to camera position. Taking the distance from aircraft position C to camera position as an example, the pixel distance from aircraft position C to camera position is first measured, and then the distance from aircraft position C to camera position can be obtained through a conversion system. Given the measured latitude, longitude, and elevation of reference object A, and the distance from reference object A to camera position, as well as the latitude, longitude, and elevation of camera position, the latitude, longitude, and elevation corresponding to aircraft position C can be obtained.

[0037] The line connecting camera position Q and aircraft position C in video image P1 is denoted as line L1, and the line connecting camera position Q and aircraft position C1 in video image P2 is denoted as line L2.

[0038] S2. Construct a geographic coordinate system with the camera position as the origin and the ground as the coordinate plane. Convert the straight lines L1 and L2 with the reference objects A and B using conversion factors and then express them uniformly on the geographic coordinate system. Calculate the geographic coordinate information of the aircraft position C and the aircraft position C1 respectively and fit a line to form the predicted trajectory. The geographic coordinate information includes latitude, longitude and elevation. Construct a flight dataset, store the predicted trajectory containing geographic coordinate information in the flight dataset and output it.

[0039] S3. Extract the time interval t between the two consecutive video images P1 and P2 (since the video stream data is synchronized with the aircraft in time, the time difference between the two video images P1 and P2 in the video stream data is directly extracted as the time interval t) and the distance between the aircraft position C and the aircraft position C1. According to the formula Calculated ground speed Stored in the flight dataset.

[0040] S4. Extract the time interval t between the two consecutive video images P1 and P2 (since the video stream data is synchronized with the aircraft in time, the time difference between the two video images P1 and P2 in the video stream data is directly extracted as the time interval t) and the elevation difference between the aircraft position C and the aircraft position C1. According to the formula The rate of decline was calculated. Stored in the flight dataset.

[0041] In some embodiments, N video images arranged in chronological order are filtered from the video stream data to form video image time-series data. This video image time-series data is then segmented and organized (two adjacent video images constitute video images P1 and P2 in method S1, respectively; first, reference objects are included on the line connecting the camera position and the aircraft position in the video image, where the camera position is the shooting position of the camera, such as a Skynet surveillance system camera or a handheld camera by a witness, which is acquired during the accident investigation and marked accordingly in the video stream data). In the N video images of the video image time-series data, a total of N-1 segments of preceding and following data are formed (the i-th segment of preceding and following data is one of the preceding and following data segments in the N-1 segments). See [link to relevant documentation]. Figure 5 A simple example is provided by extracting three video images from the time-series video image data. Figure 5 The images in the middle, from left to right, are screenshots taken sequentially over time. Figure 5 The leftmost video image shows the relationship between the camera position and the aircraft, including a corresponding reference point. Figure 5 The relationship between the camera position and the aircraft in the intermediate video images includes corresponding reference objects. Figure 5 The connection between the camera position and the aircraft in the rightmost video image includes a corresponding reference point.

[0042] In the video image time series data, the data group before and after the i-th segment consists of the i-th video image and the (i+1)-th video image (the i-th and (i+1)-th video images are selected video images from the video stream data, not adjacent frames in the video stream data, but video images with several frames in between. The time interval between each data group before and after the i-th segment is generally different. Since the video stream data is synchronized with the aircraft in time, the data group before and after the i-th segment is the time length from the i-th video image to the (i+1)-th video image in the video stream data). The data group before and after the i-th segment is used as the two video images P1 and P2 before and after the i-th segment in method S1 (the i-th video image corresponds to video image P1, and the (i+1)-th video image corresponds to video image P2). The predicted trajectory of the data group before and after the i-th segment is obtained by processing according to methods S1 to S2. The inferred trajectories of all data groups before and after each segment (i.e., data groups before and after N-1 segments) are obtained sequentially according to the time series. All inferred trajectories are then curve-fitted to obtain the fitted inferred trajectories corresponding to each segment, which are then stored in the flight dataset. In this way, the fitted inferred trajectories corresponding to the aircraft can be obtained from the video stream data. The fitted inferred trajectories are stored in the flight dataset. During the accident investigation, the fitted inferred trajectories are extracted from the flight dataset for accident investigation and analysis, providing trajectory data support for aircraft accident investigation.

[0043] Using the data sets before and after the i-th segment as two video images P1 and P2 in method S1, the ground velocity of the data sets before and after the i-th segment is obtained according to method S3. The ground velocity of all data groups before and after each segment is obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted inferred trajectory. This allows us to obtain the ground speed calculated from each segment of the aircraft's trajectory (e.g., the inferred trajectory of segment i) from the video stream data. The corresponding ground speed The data will determine the ground speed corresponding to each segment of the predicted trajectory. The data is correlated and stored in the flight dataset (based on the predicted trajectory and corresponding ground speed). (Through associated storage), the ground speed of each segment of the inferred trajectory is extracted from the flight dataset during the accident investigation to conduct accident investigation and analysis, providing ground speed data support for aircraft accident investigation.

[0044] Using the data sets before and after the i-th segment as two video images P1 and P2 in method S1, the decrease rate of the data sets before and after the i-th segment is obtained according to method S4. The rate of decline of all data groups before and after each segment was obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted inferred trajectory. This allows us to obtain the descent rate calculated from each segment of the aircraft's trajectory (e.g., the inferred trajectory of segment i) from the video stream data. The corresponding descent rate data will be used to determine the descent rate for each segment of the predicted trajectory. The data is associated and stored in the flight dataset (according to the predicted trajectory and the corresponding descent rate). During the accident investigation, the descent rate of each predicted trajectory segment is extracted from the flight dataset for accident investigation and analysis, providing descent rate data support for aircraft accident investigation.

[0045] In some embodiments, the preferred video stream data of the present invention includes video data of the entire process from the aircraft in the air to its crash, and the last video image in the video image time sequence data is a video image of the aircraft crashing to the ground or above the crash site.

[0046] In some embodiments, see Figure 3 The physical length of line L1 in video image P1 (Actual Length) for Figure 3 The calculation method for the slope distance D in the curve is as follows:

[0047] Obtain the length pixel value of the aircraft in video image P1. Obtain the physical length of the camera's image sensor corresponding to the video stream data. The length of the video image corresponding to the image sensor (in pixels). The focal length captured by the video stream data Obtain the measured physical length of the aircraft. The physical length of line L1 The expression is as follows:

[0048] ;

[0049] Obtain the pixel value representing the distance from the center of the aircraft to the camera position in video image P1. Based on the actual measured length Divide by length pixel value The conversion factor can then be obtained (this invention also provides a method for obtaining the conversion factor).

[0050] If focal length Alternatively, the distance from the aircraft crash site to the camera location can be calculated using the crash site itself. (distance See Figure 3 As shown), the length in pixels of the aircraft during the crash in the video. ,focal length The expression is as follows:

[0051] The physical length of the camera's image sensor corresponds to the video stream data. The length of the video image corresponding to the image sensor (in pixels). The measured physical length of the aircraft .

[0052] A key aspect of this invention is the calculation of the distances from aircraft position C to the camera position, and from aircraft position C1 to the camera position. Taking the distance from aircraft position C to the camera position as an example, the pixel distance between aircraft position C and the camera position is first measured, and then the distance from aircraft position C to the camera position is obtained through a conversion system. Given the measured latitude, longitude, and elevation of reference object A, and the distance from reference object A to the camera position, as well as the latitude, longitude, and elevation of the camera position, the latitude, longitude, and elevation of aircraft position C can be obtained. In some embodiments, the latitude and longitude expression of aircraft position C in video image P1 is as follows:

[0053]

[0054] ,in , These are the longitude and latitude of the shooting location, respectively. Let C be the angle between the aircraft's position and the ground. , These are the longitude and latitude of the aircraft's position C, respectively. Let L1 be the physical length of the line. Following the method described above, the latitude and longitude data of the flight position in each video image across all preceding and following data sets can be calculated. In method S3, the latitude and longitude data of both preceding and following video images P1 and P2 are obtained, allowing the distance to be calculated. According to the formula Calculated ground speed Stored in the flight dataset.

[0055] like Figure 3 As shown, assuming the aircraft position C1 in video image P2 is the crash site, obtain the longitude of the crash site. with latitude Then the distance between the aircraft position C and the aircraft position C1 , Let L1 be the angle between lines L1 and L2, and its expression is as follows:

[0056] .

[0057] like Figure 4 As shown, preferably, the altitude of the aircraft position C in video image P1 is... The expression is as follows:

[0058] ,in The height of reference object A, The height of the camera at the camera position. Let L1 be the physical length of line L1. The horizontal distance from reference object A to the camera; then the height of the aircraft position C in video image P1. The data is converted to elevation data. Following the method described above, the altitude and elevation data of the flight position in each video image across all preceding and following data sets can be calculated. In method S4, the altitude or elevation of both preceding and following video images P1 and P2 is acquired, allowing for the calculation of the elevation difference. According to the formula The rate of decline was calculated. Stored in the flight dataset.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for investigating and retrieving aircraft flight data accidents based on video image data, characterized in that: The methods include: S1. Obtain video stream data of the same camera position of the aircraft before the accident. Filter the video stream data into two video images, P1 and P2. The lines connecting the camera position and the aircraft position in both video images contain reference objects A and B located on the ground. Obtain the measured latitude, longitude, elevation, and distance from the camera position to reference objects A and B, and obtain the conversion factor between the pixel distance in the video image and the measured distance. The line connecting the camera position Q and the aircraft position C in video image P1 is denoted as line L1, and the line connecting the camera position Q and the aircraft position C1 in video image P2 is denoted as line L2. S2. Construct a geographic coordinate system with the camera position as the origin and the ground as the coordinate plane. Convert the lines L1 and L2 with reference objects A and B using conversion factors and then express them uniformly on the geographic coordinate system. Calculate the geographic coordinate information of the aircraft positions C and C1 respectively and fit a line to the system as the predicted trajectory. The geographic coordinate information includes latitude, longitude, and elevation. Construct a flight dataset, store the predicted trajectory containing geographic coordinate information in the flight dataset, and then output it. S3. Extract the time interval t between the two consecutive video images P1 and P2, and the distance between the aircraft position C and the aircraft position C1. According to the formula Calculated ground speed Stored in the flight dataset; S4. Extract the time interval t between the two consecutive video images P1 and P2, and the elevation difference between the aircraft position C and the aircraft position C1. According to the formula The rate of decline was calculated. Stored in the flight dataset; Video image time-series data is formed by filtering several video images arranged in chronological order from the video stream data. The video image time-series data is then segmented and organized, where the data group before and after the i-th segment consists of the i-th video image and the (i+1)-th video image. The data group before and after the i-th segment is used as the two video images P1 and P2 before and after the i-th segment in method S1. The predicted trajectory of the data group before and after the i-th segment is obtained by processing it according to methods S1 to S2. The predicted trajectories of the data groups before and after all segments are obtained in sequence according to the time series. All predicted trajectories are curve fitted to obtain the corresponding fitted predicted trajectories for all segments and stored in the flight dataset.

2. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 1, characterized in that: Using the data sets before and after the i-th segment as two video images P1 and P2 in method S1, the ground velocity of the data sets before and after the i-th segment is obtained according to method S3. The ground velocity of all data groups before and after each segment is obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted and inferred trajectory; the data sets before and after the i-th segment are used as two video images P1 and P2 before and after the i-th segment in method S1, and the descent rate of the data sets before and after the i-th segment is obtained according to method S4. The rate of decline of all data groups before and after each segment was obtained sequentially according to the time series. The flight dataset is segmented and associated with the fitted inferred trajectory for storage.

3. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 1, characterized in that: The video stream data includes video data of the entire process from the aircraft's flight to its crash, and the last video image in the video image time sequence data is a video image of the aircraft crashing to the ground or above the crash site.

4. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 1, characterized in that: The physical length of line L1 in video image P1 is calculated as follows: Obtain the length pixel value of the aircraft in video image P1. Obtain the physical length of the camera's image sensor corresponding to the video stream data. The length of the video image corresponding to the image sensor (in pixels). The focal length captured by the video stream data Obtain the measured physical length of the aircraft. The physical length of line L1 The expression is as follows: ； Obtain the pixel value representing the distance from the center of the aircraft to the camera position in video image P1. Based on the actual measured length Divide by length in pixels The conversion factor can then be obtained.

5. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 4, characterized in that: The latitude and longitude expressions for the aircraft's position C in video image P1 are as follows: ， ,in , These are the longitude and latitude of the shooting location, respectively. Let C be the angle between the aircraft's position and the ground. , These are the longitude and latitude of the aircraft's position C, respectively. Let L be the physical length of line L1.

6. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 4, characterized in that: If the aircraft position C1 in video image P2 is the crash site, obtain the longitude of the crash site. with latitude Then the distance between the aircraft position C and the aircraft position C1 , Let L1 be the angle between lines L1 and L2, and its expression is as follows: .

7. The method for investigating and retrieving aircraft flight data accidents based on video image data according to claim 4, characterized in that: The altitude of the aircraft position C in video image P1 The expression is as follows: ,in The height of reference object A, The height of the camera at the camera position. Let L1 be the physical length of line L1. The horizontal distance from reference object A to the camera is given.

Images