Estimating position, size, and / or trajectory of an aerial phenomenon from multiple captured videos

Abstract

A system for detection of an aerial phenomenon from captured videos includes at least one processor and at least one memory storing instructions. The instructions, when executed by the processor(s), cause the system at least to perform: accessing a first captured video and a second captured video which captured different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the plurality of 3D real-world position in response to an electronic inquiry.

Description

Attorney Docket No.: 3218-5 PCT ESTIMATING POSITION, SIZE, AND / OR TRAJECTORY OF AN AERIAL PHENOMENON FROM MULTIPLE CAPTURED VIDEOS CROSS REFERENCE TO RELATED APPLICATION

[0001] The present application claims the benefit of and priority to U.S. Provisional Application No.63 / 519,332, filed on August 14, 2023, which is hereby incorporated by reference herein in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates generally to aerial phenomena and, more particularly, to estimating the position, size, and / or trajectory of an aerial phenomenon from multiple captured videos. BACKGROUND

[0003] Unidentified aerial phenomena (UAP) are any perceived aerial phenomena that cannot be immediately identified or explained. While unusual sightings have been reported in the sky throughout history, UAPs did not achieve their current cultural prominence until after World War II. There is interest in exploring unidentified aerial phenomena both among the public and, more recently, the U.S. Congress, which has formally directed the creation of a new government office to investigate them. SUMMARY

[0004] The present disclosure relates to estimating the position, size, and / or trajectory of an aerial phenomenon from multiple captured videos.

[0005] In accordance with aspects of the present disclosure, a system for detection of an aerial phenomenon from captured videos includes: at least one processor, and at least one memoryAttorney Docket No.: 3218-5 PCT storing instructions. The instructions, when executed by the at least one processor, cause the system at least to perform: accessing a first captured video and a second captured video, where the first and second captured videos capture different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the at least one 3D real- world position in response to an electronic inquiry.

[0006] In various embodiments of the system, in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform stereo matching.

[0007] In various embodiments of the system, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

[0008] In various embodiments of the system, in the performing triangulation, the instructions, when executed by the at least one processor, cause the system at least to perform: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.Attorney Docket No.: 3218-5 PCT

[0009] In various embodiments of the system, in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform a smoothing operation, where the plurality of 3D real-world positions are provided by the smoothing operation.

[0010] In various embodiments of the system, the instructions, when executed by the at least one processor, further cause the system at least to perform: estimating a size of the aerial phenomenon based on the plurality of 3D real-world positions; and estimating a trajectory of the aerial phenomenon based on the plurality of 3D real-world positions.

[0011] In various embodiments of the system, the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

[0012] In accordance with aspects of the present disclosure, a method for detection of an aerial phenomenon from captured videos includes: accessing a first captured video and a second captured video, where the first and second captured videos capture different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the at least one 3D real-world position in response to an electronic inquiry.

[0013] In various embodiments of the method, the estimating the plurality of 3D real-world positions includes performing stereo matching.

[0014] In various embodiments of the method, the estimating the plurality of 3D real-worldAttorney Docket No.: 3218-5 PCT positions includes performing triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

[0015] In various embodiments of the method, the performing triangulation includes: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.

[0016] In various embodiments of the method, the estimating the plurality of 3D real-world positions includes performing a smoothing operation, and the plurality of 3D real-world positions are provided by the smoothing operation.

[0017] In various embodiments of the method, the method further includes: estimating a size of the aerial phenomenon based on the plurality of 3D real-world positions; and estimating a trajectory of the aerial phenomenon based on the plurality of 3D real-world positions.

[0018] In various embodiments of the method, the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

[0019] In accordance with aspects of the present disclosure, a non-transitory processor- readable medium stores instructions which, when executed by at least one processor of a system, cause the system at least to perform: accessing a first captured video and a second captured video, where the first and second captured videos capture different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D)Attorney Docket No.: 3218-5 PCT real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the at least one 3D real-world position in response to an electronic inquiry.

[0020] In various embodiments of the non-transitory processor-readable medium, in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform stereo matching.

[0021] In various embodiments of the non-transitory processor-readable medium, the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

[0022] In various embodiments of the non-transitory processor-readable medium, in the performing triangulation, the instructions, when executed by the at least one processor, cause the system at least to perform: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.

[0023] In various embodiments of the non-transitory processor-readable medium, the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform a smoothing operation, and the plurality of 3D real-world positions are provided by the smoothing operation.

[0024] In various embodiments of the non-transitory processor-readable medium, the instructions, when executed by the at least one processor, further cause the system at least to perform: estimating a size of the aerial phenomenon based on the plurality of 3D real-worldAttorney Docket No.: 3218-5 PCT positions; and estimating a trajectory of the aerial phenomenon based on the plurality of 3D real- world positions.

[0025] In various embodiments of the non-transitory processor-readable medium, the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

[0026] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] A detailed description of embodiments of the disclosure will be made with reference to the accompanying drawings, wherein like numerals designate corresponding parts in the figures:

[0028] FIG.1 is a flow diagram of an example of an operation relating to aerial phenomena, in accordance with aspects of the disclosure;

[0029] FIG. 2 is an diagram of an example of observers capturing videos of an aerial phenomena, in accordance with aspects of the present disclosure;

[0030] FIG.3 is an flow diagram of an example of an operation relating to estimating position, size, and / or trajectory of an aerial phenomena, in accordance with aspects of the present disclosure;

[0031] FIG.4 is a diagram of an example of positions and focal vectors for estimating position of an aerial phenomena, in accordance with aspects of the present disclosure;

[0032] FIG. 5 is a diagram of another example of positions and focal vectors for estimating position of an aerial phenomena, in accordance with aspects of the present disclosure;

[0033] FIG.6 is a diagram of an example of pitch, yaw, and roll of a video capture device, inAttorney Docket No.: 3218-5 PCT accordance with aspects of the present disclosure;

[0034] FIG.7 is a diagram of an example of a coordinate system for the real-world position of an aerial phenomenon, in accordance with aspects of the present disclosure;

[0035] FIG. 8 is a diagram of an example of a minimum closest segment between two line vectors for estimating the position of an aerial phenomenon, in accordance with aspects of the present disclosure;

[0036] FIG.9 is an flow diagram of an example of an operation relating to estimating position of an aerial phenomena, in accordance with aspects of the present disclosure; and

[0037] FIG.10 is an flow diagram of an example of an operation relating to estimating position of an aerial phenomena, in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0038] The present disclosure relates to estimating the position, size, and / or trajectory of an aerial phenomenon from multiple captured videos. As used herein, the term “aerial phenomenon” refers to and includes any phenomenon that appears in the sky and may include known objects or phenomena (e.g., airplanes, clouds, lightning, aurora borealis, etc.) and may include unknown phenomena. Although smartphones may be used herein as examples of video capturing device that can capture videos, other video capturing devices are contemplated to be within the scope of the present disclosure.

[0039] In the following description, certain specific details are set forth in order to provide a thorough understanding of disclosed aspects. However, one skilled in the relevant art will recognize that aspects may be practiced without one or more of these specific details or with other methods, components, materials, etc. In other instances, well-known structures, e.g., associatedAttorney Docket No.: 3218-5 PCT with transmitters, receivers, or transceivers, have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the aspects.

[0040] Reference throughout this specification to “one aspect” or “an aspect” means that a particular feature, structure, or characteristic described in connection with the aspect is included in at least one aspect. Thus, the appearances of the phrases “in one aspect” or “in an aspect” in various places throughout this specification are not necessarily all referring to the same aspect. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more aspects.

[0041] Unidentified aerial phenomena (UAPs) present both a challenge and an opportunity for scientific inquiry. While data regarding UAPs exist, a significant proportion of these observations remain unexplained. Limited methodological rigor, combined with social stigma, have historically restricted robust scientific engagement with this field.

[0042] Technological developments have revolutionized potential approaches to UAP study. Advancements in machine learning (ML) and artificial intelligence (AI) have introduced new capabilities in image recognition and analysis. The precision of these tools allows a degree of accuracy and objectivity in data analysis not previously possible. In general, machine learning has been underutilized as a tool for UAP analysis. The reasons for this are multifaceted, including a lack of awareness of machine learning capabilities, limited access to high-quality data, and the absence of a scientific consensus on the value of machine learning in this field. However, the technological developments of recent years present an opportunity. The potential for ML and AI to discern patterns in large data sets presents an opportunity for increased understanding of UAPs.

[0043] Simultaneously, the proliferation of high-resolution imaging devices, particularly in the form of smartphone cameras, has increased the volume and quality of UAP data available forAttorney Docket No.: 3218-5 PCT analysis. This development provides the potential for an unprecedented level of public engagement in data collection, creating a vast and diverse dataset for scientific inquiry. The potential of crowd- sourced, multi-modal data in this field is significant. The variety of data sources, coupled with the volume of data available, presents a valuable opportunity for applying machine learning techniques. Properly managed and analyzed, this data could contribute to significant advancements in the understanding of UAPs.

[0044] This analysis depends on the development of methodologies for differentiating genuine UAP from aerial clutter or misidentified phenomena. Misidentification of known aerial objects has historically been a significant challenge in the study of UAPs. Common objects such as aircraft, weather balloons, and atmospheric phenomena are often mistaken for UAPs, contributing to noise in the data and hampering efforts to analyze and understand genuine UAPs. The task of discerning genuine UAP from known objects is complex and involves a comprehensive understanding of both known aerial objects and the behaviors and characteristics of genuine UAPs. Unique characteristics associated with UAP, such as anomalous shapes, flight characteristics, unusual sounds, and other specific particulars linked with the phenomenon, present a unique set of challenges for data analysis.

[0045] A robust system can provide accurate assessments of both the known and unknown elements of observed phenomena. FIG. 1 is a flow diagram of an example of an operation for processing data to make an UAP assessment. In accordance with aspects of the present disclosure, a multi-phased operation is disclosed that includes data source validation, sky object classification, content ranking and filtering, and UAP assessment. The operation can be implemented in a processing system, such as the processing system of FIG.10.Attorney Docket No.: 3218-5 PCT

[0046] At block 110, the operation involves data source validation. UAP data often originates from disparate sources, including video footage and eyewitness accounts. In aspects of the present disclosure, authenticity of the data source can be validated to ensure that data is obtained from reliable and / or verified sources. In aspects of the present disclosure, data points such as timestamp and location metadata may be used for validation. Additionally, after validation, these data points may be cross-referenced with known celestial events, weather phenomena, or scheduled flights to differentiate UAP from identifiable objects.

[0047] At block 120, the operation involves sky object classification. The sky object classification involves the categorization of sky objects. Image preprocessing techniques, such as noise reduction and contrast enhancement, are applied to improve the visibility and discernibility of objects in the images. The processed images are then subjected to classification techniques, which are described in more detail later herein.

[0048] Image processing forms a facet in the classification of sky objects, particularly in the context of UAP. In various embodiments, image preprocessing involves normalization of resolution and aspect ratio across all images to provide consistency. In various embodiments, noise reduction techniques, such as spatial and frequency domain filters, can help minimize the effects of environmental factors such as atmospheric distortions, light pollution, and sensor noise. In various embodiments, image enhancement techniques, such as color adjustment, contrast stretching, and histogram equalization, can help highlight potential UAP, making them more discernible. Once the images are preprocessed, quality control measures can be implemented. These include checking the effectiveness of noise reduction and enhancement techniques. If an image still contains significant noise or the potential UAP is not clearly visible, it may be subjected to further preprocessing or, in some cases, may be flagged for manual review.Attorney Docket No.: 3218-5 PCT

[0049] In accordance with aspects of the present disclosure, sky object classification techniques are trained to differentiate not only between common sky objects, such as planes, birds, or celestial bodies, but also potential UAPs that defy standard categorization. Training of classification techniques involves inputting vast, diverse datasets of known sky objects. The quality of this training data directly impacts the classification technique's ability to correctly classify new data. Consequently, validation checks are implemented to confirm the correctness of labels in the training data, so that the classification technique learns from accurate, reliable information. Once the classification technique is trained, it is tested on validation datasets different from the training set. This helps to confirm that the classification technique's performance is not merely a result of overfitting to the training data but rather indicative of its capacity to classify new, unseen data. Quality control measures, such as accuracy, precision, recall, and F1 scores, provide insight into the classification technique's effectiveness, which person skilled in the art will understand. As more data is collected, and the classification technique learns and adapts, these quality control measures may be reassessed to ensure they remain relevant and conducive to high- quality, reliable UAP analysis.

[0050] Establishing comprehensive and distinct content categories / classifications provides a foundation for the later task of content ranking and filtering. As used herein, and unless indicated otherwise, the terms “classification” and “category” will be used interchangeably. Categories serve as the backbone for structuring data, allowing for easier navigation, effective retrieval, and efficient analysis. A well-defined categorization system aids in meaningful differentiation of data, so that that the most relevant content surfaces to the top during the later content ranking and filtering process.Attorney Docket No.: 3218-5 PCT

[0051] Defining the categories takes into account a wide range of variables that constitute sky object data. These may include, but are not limited to, object spatial attributes, time of observation, celestial coordinates, object morphology, and / or spectral properties, among other potential variables. More nuanced categories may consider variables such as atmospheric conditions during the sighting or the technological specifications of the observing equipment.

[0052] In various embodiments, the names of various categories may correspond to the vernacular and shared understanding used by participants of the UAP community. This could include categories defined by shapes commonly reported in UAP sightings (e.g., disk, triangle, cigar, etc.), observed behavior (e.g., hovering, rapid acceleration, etc.), or reported associated phenomena (e.g., electromagnetic effects, physical traces, etc.). Accordingly, classifications for the sky object classification techniques may include disk, triangle, cigar, hovering, rapid acceleration, electromagnetic effects, and / or physical traces, among other classifications.

[0053] The basis and naming of categories may be flexible, as UAP data, by its nature, can be elusive and unpredictable. UAPs are by definition unidentified, and there are no complete datasets of confirmed UAP. The classification approach can maintain flexibility by flagging objects that do not fit traditional categories for further analysis, rather than trying to force all objects into predefined classes. Categories may be inclusive enough to accommodate new and unforeseen data but sufficiently distinct to reduce ambiguity or overlap. In various embodiments, an "uncategorized" or "other" category may be beneficial and may be defined for data that does not fit neatly into any defined category.

[0054] In aspects of the present disclosure, to facilitate continuous refinement, an iterative feedback mechanism may be established, encouraging users to suggest improvements to theAttorney Docket No.: 3218-5 PCT categorization. Such community input can be invaluable in aligning the system with user needs and expectations, thereby increasing user satisfaction and system usability.

[0055] The use of machine learning algorithms and deep learning techniques, parallel processing, and efficient data structures can be implemented to handle large volumes of images swiftly and facilitate the sky object classification.

[0056] At block 130, the operation involves content ranking and filtering. In accordance with aspects of the present disclosure, the content is ranked and filtered based on its relevance. Defined content categories were described above, and they are used as a basis for ranking. Techniques are employed to systematically rank the content, with higher ranks given to more relevant data. In various embodiments, for relevance assessment, images in which the potential UAP is too small, blurred, or obscured may not provide sufficient information and may be considered less relevant. In various embodiments, image processing techniques can quantify the level of visibility and size of the potential UAP in the image and assess their relevance based on these parameters.

[0057] Regarding filtering, a mechanism is implemented for detecting and removing inappropriate content, thereby retaining pertinent and appropriate data for further analysis. The performance of the system and its impact on user experience may be regularly evaluated and adjusted based on user feedback.

[0058] In various embodiments, language analysis may be incorporated to enhance the content ranking and filtering task. For example, if a video includes spoken words which indicate that a sky object may be a known sky object, such information may be considered to rank the information lower and / or to filter out the video. As another example, if a video includes spoken words which indicate genuine surprise and inability to name a sky object, such information may be considered to rank the information higher and / or to not filter out the video.Attorney Docket No.: 3218-5 PCT

[0059] At block 140, the operation involves UAP assessment. For the UAP assessment, technology stacks and resources are utilized in the analysis of additional sources of information. A training dataset is created for supervised learning models, which are then used to determine the likelihood of a report detailing a genuine unknown phenomenon.

[0060] The aspects, embodiments, and descriptions provided in connection with FIG. 1 are merely examples and do not limit the scope of the present disclosure. Other examples and variations are contemplated to be within the scope of the present disclosure. For example, in various embodiments, the operation may include other blocks not shown in FIG. 1. In various embodiments, the operation may not include every block shown in FIG. 1. In various embodiments, the operation may implement the blocks in a different order than as shown in FIG. 1. Such and other embodiments are contemplated to be within the scope of the present disclosure.

[0061] As described above, various tasks in the operation of FIG.1 involve variables relating to spatial position, size, and / or trajectory of a known aerial phenomenon or of an unidentified aerial phenomenon. In accordance with aspects of the present disclosure, an approach is disclosed for estimating the position (e.g., three-dimensional (3D) position), the size (e.g., dimensions), and / or the trajectory of an aerial phenomenon from multiple videos.

[0062] FIG.2 is a diagram of observers capturing an aerial phenomenon using video capture devices, e.g., smartphones. An aerial phenomenon 210 appears above a horizon 220. A first observer’s video capture device 230 has a line of sight 232 to the aerial phenomenon 210, and a second observer’s video capture device 240 has another line of sight 242 to the aerial phenomenon 210. FIG. 2 is merely an example. In various scenarios, two or more video capture devices may capture an aerial phenomenon from one or more lines of sight, and all such scenarios are within the scope of the present disclosure.Attorney Docket No.: 3218-5 PCT

[0063] FIG.3 is a flow diagram of an operation for estimating position, size, and trajectory of an aerial phenomenon from multiple videos.

[0064] At block 310, the operation involves calibrating video capture devices, e.g., smartphones, which may also be referred to as cameras. Camera calibration is a process for estimating the intrinsic and extrinsic parameters of a camera. The intrinsic parameters, such as focal length and optical centers, are specific to the camera and lens. Extrinsic parameters relate to the camera's position and orientation in the world. In various embodiments, a checkerboard pattern is used to calibrate each camera. Each camera can capture several images of the checkerboard pattern from different angles and distances to cover as much of the field of view as possible. In various embodiments, the captured images may be converted to grayscale. Based on the captured images, a computational technique is used to obtain the camera matrix and distortion coefficients. These parameters will be used for subsequent stages of the operation of FIG.3.

[0065] The operation at block 310 finds the corners of the checkerboard in each image. If the corners are found, their coordinates in the image are defined. Once all the corners have been found in all images, a calibration technique performs the camera calibration. The calibration technique uses lists of the object points (the real-world coordinates of the corners) and the image points (the coordinates of the corners in the image), and returns the camera matrix, the distortion coefficients, and the rotation and translation vectors for each image. These parameters are saved for later use in the operation of FIG.3. Thus, the operation of block 310 calibrates cameras that may be used in capturing videos of aerial phenomena. It may reveal important operational characteristics and limitations on what various models of cameras may or may not be able to detect.

[0066] At block 320, the operation involves synchronizing all captured videos of an aerial phenomenon. The synchronization aligns the frames from different videos so that they correspondAttorney Docket No.: 3218-5 PCT to the same moments in time. In various embodiments, the operation of block 320 extracts the timestamps for each frame, finds a common start time for all videos, and aligns the frames in different videos based on the timestamps. In such embodiments, the captured videos should include associated metadata including timestamp information. The timestamp metadata from each frame in each video is extracted. This could involve reading a separate metadata file, or the metadata might be embedded in the video file. The operation finds the earliest common timestamp across all captured videos, and this earliest timestamp will be the start point for synchronization. In various embodiments, for each frame in each captured video, the difference between the frame's timestamp and the common start time is calculated to provide the "synchronized" timestamp for each frame. The synchronized timestamps are used to align the frames from different videos. In various embodiments, the timestamps can be placed in increasing order, and for each timestamp, the frame from each video that has the closest timestamp can be selected for alignment. Aligning the frames from the captured videos allow for comparison of different views at the same or substantially the same moment in time for object detection.

[0067] At block 330, the operation involves detecting an aerial phenomenon from the synchronized videos. The detection operates to identify the location of an aerial phenomenon in each frame of the synchronized videos. A machine learning-based object detection technique can be used to identify the aerial phenomenon in each frame. In various embodiments, different object detection techniques may be most suitable for different situations. Such object detection techniques include, but are not limited to, YOLO (you only look once), SSD (single shit detector), and / or Faster R-CNN (region convolutional neural network), among other techniques. Persons skilled in the art will understand how to implement and train such machine learning techniques.Attorney Docket No.: 3218-5 PCT

[0068] In various embodiments, pre-trained machine learning models for the aerial phenomena detection may be used. In various embodiments, machine learning models may be trained on large datasets such as COCO (common objects in context) or ImageNet, which are publicly available. For example, machine learning models may be trained to detect known aerial phenomena such as airplanes, air balloons, clouds, lightning, and / or other aerial phenomena. In various embodiments, the machine learning models may be trained to detect aerial phenomena such as particular shapes commonly reported in UAP sightings (e.g., disk, triangle, cigar, etc.), particular observed behavior (e.g., hovering, rapid acceleration, etc.), and / or reported associated phenomena (e.g., electromagnetic effects, physical traces, etc.). Accordingly, the machine learning models may be trained to detect disk, triangle, cigar, hovering, rapid acceleration, electromagnetic effects, and / or physical traces, among other aerial phenomena.

[0069] The machine learning models may be implemented using OpenCV and / or other libraries (e.g., TensorFlow or PyTorch) for Python. Pre-trained models may be loaded using the appropriate library (e.g., TensorFlow or PyTorch). For each set of synchronized frames from the synchronized videos, the set of synchronized frames can be processed by the object detection models. In various embodiments, the models return a list of bounding boxes with a classification label and a confidence score for each bounding box. In various embodiments, the list can be filtered to include only bounding boxes for which the classification label matches an object of interest and the confidence score is above a threshold. The bounding box coordinates for the detected objects in each frame are recorded and are used in the next block of the operation of FIG.3.

[0070] At block 340, the operation involves estimating position of a detected aerial phenomenon. In aspects of the present disclosure, a 3D position of a detected aerial phenomenon may be estimated using camera parameters and triangulation. Such an approach will be describedAttorney Docket No.: 3218-5 PCT below in connection with FIGS. 4–8. In aspects of the present disclosure, a stereo matching algorithm such as block matching, semi-global block matching, or graph cuts may be used to estimate the 3D position of an aerial phenomenon. Such an approach involves using the camera parameters and the relative orientations and positions of the cameras. Stereo matching operates to find the same physical point in multiple images. The difference in position of this point between the images, also known as disparity, can be used to calculate the 3D position of the point in real- world coordinates. Stereo matching algorithms, such as block matching, semi-global block matching, or graph cuts may be implemented in OpenCV.

[0071] In various embodiments, a stereo matching approach uses the object detection results from the previous block 330, which provides the locations of the aerial phenomenon in each image; the camera parameters from the calibration block 310, including the camera matrix and distortion coefficients; and the relative orientations and positions of the cameras, which are available from the metadata.

[0072] For each pair of synchronized frames, the aerial phenomenon detection result in each frame is identified. The disparity between the aerial phenomenon’s position in the two images, along with the camera parameters and the relative positions and orientations of the cameras, are used to estimate the 3D position of the aerial phenomenon. The approach can be implemented in OpenCV using stereo matching functions and the triangulation principle. This process is repeated for each pair of synchronized frames to estimate the 3D position of the aerial phenomenon in each frame. By leveraging the object detection results and the camera parameters, the 3D position of the aerial phenomenon in each frame can be estimated, which aids with later blocks for estimating the aerial phenomenon’s distance and dimensions.Attorney Docket No.: 3218-5 PCT

[0073] At block 350, the operation involves filtering and / or smoothing the estimated positions. After estimating the 3D positions, a filtering or smoothing technique, such as a Kalman filter, is applied to the data to reduce noise and improve consistency. The filtering / smoothing reduces noise and improves the consistency of the estimated 3D positions, which can be particularly useful if there is a significant amount of noise in the position estimates, which may be the case with stereo matching. A Kalman filter can be used for this purpose, and persons skilled in the art will recognize how to implement a Kalman filter.

[0074] The filter / smoothing technique uses the 3D position estimates from the previous block 340; a model of the dynamics of the aerial phenomenon (e.g., how it moves from one frame to the next), which could be as simple as assuming constant velocity or constant acceleration; and a Kalman filter implementation (e.g., implementation in OpenCV).

[0075] The Kalman filter is initialized with appropriate dimensions. For the Kalman filter, a state will include the estimated 3D position and possibly the velocity or acceleration of the aerial phenomenon, and the measurement will be the 3D position. The initial state and error covariance can be set based on the first few position estimates. For each 3D position estimate, the prediction is performed and the update steps of the Kalman filter are performed. The updated state estimate from the Kalman filter will be the filtered 3D position. With the Kalman filter, it will be possible to reduce noise and improve the consistency of the 3D position estimates, making the subsequent blocks of distance and dimension measurement more reliable.

[0076] At block 360, the operation involves estimating the size of a detected aerial phenomenon. In various embodiments, the dimension estimation operates to determine the real- world size of the aerial phenomenon based on its appearance in the images and its estimated 3D position. In various embodiments, the size of the aerial phenomenon can be estimated based onAttorney Docket No.: 3218-5 PCT finding the maximum and minimum points in each dimension. In cases where the shape of the aerial phenomenon is known, a 3D shape fitting algorithm can be used to fit a shape to the 3D points, which will provide more accurate dimensions than just finding the maximum and minimum points in each dimension. The 3D shape fitting algorithm can use models of various shapes of aerial phenomena, such as a 3D model file or a parametric model, among other possibilities.

[0077] Based on the 3D position of the aerial phenomenon, the orientation of the aerial phenomenon can be estimated by, for example, fitting a 3D model to the position estimates and finding the orientation that minimizes the distance between the model and the position estimates. Based on the orientation, the 3D model can be scaled to match the size of the aerial phenomenon in the images. This can be implemented by finding the scale factor which minimizes the distance between the scaled model and the position estimates. The dimensions of the object can be calculated from the scaled and oriented 3D model.

[0078] In aspects of the present disclosure, various approaches may be used to estimate the scale or dimensions of an aerial phenomenon in an image or video without using a full 3D model, as described below.

[0079] An approach relates to use of known object or reference scale. This approach involves having an object with known dimensions in the scene along with the aerial phenomenon of interest. For instance, if the height of a person in the scene is known, this information can be used to estimate the size of other objects and of the aerial phenomenon. This approach assumes that the known object and the aerial phenomenon are at approximately the same distance from the camera.

[0080] An approach relates to assuming a standard size. For certain types of objects, a standard size can be assumed. For example, cars have an average car length, which can be used to estimateAttorney Docket No.: 3218-5 PCT scale. However, this approach provide rough estimates and is not applicable for objects with significant size variations.

[0081] An approach relates to using camera parameters and depth information. If the focal length of the camera is known and some form of depth information is available, the size of the aerial phenomenon can be estimated. The depth information may be available from stereo vision, depth cameras, or methods such as SLAM (simultaneous localization and mapping). Based on depth information, the size of the aerial phenomenon in the image, and the focal length, similar triangles can be used to estimate the real-world size of the aerial phenomenon.

[0082] An approach relates to using multiple images. If there are multiple images of the same aerial phenomenon taken from different distances, the change in the apparent size of the aerial phenomenon can be used to estimate its actual size. This approach is based on knowing the distance from the camera to the aerial phenomenon for at least two images.

[0083] An approach relates to using machine learning. Advances in machine learning have made it possible to estimate the size of objects in images without explicit depth information or a 3D model. Techniques such as monocular depth estimation or 3D object detection can be trained on large datasets to estimate the size and position of objects in a single image.

[0084] Machine learning and YOLO 3D object detection methods offer robust alternatives to the traditional 3D model fitting technique for detecting object location and scale. Unlike model fitting which requires a known 3D model of the object, machine learning approaches learn to detect objects and estimate their dimensions directly from data, making them more flexible and potentially more accurate, especially when dealing with unknown or variable objects. For machine learning approaches to be successful, a large dataset of images annotated with 3D bounding boxes is needed to train or fine-tune the models. This dataset should be representative of the types ofAttorney Docket No.: 3218-5 PCT images and objects the system will encounter in operation. The implementation of such approaches may use a machine learning library, such as TensorFlow or PyTorch, which supports the machine learning model, such as YOLO for 3D object detection.

[0085] In various situations, machine learning and YOLO 3D object detection methods could provide a more accurate and flexible approach to object location and scale estimation than 3D model fitting. The use of machine learning approaches may be particularly beneficial in scenarios where the shape of the aerial phenomenon can vary or is not known in advance, or where the aerial phenomenon can appear in different poses or orientations.

[0086] Each of the approaches described above has its own set of assumptions and limitations, and the most appropriate approach to use depends on the specific situation, such as the availability of certain information, the type of aerial phenomena being observed, and the requested accuracy of the size estimates.

[0087] At block 370, the operation involves, optionally, estimating the trajectory of a detected aerial phenomenon, if the aerial phenomenon is moving in the videos. To track the aerial phenomenon over time, the filtered 3D positions from each frame can be used to estimate the aerial phenomenon's trajectory. The trajectory can then be used to analyze the object's movement over time. Estimating trajectory allows for monitoring the path of the aerial phenomenon through 3D space over time. Analyzing this trajectory provides insights into the object's behavior and movement patterns.

[0088] At block 380, the operation involves estimating error and / or uncertainty in the determinations. Estimating error and / or uncertainty allows for assessing the reliability of the estimated 3D positions and dimensions. Estimating error and / or uncertainty can be implemented in various ways, such as by calculating the standard deviation or by implementing more complexAttorney Docket No.: 3218-5 PCT statistical approaches. If true 3D positions and dimensions are available, the error can be calculated as the difference between the estimated and true values. The standard deviation of the errors can be computed to provide a measure of the spread of the errors. In various embodiments, statistical analysis methods may be used to analyze the distribution of errors and identify any patterns or trends. The error / uncertainty analysis can provide valuable information about the reliability and accuracy of the estimates and can be used to validate the operation of FIG.3.

[0089] The aspects, embodiments, and descriptions above in connection with FIG. 3 are merely examples, and variations are contemplated to be within the scope of the present disclosure. In various embodiments, the operation may involve blocks other than those shown in FIG.3. In various embodiments, the operation may not involve every block shown in FIG. 3. In various embodiments, the operation may implement the blocks in a different order than that shown in FIG. 3. Such and other embodiments are contemplated to be within the scope of the present disclosure.

[0090] The following description of FIGS. 4–8 relate to block 340 described above. As mentioned above, in aspects, the operation of block 340 involves estimating a 3D position of a detected aerial phenomenon using camera parameters and triangulation. Such an approach will be described below in connection with FIGS.4–8.

[0091] Triangulation refers to a technique for determining the location of a point by forming triangles to it from known points. The principle of triangulation can be used to determine the 3D position of an aerial phenomenon that is captured by cameras (e.g., smartphones) observing the same aerial phenomenon simultaneously. Determining the 3D position of an aerial phenomenon over time allows the velocity and the acceleration of the aerial phenomenon to also be determined.

[0092] As an example of triangulation, the simplest possible case involves two observers in two different locations who capture videos of the same aerial phenomenon, as shown in FIG.2. AAttorney Docket No.: 3218-5 PCT mathematical formulation of the problem of finding the intersection point from the two lines-of- sight (LOS) 232, 242 is shown in FIG.4 and FIG.5.

[0093] Suppose a first video capture device has a position p1and a focal vector v1, and the second video capture device has a position p2 and focal vector p2. Position and focal vector are described further below herein. The line span from observer 1 and 2 can be represented by the line span vectors l1and l2:Solving for the µ and λ so that and X1 = X2= X

[0094] To fully describe the position of the video capture device in 3D space, the GPS location (latitude / longitude) and the elevation of the video capture device can provide such information. Smartphones are able to provide such information. For example, in Apple iPhone smartphones, the iOS CLLocation class holds such data.

[0095] The LOS vector can be constructed by knowing the focal vector and the video capture device’s orientation. The focal vector is the vector pointing from the camera center to the pixel of the aerial phenomenon on a camera screen. In various embodiments, this vector depends on where the object appears on the screen. In various embodiments, it is assumed that the object is at the center of the screen, meaning the focal vector is perpendicular to the screen itself. Once objectAttorney Docket No.: 3218-5 PCT detection models are applied to the videos, the focal vector can reflect the pixel location of the detected aerial phenomenon on the screen.

[0096] The attitude of a device refers to the amount of rotation around each of the device’s axes, which can be defined using standard Euler angles or any equivalent representations. For example, the yaw / pitch / roll of an Apple iPhone device is defined as shown in FIG.6.

[0097] Gimble locks are loss of one degree of freedom in a multi-dimensional mechanism at certain alignments of the axes. To avoid possible numerical instabilities from Gimbal locks, the quaternion representation of the attitude can be used. In an Apple iPhone device, this data is encapsulated in the CMAttitude class. The reference frame on which the attitude is based is chosen to be xTrueNorthZVertical. This is a reference frame where the Z axis is vertical and the X axis points to the geographic north pole. This ensures the frame is well-defined at all times and does not require corrections for magnetic versus grid north. In this frame, the default focal vector has the value (0, 0, −1).

[0098] Thus, the telemetry data include are GPS location (latitude / longitude), altitude, and device attitude (quaternion). Given the simple case of a simultaneous observation of the same aerial phenomenon from two observers in different locations, two techniques can be used to triangulate the aerial phenomenon’s position from the telemetry data: (1) express the two sets of p and v values in a common frame of reference; and (2) given the result of (1), find the LOS intersection point. In various embodiments, this approach transforms into a problem of finding the middle-point of the “closest intersection” segment between the two LOS due to measurement uncertainties.

[0099] With regard to the first technique of finding a common frame, in various embodiments, the Earth-centered, Earth-fixed coordinate system (ECEF), also known as the geocentricAttorney Docket No.: 3218-5 PCT coordinate system, can be used as the common frame into which all measurements will be transformed. This has the advantage of being a coordinate system that is Z-aligned with grid north, and is suitable for calculations around the Earth’s surface. An intermediate frame can be used to facilitate the transformation between a frame at Earth’s surface to ECEF. This intermediate frame is the east-north-up (ENU) coordinate system. The relationship between a ENU frame and the ECEF frame is shown in FIG.7.

[0100] Therefore, there are a total of three (3) transformations between four (4) frames to be performed. The four (4) frames are: device’s body frame (as seen in FIG. 6), device attitude reference frame (this is the xTrueNorthZVertical) frame, ENU frame, and ECEF frame.

[0101] A reference frame transformation includes two steps - rotational transformation and translational transformation. The rotational transformations expressed in rotations around axes are:The final rotation is simply:There is on where hae is the altitude. This transformation can be implemented with Python’s pyproj package.

[0102] Finally, given a pair of (p, v), the following transformation can be applied to express it in the ECEF frame:Attorney Docket No.: 3218-5 PCT Once all pairs ofcan proceed to find the LOS intersection point, which is a problem of finding the middle-point of the “closest intersection” segment between the two LOS due to measurement uncertainties.

[0103] As was described in Equation (1), in the case of two observers, the vectors originating from the center of the two phones are denoted as l1 and l2. With reference to FIG. 8, to find the midpoint of the closest intersection segment, the operations finds such a segment, which is defined as the shortest segment, k, between the two vectors l1 and l2. To find such a segment, we find the values of µ and λ that define the points on the two vectors where the norm of X1 − X2 is smallest:

[0104] Thederivative with respect to both µ and λ set to 0 and solving the system of equations:After systemEliminating µ and solving for λ, provides the equation:Attorney Docket No.: 3218-5 PCT Then, the value

[0105] Have the lengths of the vectors that define the points X1 and X2 on the two vectors where the distance between them is at their smallest, Equations (13) and (15) can be substituted into Equation (1) to solve for the points X1 and X2. Then the midpoint is: Thus, X is thetriangulation approach. The estimated position can be used in the operation of FIG.3 as described above.

[0106] The example of the triangulation approach shown in FIGS.4–8 is merely an example, and variations are contemplated to be within the scope of the present disclosure. In various embodiments, the triangulation approach may be implemented in a different way than as shown and described in connection with FIGS.4–8.

[0107] FIG.9 is a flow diagram of an operation according to aspects of the present disclosure. At block 910, the operation involves accessing a first captured video and a second captured video, the first and second captured videos capturing different lines-of-sight of an aerial phenomenon. The captured videos are videos captured by, e.g., the observers shown in FIG. 2. In various embodiments, the video capturing devices are calibrated in accordance with block 310 of FIG.3. Such and other approaches are contemplated to be within the scope of the present disclosure.

[0108] At block 920, the operation involves synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos. In various embodiments, the operation of block 920 may utilize timestamps in the mannerAttorney Docket No.: 3218-5 PCT described in connection with block 320 of FIG. 3 to synchronize the captured videos. Such and other approaches are contemplated to be within the scope of the present disclosure.

[0109] At block 930, the operation involves detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos. In various embodiments, the operation of block 330 of FIG.3. For example, the detection may apply trained machine learning models to classify aerial phenomena and provide bounding boxes. Such and other approaches are contemplated to be within the scope of the present disclosure.

[0110] At block 940, the operation involves estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos. In various embodiments, the operation of estimating the 3D positions may utilize stereo matching techniques described in connection with block 340 of FIG. 3. In various embodiments, the operation of estimating the 3D positions may utilize triangulation techniques described in connection with FIGS.4–8. Such and other approaches are contemplated to be within the scope of the present disclosure.

[0111] At block 950, the operation involves providing access to the synchronized first and second captured videos and the at least one 3D real-world position in response to an electronic inquiry. As described above, the information relation 3D real-world positions may be utilized in various ways, such as for UAP studies, scientific studies, access by members of the UAP community, among other uses. The captured videos and generated estimates, e.g., UAP classification, UAP position, UAP size, and / or UAP trajectory, among others, may be stored in a database, in a cloud system, and / or in a proprietary system. Persons requesting such information may submit an electronic request for such information. The electronic request may be a remoteAttorney Docket No.: 3218-5 PCT request that is transmitted to the system storing the information. The request may be transmitted in various ways, such as through an dedicated application and / or through a web portal, among other possibilities. Such and other approaches are contemplated to be within the scope of the present disclosure.

[0112] The aspects, examples, and descriptions with respect to FIG. 9 are merely examples, and variations are contemplated to be within the scope of the present disclosure. In various embodiments, the operation may perform other blocks not shown in FIG. 9. In various embodiments, the operation may not perform every block shown in FIG. 9. Such and other variations are contemplated to be within the scope of the present disclosure.

[0113] FIG.10 illustrates a computing system 1000 and includes a processor 1020 connected to a processor-readable storage medium or a memory 1030. The system 1000 may be used to control and / or execute operations described above herein. In various embodiments, the system 1000 is the video capture device of FIG. 2, such as a smartphone. In various embodiments, the system 1000 is a cloud system and / or a proprietary system.

[0114] The processor-readable storage medium or memory 1030 may be a volatile type of memory, e.g., RAM, or a non-volatile type of memory, e.g., flash media, disk media, etc. In various aspects of the disclosure, the processor 1020 may be another type of processor, such as a digital signal processor, a microprocessor, an ASIC, a graphics processing unit (GPU), a field- programmable gate array (FPGA), or a central processing unit (CPU). In certain aspects of the disclosure, network inference may also be accomplished in systems that have weights implemented as memristors, chemically, or other inference calculations, as opposed to processors.

[0115] In aspects of the disclosure, the memory 1030 can be random access memory, read- only memory, magnetic disk memory, solid-state memory, optical disc memory, and / or anotherAttorney Docket No.: 3218-5 PCT type of memory. In some aspects of the disclosure, the memory 1030 can be separate from the system 1000 and can communicate with the processor 1020 through communication buses of a circuit board and / or through communication cables such as serial ATA cables or other types of cables. The memory 1030 includes processor-readable instructions that are executable by the processor 1020 to perform the operations described herein. In other aspects of the disclosure, the system 1000 may include a network interface 1040 to communicate with other devices or to a server. A storage device 1010 may be used for storing data.

[0116] The embodiments disclosed herein are examples of the disclosure and may be embodied in various forms. For instance, although certain embodiments herein are described as separate embodiments, each of the embodiments herein may be combined with one or more of the other embodiments herein. Specific structural and functional details disclosed herein are not to be interpreted as limiting, but as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present disclosure in virtually any appropriately detailed structure. Like reference numerals may refer to similar or identical elements throughout the description of the figures.

[0117] The phrases “in an embodiment,” “in embodiments,” “in various embodiments,” “in some embodiments,” or “in other embodiments” may each refer to one or more of the same or different embodiments in accordance with the present disclosure. A phrase in the form “A or B” means “(A), (B), or (A and B).” A phrase in the form “at least one of A, B, or C” means “(A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C).”

[0118] The systems, devices, and / or servers described herein may utilize one or more processors to receive various information and transform the received information to generate an output. The processors may include any type of computing device, computational circuit, or anyAttorney Docket No.: 3218-5 PCT type of controller or processing circuit capable of executing a series of instructions that are stored in a memory. The processor may include multiple processors and / or multicore central processing units (CPUs) and may include any type of device, such as a microprocessor, graphics processing unit (GPU), digital signal processor, microcontroller, programmable logic device (PLD), field programmable gate array (FPGA), or the like. The processor may also include a memory to store data and / or instructions which, when executed by the one or more processors, causes the one or more processors to perform one or more methods and / or algorithms.

[0119] Any of the herein described methods, programs, algorithms or codes may be converted to, or expressed in, a programming language or computer program. The terms “programming language” and “computer program,” as used herein, each include any language used to specify instructions to a processor, and include (but is not limited to) the following languages and their derivatives: Assembler, Basic, Batch files, BCPL, C, C+, C++, Delphi, Fortran, Java, JavaScript, machine code, operating system command languages, Pascal, Perl, PL1, Python, scripting languages, Visual Basic, metalanguages which themselves specify programs, and all first, second, third, fourth, fifth, or further generation computer languages. Also included are database and other data schemas, and any other meta-languages. No distinction is made between languages which are interpreted, compiled, or use both compiled and interpreted approaches. No distinction is made between compiled and source versions of a program. Thus, reference to a program, where the programming language could exist in more than one state (such as source, compiled, object, or linked) is a reference to any and all such states. Reference to a program may encompass the actual instructions and / or the intent of those instructions.

[0120] It should be understood that the foregoing description is only illustrative of the present disclosure. Various alternatives and modifications can be devised by those skilled in the art withoutAttorney Docket No.: 3218-5 PCT departing from the disclosure. Accordingly, the present disclosure is intended to embrace all such alternatives, modifications and variances. The embodiments described with reference to the attached drawing figures are presented only to demonstrate certain examples of the disclosure. Other elements, steps, methods, and techniques that are insubstantially different from those described above and / or in the appended claims are also intended to be within the scope of the disclosure.

Claims

Attorney Docket No.: 3218-5 PCT WHAT IS CLAIMED IS:

1. A system for detection of an aerial phenomenon from captured videos, the system comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, cause the system at least to perform: accessing a first captured video and a second captured video, the first and second captured videos capturing different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the plurality of 3D real-world position in response to an electronic inquiry.

2. The system of claim 1, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform stereo matching.Attorney Docket No.: 3218-5 PCT 3. The system of claim 1, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

4. The system of claim 3, wherein in the performing triangulation, the instructions, when executed by the at least one processor, cause the system at least to perform: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.

5. The system of claim 1, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform a smoothing operation, wherein the plurality of 3D real-world positions are provided by the smoothing operation.

6. The system of claim 1, wherein the instructions, when executed by the at least one processor, further cause the system at least to perform:Attorney Docket No.: 3218-5 PCT estimating a size of the aerial phenomenon based on the plurality of 3D real-world positions; and estimating a trajectory of the aerial phenomenon based on the plurality of 3D real-world positions.

7. The system of claim 6, wherein the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

8. A method for detection of an aerial phenomenon from captured videos, the method comprising: accessing a first captured video and a second captured video, the first and second captured videos capturing different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the plurality of 3D real-world position in response to an electronic inquiry.

9. The method of claim 8, wherein the estimating the plurality of 3D real-world positions comprises performing stereo matching.Attorney Docket No.: 3218-5 PCT 10. The method of claim 8, wherein the estimating the plurality of 3D real-world positions comprises performing triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

11. The method of claim 10, wherein the performing triangulation comprises: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.

12. The method of claim 8, wherein the estimating the plurality of 3D real-world positions comprises performing a smoothing operation, wherein the plurality of 3D real-world positions are provided by the smoothing operation.

13. The method of claim 8, wherein further comprising: estimating a size of the aerial phenomenon based on the plurality of 3D real-world positions; andAttorney Docket No.: 3218-5 PCT estimating a trajectory of the aerial phenomenon based on the plurality of 3D real-world positions.

14. The method of claim 13, wherein the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

15. A non-transitory processor-readable medium storing instructions which, when executed by at least one processor of a system, cause the system at least to perform: accessing a first captured video and a second captured video, the first and second captured videos capturing different lines-of-sight of an aerial phenomenon; synchronizing, in time, frames of the first captured video and frames of the second captured video to provide synchronized first and second captured videos; detecting, based on the synchronized first and second captured videos, the aerial phenomenon captured in the synchronized first and second captured videos; estimating a plurality of three-dimensional (3D) real-world positions of the aerial phenomenon over time based on the synchronized first and second captured videos; and providing access to the synchronized first and second captured videos and the plurality of 3D real-world position in response to an electronic inquiry.

16. The non-transitory processor-readable medium of claim 15, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform stereo matching.Attorney Docket No.: 3218-5 PCT 17. The non-transitory processor-readable medium of claim 15, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform triangulation based at least on: a position and a focal vector of a first video capture device which captured the first captured video, and a position and a focal vector of a second video capture device which captured the second captured video.

18. The non-transitory processor-readable medium of claim 17, wherein in the performing triangulation, the instructions, when executed by the at least one processor, cause the system at least to perform: estimating a midpoint of a closest segment between: a first line vector based on the position and the focal vector of the first video capture device, and a second line vector based on the position and the focal vector of the second video capture device.

19. The non-transitory processor-readable medium of claim 15, wherein in the estimating the plurality of 3D real-world positions, the instructions, when executed by the at least one processor, cause the system at least to perform a smoothing operation, wherein the plurality of 3D real-world positions are provided by the smoothing operation.Attorney Docket No.: 3218-5 PCT 20. The non-transitory processor-readable medium of claim 15, wherein the instructions, when executed by the at least one processor, further cause the system at least to perform: estimating a size of the aerial phenomenon based on the plurality of 3D real-world positions; and estimating a trajectory of the aerial phenomenon based on the plurality of 3D real-world positions.

21. The non-transitory processor-readable medium of claim 20, wherein the size of the aerial phenomenon is estimated using at least one of: a trained machine learning model, or 3D shape fitting.

Images