Runway intrusion detection

Abstract

The invention relates to runway intrusion detection. An airport object location system includes a plurality of vehicle location units, a sensor system, and a model generator. The plurality of carrier position units are connected to a plurality of carriers. The plurality of vehicle location units generate vehicle location information for a plurality of vehicles in an area including an operating ground at an airport and vehicle timestamps for the vehicle location information. The sensor system is connected to the reference vehicle. The sensor system is configured to generate sensor data of the area, wherein the reference timestamp and the reference location information are associated with the sensor data. The model generator is configured to correlate the vehicle position information of the vehicle with the sensor data using the vehicle timestamp, the reference position information, and the reference timestamp to form a data set.

Description

Technical Field

[0001] The present disclosure relates generally to aircraft and, more particularly, to detecting runway incursions. Background Art

[0002] A runway incursion may occur when an object enters a runway without authorization. A runway incursion can be any incident at an airport involving an aircraft, vehicle, or person improperly present in a protected surface area designated for aircraft takeoffs and landings. A runway incursion may cause disruption and delays to airport operations.

[0003] Airport ground detection systems can alert traffic controllers to potential runway conflicts by providing detailed coverage of movement on runways and taxiways. Data can be collected by different sensors. These sensors can include, for example, electro-optical sensor systems, infrared imaging systems, inductive coils, microwave positioning sensors, radar systems, automatic dependent surveillance-broadcast (ADS-B) and other types of sensors and systems that can be used to obtain position information of different objects. The location and movement of aircraft and vehicles can be displayed on an airport map. This display can provide situational awareness about potential runway incursions.

[0004] Radar systems provide a display of the layout of aircraft and vehicle movement on runways and taxiways. Electro-optical sensor systems may also be used to provide video so that vehicles and other moving objects can be seen at the airport. These and other types of perception detection systems can inform pilots and other operators whether the runway is clear. Summary of the invention

[0005] One embodiment of the present disclosure provides an airport object location system, which includes a plurality of vehicle location units, an electro-optical sensor system, and a model generator. The plurality of vehicle location units are connected to a plurality of vehicles. The plurality of vehicle location units generate vehicle location information for a plurality of vehicles in an area including an operating surface at an airport and a vehicle timestamp for the vehicle location information. The electro-optical sensor system is connected to an aircraft. The electro-optical sensor system is configured to generate a video including frames of the area. Reference timestamps and reference location information are associated with frames in the video. The model generator is configured to associate the vehicle location information of the vehicle with frames in the video using the vehicle timestamps, reference location information, and reference timestamps to form a data set.

[0006] Another embodiment of the present disclosure provides an airport object location system, which includes a plurality of vehicle location units, a sensor system, and a model generator. The plurality of vehicle location units are connected to a plurality of vehicles. The plurality of vehicle location units generate vehicle location information for a plurality of vehicles in an area including an operating surface at an airport and a vehicle timestamp for the vehicle location information. The sensor system is connected to a reference vehicle. The sensor system is configured to generate sensor data for the area. A reference timestamp and a reference location information are associated with the sensor data. The model generator is configured to correlate the vehicle location information of the vehicle with the sensor data using the vehicle timestamp, the reference location information, and the reference timestamp to form a data set.

[0007] Another embodiment of the present disclosure provides a method for detecting objects at an airport. Using a plurality of vehicle location units connected to the vehicles, vehicle location information for a plurality of vehicles in an area including an operating surface at the airport and a vehicle timestamp for the vehicle location information are generated. Sensor data for the area is generated using a sensor system connected to a reference vehicle. A reference timestamp and reference location information are associated with the sensor data. The reference location information is for a reference vehicle. The vehicle location information of the vehicle is associated with the sensor data using the vehicle timestamp, the reference location information, and the reference timestamp to form a data set.

[0008] The features and functions can be achieved independently in various embodiments of the present disclosure or may be combined in yet other embodiments in which further details can be seen with reference to the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The novel features which are believed to be characteristic of the illustrative embodiments are set forth in the appended claims. However, the illustrative embodiments and their preferred modes of use, further objects and features thereof will be best understood by reference to the following detailed description of illustrative embodiments of the present disclosure when read in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is a diagram of intrusion detection according to an illustrative embodiment;

[0011] Figure 2 is an illustration of a block diagram of an object detection environment in accordance with an illustrative embodiment;

[0012] Figure 3 is an illustration of a sensor system in sensor data according to an illustrative embodiment;

[0013] Figure 4 is a diagram of a position generation unit according to an illustrative embodiment;

[0014] Figure 5 is a diagram of a position generation unit according to an illustrative embodiment;

[0015] Figure 6 is a diagram of a reference location unit according to an illustrative embodiment;

[0016] Figure 7 is another illustration of a reference location unit in accordance with an illustrative embodiment;

[0017] Figure 8 is a diagram of an antenna system connected to a vehicle according to an illustrative embodiment;

[0018] Fig. 9 is a diagram of an antenna system according to an illustrative embodiment;

[0019] Fig.10 is an illustration of a flow chart of a process for detecting an object in accordance with an illustrative embodiment;

[0020] Fig.11 is an illustration of a flow chart of a process for training a machine learning model according to an illustrative embodiment;

[0021] Fig.12 is an illustration of a flow chart of a process for evaluating and encouraging according to an illustrative embodiment;

[0022] Fig.13 is an illustration of a flow chart of a process for assessing and intruding according to an illustrative embodiment;

[0023] Fig.14 is an illustration of a flow chart of a process for assessing and intruding according to an illustrative embodiment;

[0024] Fig.15 is an illustration of a flow chart of a process for performing an action on an intrusion according to an illustrative embodiment;

[0025] Fig.16 is an illustration of a flowchart of a process for performing actions in response to an intrusion in accordance with an illustrative embodiment; and

[0026] Fig.17 is an illustration of a block diagram of a data processing system in accordance with an illustrative embodiment. DETAILED DESCRIPTION

[0027] The illustrative embodiments recognize and take into account one or more different considerations. Many reception-based detection systems employ cameras, lidars, and other sensors. The system may use machine learning models to detect intrusions of vehicles and other aircraft.

[0028] Training a machine learning model to detect runway incursions can be challenging. For example, obtaining the desired amount of data to train a machine learning model can be more difficult than obtaining the desired level of accuracy required to detect incursions on a runway.

[0029] One or more illustrative embodiments may improve at least one of the quantity and quality of training data used to train a machine learning model to detect intrusions. For example, in developing training data, it is important to identify different vehicles that may be involved in a runway intrusion event. An aircraft may be approaching when there are other vehicles such as aircraft or ground vehicles on the runway. Training data may be obtained for the locations of these and other vehicles or objects. This location information includes a timestamp.

[0030] When obtaining position information, a vehicle position unit can be connected to the vehicle to generate position information. These systems can be placed on a vehicle near or on a runway. In an illustrative example, a vehicle position unit is designed to work on any ground vehicle traveling in an airport environment. These so-called position units provide accurate global positioning system (GPS) data. The GPS data can be collected in a manner that is accurate to within one meter of deviation. In these examples, the GPS data can be integrated with real-time kinematics (RTK) to improve the accuracy of the vehicle position information. With real-time kinematics, the accuracy can be improved to one centimeter.

[0031] In an illustrative example, the position data collection system collects high fidelity "real" GPS and heading data from a vehicle position unit connected to the vehicle and synchronizes this information in time with the vehicle's position in the sensor system of the approaching aircraft. In other words, the position information can be synchronized or correlated with the video generated by the approaching aircraft.

[0032] Furthermore, in addition to the position information obtained from the vehicle, a video of the runway can also be used as additional information for training data. For example, an aircraft approaching the runway can generate a video that includes frames with aircraft position information and timestamps. The video can be correlated with the position of the vehicle using the timestamp.

[0033] Thus, video from an aircraft landing at an airport includes location information that can be correlated with location information of vehicles at the airport. The video and location information can form a data set that is used to train a machine learning model to detect intrusions more accurately than current technology.

[0034] In one illustrative example, an airport object location system includes a plurality of vehicle location units connected to a plurality of vehicles. The plurality of location generating units generate vehicle location information for a plurality of vehicles in an area including an operating surface at an airport and a vehicle timestamp for the vehicle location information. An electro-optical sensor system is connected to an aircraft. The electro-optical sensor system is configured to generate a video including frames of the area, wherein a reference timestamp and reference location information are associated with the frames in the video. A model generator is configured to associate the vehicle location information of the vehicles with the frames in the video using the vehicle timestamps, the reference location information, and the reference timestamps to form a data set. The data set can be used to train a machine learning model to detect intrusions into an operating surface (e.g., a runway or taxiway).

[0035] In this example, the timestamps are used to identify where in the video a vehicle exists. Therefore, this information can be used to identify useful parts of the video for creating a dataset. For example, the timestamps can be correlated with the location information of the vehicle that also has a timestamp. These timestamps can be matched to identify parts of the video that can be used for training data.

[0036] Additionally, when using video, the outline or visibility of the location generating unit attached to the vehicle may become an issue when using the video in conjunction with location information to train a machine learning model. When achieving the desired location accuracy, the antenna system may have a silhouette or visibility such that the machine learning model begins to learn the presence of the antenna rather than the presence of the vehicle. Therefore, the location generating unit may be designed to have visibility in the video that avoids the location generating unit being identified by an intrusion model trained using the dataset.

[0037] Referring now to the drawings, and in particular to Figure 1 , depicts an illustration of intrusion detection according to an illustrative embodiment. As shown, a graphical user interface / interface 100 is an interface that displays a frame 102 in a video. The video is generated by an electro-optical sensor system in an aircraft during landing. The aircraft may be selected from the group consisting of: a commercial aircraft, a passenger aircraft, a cargo aircraft, a tilt-rotor aircraft, a tilt-wing aircraft, a vertical take-off and landing aircraft, an unmanned aerial vehicle, a drone, an electric vertical take-off and landing aircraft, and a personal aerial vehicle.

[0038] As shown, a runway 104 can be seen within an area 105 in a frame 102 of a video. In this example, a vehicle 106 is present on the runway 104. The area is within an airport 107.

[0039] In this example, the video has been analyzed by an intrusion model, which has identified the presence of a vehicle 106 on the runway 104. The intrusion model is a machine learning model that is trained to detect the presence of vehicles within an airport's operating area.

[0040] As shown, a graphical indicator in the form of a bounding box 108 is associated with vehicle 106 to indicate an intrusion of vehicle 106 on runway 104. Thus, a number of different actions may be performed in response to such an intrusion indication in the video. For example, the pilot may abort the landing. The vehicle may be driven off the runway in time for the aircraft to land. In another example, the pilot may land the aircraft on a different runway. These and other actions may be performed in response to detecting an intrusion on runway 104.

[0041] As used herein, when used with reference to an item, "plurality" refers to one or more items. For example, "plurality of different actions" can be one or more different actions.

[0042] Different illustrative examples may provide training data that improves the accuracy of detecting or predicting (at least one of) an intrusion into an operating area (e.g., a runway). As used herein, when used with a list of items, the phrase "at least one of..." means that different combinations of one or more of the listed items may be used, and only one of each item in the list may be required. In other words, "at least one of..." refers to any combination of items and numbers of items that can be used in the list, but not all items in the list are required. An item may be a specific object, thing, or category.

[0043] For example, but not limited to, "at least one of item A, item B, or item C" may include item A, item A and item B, or item B. This example may also include A, item B, and item C, or item B and item C. Of course, any combination of these items may exist. In some illustrative examples, "at least one of..." may be, for example, but not limited to, two of item A; one of item B; and ten of item C; four of item B and seven of item C; or other suitable combinations.

[0044] Reference now Figure 2 , an illustration of a block diagram of an object detection environment is depicted in accordance with an illustrative embodiment. In this illustrative example, airport object location system 202 may be used to assess an intrusion into an operating surface at an airport. In this illustrative example, an intrusion assessment may be performed using intrusion model 244, which is created using data set 238. The assessment may include at least one of detecting an intrusion or predicting that an intrusion will occur.

[0045] In this illustrative example, airport object location system 202 includes plurality of vehicle location units 208 , sensor system 209 , and model generator 214 .

[0046] In this illustrative example, vehicle location unit 208 is a hardware system that may also include software. Vehicle location unit 208 is configured to connect to vehicle 220 .

[0047] When one component is "connected" to another component, the connection is a physical connection. For example, a first component (i.e., a carrier location unit) can be considered to be physically connected to a second component (i.e., a carrier) by at least one of the following: being fixed to the second component, being bonded to the second component, being mounted to the second component, being welded to the second component, being fastened to the second component, or being connected to the second component in some other suitable manner. The first component can also be connected to the second component using a third component. The first component can also be considered to be physically connected to the second component by being formed as a part of the second component, an extension of the second component, or both. In some examples, the first component can be physically connected to the second component by being located within the second component.

[0048] In one illustrative example, number of carrier location units 208 are detachably connected to number of carriers 220. In other words, number of carrier location units may be temporarily connected to number of carriers 220.

[0049] When connected to the plurality of vehicles 220, the plurality of vehicle location units 208 generate vehicle location information 222 for the plurality of vehicles 220 and vehicle timestamps 224 for the vehicle location information 222 for the area 226 including the operating surface 228 at the airport 230 when the plurality of vehicles 220 are in the area 226. The operating surface 228 can take a variety of different forms. For example, the operating surface 228 can be selected from the group including: a runway, a taxiway, a ramp, a helipad, or other surface on which operations can be performed by different types of vehicles. In one example, the operating surface 228 is a surface on which an aircraft can operate. In another example, the operating surface 220 can be a surface for other types of vehicles (e.g., a refueling truck, a de-icing truck, or other types of vehicles).

[0050] The sensor system 209 is a hardware system that may also include software. The sensor system 209 is configured to be connected to a reference vehicle 232 .

[0051] In this example, reference vehicle 232 may take a variety of different forms. For example, reference vehicle 232 may be an aircraft, a ground vehicle, or some other suitable type of vehicle to which sensor system 209 may be connected.

[0052] When connected to reference vehicle 232, sensor system 209 generates sensor data 234 for area 226. In this example, reference timestamp 235 and reference location information 236 are associated with sensor data 234. In this example, sensor data 234 may include data about multiple vehicles 220 detected by sensor system 209 in area 226.

[0053] In this illustrative example, model generator 214 may be implemented in software, hardware, firmware, or a combination thereof. When software is used, the operations performed by model generator 214 may be implemented in program instructions configured to run on hardware (e.g., a processor unit). When firmware is used, the operations performed by model generator 214 may be implemented in program instructions and data and stored in persistent memory to run on a processor unit. When hardware is employed, the hardware may include circuitry that operates to perform the operations in model generator 214.

[0054] In the illustrative examples, the hardware can take the form of at least one selected from a circuit system, an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform multiple operations. Using a programmable logic device, a device can be configured to perform multiple operations. The device can be reconfigured at a later time, or can be permanently configured to perform multiple operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field programmable logic arrays, field programmable gate arrays, and other suitable hardware devices. In addition, these processes can be implemented in organic components integrated with inorganic components, and can be composed entirely of organic components (excluding humans). For example, these processes can be implemented as circuits in organic semiconductors.

[0055] Computer system 212 is a physical hardware system and includes one or more data processing systems. When there is more than one data processing system in computer system 212, those data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing system may be selected from at least one of a computer, a server computer, a tablet computer, or some other suitable data processing system.

[0056] As shown, computer system 212 includes a plurality of processor units 216 that are capable of executing program instructions 218 that implement the processes in the illustrative example. In other words, program instructions 218 are computer-readable program instructions.

[0057] As used herein, a processor unit in plurality of processor units 216 is a hardware device and includes hardware circuits, such as on an integrated circuit, that respond to and process instructions and program codes to operate a computer.

[0058] When multiple processor units 216 execute program instructions 218 of a process, the multiple processor units 216 may be one or more processor units in the same computer or in different computers. In other words, the process may be distributed among processor units 216 on the same or different computers in computer system 212.

[0059] In addition, the plurality of processor units 216 may be of the same type or of different types. For example, the plurality of processor units 216 may be selected from at least one of a single core processor, a dual core processor, multiple processor cores, a general purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0060] In this illustrative example, as part of model creation 280, model generator 214 receives vehicle timestamps 224 and vehicle location information 222 from plurality of vehicle location units 208. Additionally, model generator 214 receives sensor data 234, reference timestamps 235, and reference location information 236 from sensor system 209.

[0061] The model generator correlates vehicle position information 222 of vehicle 220 with sensor data 234 using vehicle timestamp 224, reference position information 222, and reference timestamp 235 to form data set 238. In other words, vehicle position information 222 may be matched or associated with sensor data 234 using timestamps.

[0062] The data set includes sensor data 234, reference position information, and vehicle position information 222, as well as a correlation with the reference position of sensor data 234. In these examples, this correlation of the position of vehicle 220 includes an identification of where vehicle 220 is located within sensor data 234.

[0063] In some illustrative examples, one or more of vehicles 220 may be located outside the field of view of sensor system 209 . However, the locations of vehicles outside the field of view of sensor system 209 may still be included in data set 238 .

[0064] In this illustrative example, vehicle position information 222 for plurality of vehicles 220 and reference position information 236 for reference vehicle 232 may be used to determine the positions of plurality of vehicles 220 in sensor data 234. Identification of the positions of vehicles 220 in sensor data 234 may be performed using this information using known techniques.

[0065] For example, a process such as geo-referencing or geo-tagging may be used. A spatial relationship between the locations of the sensor systems 209 connected to the vehicles may be established using the vehicle location information 222 of the vehicles and the reference location information 236 of the sensor systems 209. The spatial relationship may include determining the relative location, orientation, and distance between the sensor systems 209 and the vehicles when the sensor data 234 was generated. This correlation may be performed using the vehicle timestamp 224 of the vehicles and the reference timestamp 235 of the sensor systems 209.

[0066] Once this relationship is determined, the position of the vehicle can be located in sensor data 234. For example, sensor data 234 is a frame of data, such as an image, a transformation matrix, or an algorithm that can be used to determine the position of the vehicle within sensor data 234 of sensor system 209. Therefore, in the geospatial context, vehicles can be more accurately identified and located based on the presence of these vehicles in sensor data 234 compared to current techniques.

[0067] Thus, the locations of multiple vehicles 220 in sensor data 234 may be accurately correlated using vehicle timestamps 224, vehicle location information 222, reference timestamps 235, and reference location information 236. This correlation using location information and timestamps generates greater accuracy in data set 238 than current techniques.

[0068] In this example, as part of model creation 280, model generator 214 operates to create intrusion model 244. In one illustrative example, model generator 214 trains machine learning model 240 using data set 238. In this example, data set 238 is a training data set.

[0069] For example, model generator 214 trains machine learning model 240 using data set 238 to detect intrusions into selected operating surface 242, thereby forming intrusion model 244. Intrusion model 244 can detect intrusions in selected area 243 that includes selected operating surface 242. In the depicted example, selected area 243 is one of area 226 at airport 230, a different area at airport 230, or another area at another airport. Data set 238 can also be used to test and validate machine learning model 240.

[0070] In another illustrative example, model generator 214 can create computer vision model 241. Computer vision model 241 is a mathematical model that uses algorithms to analyze and interpret visual information in video frames. This interpretation includes identifying objects and patterns within the frame. The computer vision model can be tested and validated using data set 238.

[0071] In another illustrative example, model generator 214 may combine machine learning model 240 and computer vision model 241. This combination of these two types of models forms intrusion model 244.

[0072] Therefore, when creating the intrusion model 244, the model generator 214 can create at least one of the machine learning model 240 or the computer vision model 241. In other words, the model generator 214 can create the intrusion model 244 by training the machine learning model 240, creating the computer vision model 241, or at least one of a combination of these two types of models.

[0073] When the computer vision model 241 and the machine learning model 240 are combined to form the intrusion model 244, the computer vision model 241 operates as a perception input layer that processes and extracts features from visual data (e.g., frames). This extracted data can be input into the machine learning model 240 for further analysis, pattern recognition, or decision making.

[0074] In this example, intrusion assessment system 250 can use intrusion model 244 to assess intrusion during deployment 282. Intrusion assessment system 250 can be software, hardware, or software and hardware within computer system 212. In this example, intrusion assessment system 250 can be implemented in program instructions 218 executable by processor unit 216.

[0075] As shown, intrusion assessment system 250 may receive current sensor data 252 from sensor system 253 connected to reference vehicle 254. Reference vehicle 254 may be an aircraft, a ground vehicle, or some other suitable type of vehicle. In this example, reference vehicle 254 is the same type of vehicle as reference vehicle 232 that generated sensor data to create data set 238. For example, if reference vehicle 232 is an airliner, reference vehicle 254 is also the same make and model of airliner. Further, in these examples, sensor system 253 is the same type of sensor system as sensor system 209. Further, in these examples, sensor system 253 is located and oriented the same as sensor system 209.

[0076] In this illustrative example, current sensor data 252 is for selected area 243. Current sensor data 252 is sensor data used to detect intrusions rather than for training or creating a model. Current sensor data 252 may be real-time sensor data. In these examples, real-time sensor data may be data generated as quickly as possible without intentional delays.

[0077] In this example, the intrusion assessment system 250 assesses an intrusion of the vehicle into the selected operating surface 242 using the current sensor data 252 and the intrusion model 244. For example, the intrusion assessment system 250 may perform an intrusion assessment by detecting an intrusion of the vehicle into the selected operating surface 242 using the current sensor data 252 and the intrusion model 244. In another example, the intrusion assessment system 250 may perform an assessment by predicting that an intrusion of the vehicle into the selected operating surface will occur using the current sensor data 252 and the intrusion model 244.

[0078] Additionally, intrusion assessment system 250 may display current sensor data 252 in a graphical user interface 257 on display system 255. Display system 255 is a physical hardware system and includes one or more display devices on which graphical user interface 257 may be displayed. The display device may include at least one of a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a computer monitor, a projector, a flat panel display, a head-up display (HUD), a head mounted display (HMD), smart glasses, augmented reality glasses, or some other suitable device that may output information to visually present the information.

[0079] Intrusion assessment system 250 may associate a graphical indicator with the vehicle in current sensor data 252 displayed on display system 255 in response to detecting an intrusion. The indicator may take a variety of different forms. For example, the graphical indicator may be selected from at least one of a bounding box, an icon, a pictogram, an ideogram, a graphic, an image, text, an animation, a bold font, a color, a line, an arrow, or other suitable graphics. In other words, the graphical indicator may be a single graphical element or a combination of graphical elements.

[0080] Next reference Figure 3 , depicts an illustration of a sensor system in sensor data according to an illustrative embodiment. In this illustrative example, example components that can be used to implement a sensor system are depicted according to an illustrative embodiment. In this illustrative example, sensor system 300 is Figure 2 An example of an implementation of sensor system 209 and sensor system 253 in FIG.

[0081] As shown, sensor system 300 may include at least one of an electro-optical sensor system 301, a light detection and ranging (LIDAR) system 302, a radar system 304, or some other type of sensor system that can detect the presence of a vehicle.

[0082] Electro-optical sensor system 301 is a hardware system that may include software. Electro-optical sensor system 301 may detect at least one of visible light, near infrared light, infrared radiation, ultraviolet light, or other types of radiation.

[0083] In this illustrative example, electro-optical sensor system 301 generates sensor data in the form of video 310, which includes frames 311. In this example, frames 311 may also be referred to as images. In other words, a frame is a single image in a sequence of frames 311, and the sequence of frames 311 is the image that forms video 310.

[0084] Additionally, the sensor data generated by electro-optical sensor system 301 may include location information, such as global positioning system (GPS) data of the location of electro-optical sensor system 301. In these illustrative examples, the sensor data also includes a timestamp.

[0085] As shown, the light detection and ranging system 302 is a hardware system that can include software and generate light detection and ranging data 320. This data can take a variety of different forms. For example, this data can be one of point cloud data, intensity data, or return information. In these examples, the light detection and ranging data 320 also includes global positioning system data and a timestamp of the location of the light detection and ranging system 302.

[0086] In this illustrative example, radar system 304 is a hardware system that may also include software. Radar system 304 generates radar data 324. In this example, radar data 324 may include different types of data, such as distance data, velocity data, reflectivity data, and other types of data. In this example, radar system 304 also generates global positioning system data and a timestamp of the location of radar system 304.

[0087] In these illustrative examples, different types of sensor data, video 310, light detection and ranging data 320, and radar data 324 are stored in a lossless format. The lossless format is used to improve the quality of the sensor data used to create the intrusion model. In some illustrative examples, there may be multiple types of sensors that generate multiple types of sensor data used to create the data set.

[0088] Next go to Figure 4 , depicts a diagram of a position generation unit according to an illustrative embodiment. In this example, the vehicle position unit 400 is Figure 2 An example of an implementation of the vehicle location unit 208 is shown in FIG.

[0089] In this illustrative example, the vehicle location unit 400 may be a portable location data collection system 401. In other words, the vehicle location unit 400 may be detachable from one vehicle to another. In this example, the vehicle location unit 400 has visibility 402 in the sensor data, which prevents the vehicle location unit from being identified by an intrusion model trained using a data set.

[0090] In this illustrative example, vehicle location unit 400 includes a number of different components. As shown, vehicle location unit 400 includes antenna system 410 and portable receiver 412.

[0091] In this example, antenna system 410 receives global positioning system signals. In addition, the antenna system is configured to be removably attached to vehicle 420. In addition, while reducing visibility in the video, antenna system 410 can have a height of 8 inches or less from surface 421 of top 422 of vehicle 420 when connected to top 422. In another illustrative example, the height from surface 421 of top 422 can be 6 inches or less. In one illustrative example, antenna system 410 can include multiple antennas 423. These multiple antennas can be removably connected to top 422. By increasing the number of antennas, higher accuracy can be obtained when generating vehicle location information 430.

[0092] In another illustrative example, visibility 402 may be reduced by connecting antenna system 410 to dashboard 425 of vehicle 420. In other words, antenna system 410 may be placed in a location where there is satellite visibility through the windshield of vehicle 420.

[0093] In this illustrative example, portable receiver 412 is connected to antenna system 410. Portable receiver 412 processes GPS signals 415 received by antenna system 410 to generate vehicle location information 430 in the form of GPS data 431 and a timestamp 432.

[0094] In another illustrative example, vehicle position unit 400 may also include real-time kinematic positioning 440. In this example, there may be additional antennas in antenna system 410 that provide an internet connection to a reference station. The reference station also receives global positioning system signals and can provide real-time corrections.

[0095] Reference now Figure 5 , another diagram of a position generation unit is depicted in accordance with an illustrative embodiment. In this example, the vehicle position unit 500 is Figure 2 Another illustrative example of an implementation for vehicle location unit 208 is shown in FIG.

[0096] In this illustrative example, vehicle location unit 500 includes antenna system 502 and receiver 504, which are Figure 4 An example of an implementation of the antenna system 410 and the portable receiver 412 is shown in FIG.

[0097] In this example, antenna system 502 includes a first global positioning system (GPS) antenna 511 , a second global positioning system (GPS) antenna 512 , an inertial measurement unit (IMU) 513 , and a track 514 .

[0098] In this example, track 514 is a structural component. A first GPS antenna 511, a second GPS antenna 512, and an inertial measurement unit 513 are connected to track 514. Furthermore, in this example, track 514 is designed to be connected to the top of the vehicle. Such connection can be made using various components such as suction cups, magnetic mounts, clamps, brackets, attachment farms, straps, and other suitable components.

[0099] The first GPS antenna 511 and the second GPS antenna 512 are hardware components that operate to receive GPS signals 509 from satellites. The inertial measurement unit 513 is a physical sensor device that can measure position information about an object connected to the device. The inertial measurement unit 513 may include sensors such as accelerometers and gyroscopes. This type of sensor can measure acceleration, orientation, angular velocity, and other information. This information can be used to determine the position of an object in three-dimensional space.

[0100] In this illustrative example, measurements 515 taken by first GPS antenna 511, second GPS antenna 512, and inertial measurement unit 513 are sent to receiver 504 and processed to generate position information 517. The position information generated by receiver 504 includes a position and a timestamp. The position may be in three-dimensional coordinates and may include an orientation. In some examples, a velocity or speed may also be included in the position information.

[0101] In this illustrative example, receiver 504 includes a number of different hardware components. As shown, receiver 504 includes a global navigation satellite system (GNSS) receiver 520, a gateway 521, and a network time protocol (NTP) network time server 523.

[0102] In this example, a global navigation satellite system (GNSS) receiver 520 determines a three-dimensional position using signals from multiple satellite measurements 515. In addition, this component can also determine velocity and timing information.

[0103] Additionally, in one illustrative example, the receiver may also implement real-time kinematics (RTK).Global Navigation Satellite System (GNSS) receiver 520 uses this feature to increase the accuracy of determining the position of the vehicle to which vehicle position unit 500 is attached.

[0104] Gateway 521 is a hardware device that provides a connection to other networks or devices. The connection can be used to transmit information such as location information 517.

[0105] In this example, the Network Time Protocol (NTP) network time server 523 is a hardware device that operates as a time server. This component provides sufficient time information to other devices within the receiver 504, such as a global navigation satellite system (GNSS) receiver 520.

[0106] Thus, in one or more illustrative examples, there are one or more technical solutions that overcome the technical problem of generating training data to train a machine learning model to assess intrusions. The assessment may include at least one of: detecting an intrusion or predicting that an intrusion will occur on an operating surface at an airport. In an illustrative example, location information is collected from vehicles in an area including the operating surface. A timestamp may be associated with the location information.

[0107] In addition, a reference vehicle may generate sensor data (e.g., video). The reference vehicle may be an aircraft in flight, an aircraft on the ground, a ground vehicle, or other suitable vehicle. The video includes the position information and timestamp of the reference vehicle.

[0108] Using the location information and the timestamp of the video, the timestamped location information from the vehicle can be correlated with the video. This related information can be used as training data for a machine learning model to generate an intrusion model that handles intrusions. The related information can also be used to test and validate computer vision models.

[0109] In addition, the illustrative example uses a vehicle location cell with visibility that prevents the machine learning model from learning the presence of an antenna system instead of the presence of the vehicle. In addition, this visibility also includes preventing the machine learning model from learning that a vehicle must include the presence of an antenna system. In other words, the visibility of the vehicle location cell enables the machine learning model to recognize the presence of the vehicle even in the absence of the antenna system. This visibility can prevent the computer vision model from detecting the antenna system instead of the vehicle, or requiring the presence of the antenna system to detect the vehicle.

[0110] The intrusion assessment system can then use the intrusion model to assess the intrusion. The predicted or detected (at least one of) intrusion and the operating area can be used to perform a number of actions. For example, the vehicle generating the video can be moved to avoid any vehicle detected or predicted to enter the operating surface. In another illustrative example, a vehicle in the operating surface or predicted to enter the operating surface can be moved to leave the operating surface or avoid the operating surface.

[0111] Computer system 212 may be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware, or a combination thereof. Thus, computer system 212 operates as a special-purpose computer system, wherein model generator 214 in computer system 212 is capable of detecting objects at the airport. In particular, at least one of model generator 214 or intrusion assessment system 250 converts computer system 212 into a special-purpose computer system compared to currently available general-purpose computer systems that do not have model generator 214.

[0112] In the illustrative example, the use of model generator 214 in computer system 212 integrates the process into a practical application for detecting objects, such as vehicles, that may be located in or predicted to enter an operating area.

[0113] Figure 2-Figure 5 The illustration of object detection environment 200 is not meant to imply physical or architectural limitations on the implementation of the illustrative embodiments. Other components other than the components shown or other components in place of the components shown may be used. Some components may be unnecessary. In addition, the blocks are presented to illustrate some functional components. When implemented in the illustrative embodiments, one or more of these blocks may be combined, divided, or combined and divided into different blocks.

[0114] For example, the reference location unit may be attached to other objects to generate the data set 238. For example, the reference location unit may be carried by a person, a cart, or other object that moves or can be moved.

[0115] Now go to Figure 6 , depicts an illustration of a reference location unit according to an illustrative embodiment. In this illustrative example, the reference location unit 600 is Figure 4 The carrier position unit 400 and Figure 5 An example implementation of the vehicle location unit 500 in FIG.

[0116] In this example, housing 601 is a plastic housing that houses the components of reference position unit 600. Housing 601 provides portability and transportation of reference position unit 600. In another example, housing 601 may include other materials, such as metal, aluminum, composite materials, or other suitable materials.

[0117] In this example, GPS antenna 610, GPS antenna 611, and IMU 612 are connected to track 614 within housing 601. Track 614 is configured to fold and fit within housing 601. In addition, NTP network time server 620, GNSS receiver 621, and gateway 622 are also located within housing 601.

[0118] There may also be other components in housing 601. For example, housing 601 here also includes a tool kit 630 and a cable 631. In this example, the connector is present on a panel (not shown) on the side 602 of housing 601. A fan is an example of another component that can be included in housing 601. The fan can provide cooling for other components within housing 601. In addition, in these illustrative examples, housing 601 is a waterproof or water-blocking housing.

[0119] Next go to Figure 7 , another diagram of a reference location unit is depicted according to an illustrative embodiment. In the illustrative examples, the same reference numeral may be used in more than one figure. Repeated use of reference numerals in different figures refers to the same elements in different figures.

[0120] In this example, a view of a side 602 of a housing 601 of a reference position unit 600 is shown. In this view, when components (e.g. Figure 6 When the GPS antenna 610, GPS antenna 611 and IMU 612) are deployed and connected to the top of the vehicle, the panel 700 on the side 602 provides these components or connections.

[0121] As shown, the panel 700 includes ANT1 701 connected to the GPS antenna 610, ANT 702 connected to the GPS antenna 611, and IMU 703 connected to the IMU 612. In addition, the panel 700 also includes NTP ANT 704 that provides an antenna connection for the NTP network time server 620.

[0122] In addition, a power connector 720 provides a connection to a power source. The power source can be a battery, a generator, a power inverter in the vehicle, or other power source. The panel 700 also includes other connections, such as a universal serial bus (USB) port 714 and a display port 715.

[0123] Reference now Figure 8 , an illustration of an antenna system connected to a vehicle is depicted in accordance with an illustrative embodiment. In this illustrative example, antenna system 800 is connected to top 801 of vehicle 802.

[0124] As shown in this example, the size of antenna system 800 reduces the visibility of antenna system 800 in the images generated for the dataset of vehicle 802. This visibility of antenna system 800 prevents the machine learning model from learning to recognize antenna system 800 instead of vehicle 802, or learning to recognize vehicle 802 only when antenna system 800 is present.

[0125] Now go to Fig. 9, depicts a diagram of an antenna system in accordance with an illustrative embodiment. A more detailed view of antenna system 800 is depicted. In this example, antenna system 800 includes GPS antenna 901 connected to track 904, GPS antenna 902, and IMU 903.

[0126] In this example, track 904 is connected to magnetic mounts 910, 911, and 912. Figure 8 The top 801 in the figure. In other examples, other types of connectors can be used. For example, a suction cup can be used instead of the magnetic mounting shown in the figure.

[0127] The illustration of antenna system 800 is an example of one antenna system used in a vehicle location unit and is not intended to limit the manner in which other antenna systems may be implemented. For example, other antenna systems may have other numbers of GPS antennas. For example, one or three GPS antennas may be used instead of the two GPS antennas shown in the figure. In addition, other components exist but are not shown in this example. For example, the cables connecting GPS antenna 901, GPS antenna 902, and IMU 903 to the receiver are not shown in this example.

[0128] In this illustrative example, height 920 of antenna system 800 is 6 inches or less. In another illustrative example, height 920 may be 8 inches or less. A higher height may be used when other components are located on top 801 (e.g., a top rack or rails).

[0129] In various illustrative examples, altitude 920 is selected such that visibility of antenna system 800 causes the machine learning model to identify the vehicle, but not the antennas in the antenna system. In these examples, altitude is selected such that visibility does not cause the machine learning model to identify antenna system 800, or only identifies vehicle 802 when antenna system 800 is present.

[0130] Additionally, other design changes can be made to the antenna system 800 to reduce visibility within the video generated for the dataset. For example, the color of the antenna system 800 can be selected to be similar to the color of the vehicle 802. In another example, the antenna system 800 can be mounted in the dashboard of the vehicle 802 so that there is sky view visibility for the GPS antenna. In another example, the antenna system 800 can be attached to the side of the vehicle 802 where there is still a sky view.

[0131] Next go to Fig.10 , an illustration of a flowchart of a process for detecting an object is depicted in accordance with an illustrative embodiment. Fig.10The processes in may be implemented in hardware, software, or both. When implemented in software, the processes may take the form of program instructions that are executed by one or more processor units in one or more hardware devices in one or more computer systems. For example, the processes may be implemented in Figure 2 The model generator 214 is implemented in the computer system 212 in the embodiment.

[0132] The process generates vehicle location information and vehicle timestamps for the vehicle location information for a plurality of vehicles in an area including an operating surface at an airport using a plurality of vehicle location units connected to the vehicles (operation 1000). The process generates sensor data for the area using a sensor system connected to a reference vehicle (operation 1002).

[0133] The process associates a reference timestamp and reference position information with the sensor data (operation 1004). In operation 1004, the reference position information is used for the position of the reference vehicle, and in particular, the reference position information is used for a sensor system connected to the reference vehicle.

[0134] The process correlates the vehicle position information of the vehicle with the sensor data using the vehicle timestamp, the reference position information, and the reference timestamp to form a data set (operation 1006). Thereafter, the process terminates.

[0135] Next go to Fig.11 , depicts an illustration of a flowchart of a process for training a machine learning model according to an illustrative embodiment. The process in this figure is a flowchart that can be used with Fig.10 An example of an additional operation formed by combining the operations in .

[0136] The process trains a machine learning model using the data set to assess intrusion into the selected operating surface, thereby forming an intrusion model (operation 1100). Thereafter, the process terminates. In operation 1100, the intrusion model assesses intrusion of the vehicle in the selected area including the selected operating surface.

[0137] Next, in Fig.12 In the figure, an illustration of a flowchart of a process for evaluation and encouragement is depicted in accordance with an illustrative embodiment. Fig.12 The processes in may be implemented in hardware, software, or both. When implemented in software, the processes may take the form of program instructions that are executed by one or more processor units in one or more hardware devices in one or more computer systems. For example, the processes may be implemented in Figure 2 The intrusion assessment system 250 is implemented in the computer system 212 in the embodiment.

[0138] The process receives current sensor data for the selected area from the new sensor system (operation 1200). The process uses the current sensor data and the intrusion model to assess the vehicle's intrusion into the selected operating surface (operation 1202). Thereafter, the process terminates.

[0139] Reference now Fig.13 , an illustration of a flowchart of a process for assessing and intrusion is depicted in accordance with an illustrative embodiment. The process in this figure is Fig.12 An example of an implementation of operation 1202 in FIG.

[0140] The process detects an intrusion of the vehicle into the selected operating surface using the current sensor data and the intrusion model (operation 1300 ). Thereafter, the process terminates.

[0141] Go to Fig.14 , an illustration of a flowchart of a process for assessing and intrusion is depicted in accordance with an illustrative embodiment. The process in this figure is Fig.12 An example of an implementation of operation 1202 in FIG.

[0142] The process uses the current sensor data and the intrusion model to predict that a vehicle intrusion of the selected operating surface will occur (operation 1400). Thereafter, the process terminates.

[0143] refer to Fig.15 , an illustration of a flow diagram of a process for performing an action in response to an intrusion is depicted in accordance with an illustrative embodiment. The process may be implemented by a vehicle involved in the intrusion. The vehicle may be a reference vehicle that generated sensor data or video. For example, the process may be implemented in an aircraft that is landing.

[0144] The process changes movement of a reference vehicle in response to detecting an intrusion (operation 1500 ).

[0145] exist Fig.16 In the present invention, an illustration of a flow diagram of a process for performing an action in response to an intrusion is depicted according to an illustrative embodiment. The process can be implemented by a vehicle involved in an intrusion. The vehicle can be an intrusion vehicle that has caused or is expected to cause an intrusion.

[0146] The process changes the movement of the intruding vehicle in response to detecting the intrusion (operation 1600). Thereafter, the process terminates. The movement may be to move the vehicle away from the operating surface, or to move to avoid entering the operating surface.

[0147] When Fig.15 Operation 1500 and Fig.16 When the movement of the vehicle is changed in operation 1600, this change can be a preprogrammed behavior when the vehicle is used for an unmanned vehicle.

[0148] The flow charts and block diagrams in different depicted embodiments show the architecture, function and operation of some possible implementations of the apparatus and method in the illustrative embodiments. In this regard, each square block in the flow chart or block diagram can represent at least one of a part of a module, segment, function or operation or step. For example, one or more in the square block can be implemented as a combination of program instructions, hardware or program instructions and hardware. When implemented in hardware, the hardware can, for example, take the form of an integrated circuit, which is manufactured or configured to perform one or more operations in the flow chart or block diagram. When implemented as a combination of program instructions and hardware, the embodiment can take the form of firmware. Each square block in the flow chart or block diagram can be implemented using a combination of a dedicated hardware system or dedicated hardware and a program instruction run by dedicated hardware that performs different operations.

[0149] In some alternative implementations of the illustrative embodiments, one or more functions noted in the blocks may not occur in the order noted in the figure. For example, in some cases, two blocks shown in succession may be performed substantially simultaneously, or the blocks may sometimes be performed in reverse order, depending on the functions involved. In addition, other blocks may be added in addition to the blocks shown in the flowchart or block diagram.

[0150] Now go to Fig.17 , depicts an illustration of a block diagram of a data processing system according to an illustrative embodiment. Data processing system 1700 may be used to implement Figure 2 1700. In this illustrative example, data processing system 1700 includes communications framework 1702, which provides communications between processor unit 1704, memory 1706, persistent storage 1708, communications unit 1710, input / output (I / O) unit 1712, and display 1714. In this example, communications framework 1702 takes the form of a bus system.

[0151] Processor unit 1704 is used to execute instructions of software that may be loaded into memory 1706. Processor unit 1704 includes one or more processors. For example, processor unit 1704 may be selected from at least one of a multi-core processor, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor. In addition, processor unit 1704 may be implemented using one or more heterogeneous processor systems in which a main processor and an auxiliary processor are present on a single chip. As another illustrative example, processor unit 1704 may be a symmetric multi-processor system that includes multiple processors of the same type on a single chip.

[0152] Memory 1706 and persistent storage 1708 are examples of storage devices 1716. A storage device is any hardware capable of storing information, such as, but not limited to, at least one of data, program instructions in a functional form, or other suitable information, in a temporary manner, a permanent manner, or both. In these illustrative examples, storage devices 1716 may also be referred to as computer-readable storage devices. In these examples, memory 1706 may be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 1708 may take various forms, depending on the particular implementation.

[0153] For example, persistent storage 1708 may include one or more components or devices. For example, persistent storage 1708 may be a hard drive, a solid-state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The media used by persistent storage 1708 may also be removable. For example, a removable hard drive may be used for persistent storage 1708.

[0154] In these illustrative examples, communications unit 1710 provides for communications with other data processing systems or devices. In these illustrative examples, communications unit 1710 is a network interface card.

[0155] Input / output unit 1712 allows for input and output of data with other devices that may be connected to data processing system 1700. For example, input / output unit 1712 may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Additionally, input / output unit 1712 may send output to a printer. Display 1714 provides a mechanism for displaying information to a user.

[0156] Instructions for at least one of an operating system, applications, or programs may be located in storage devices 1716, which are in communication with processor unit 1704 through communications framework 1702. The processes of the different embodiments may be performed by processor unit 1704 using computer-implemented instructions, which may be located in a memory, such as memory 1706.

[0157] These instructions are referred to as program instructions, computer usable program instructions, or computer readable program instructions that can be read and executed by a processor in processor unit 1704. The program instructions in different embodiments may be present on different physical or computer readable storage media (e.g., memory 1706 or persistent storage 1708).

[0158] Program instructions 1718 are located in functional form on selectively removable computer readable media 1720 and may be loaded or transferred to data processing system 1700 for execution by processor unit 1704. In these illustrative examples, program instructions 1718 and computer readable media 1720 form computer program product 1722. In the illustrative examples, computer readable media 1720 is computer readable storage media 1724.

[0159] Computer readable storage medium 1724 is a physical or tangible storage device for storing program instructions 1718, rather than a medium for propagating or transmitting program instructions 1718. Computer readable storage medium 1724 can be at least one of an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or other physical storage medium. Some known types of storage devices that include these media include: a floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or pits / lands formed in a major surface of a disk), or any suitable combination thereof.

[0160] The term computer-readable storage medium 1724 used in the present disclosure should not be interpreted as storage in the form of a transient signal itself, such as at least one of a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide, an optical pulse through a fiber optic cable, an electrical signal communicated through wires, or other transmission media.

[0161] In addition, during the normal operation of the storage device, data can be moved at certain occasional points in time. These normal operations include access, defragmentation, or garbage collection. However, these operations do not make the storage device transient because data is not transient when it is stored in the storage device.

[0162] Alternatively, program instructions 1718 may be transferred to data processing system 1700 using a computer readable signal medium. A computer readable signal medium is a signal, and may be, for example, a propagated data signal containing program instructions 1718. For example, a computer readable signal medium may be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted via a connection, such as a wireless connection, a fiber optic cable, a coaxial cable, an electrical wire, or any other suitable type of connection.

[0163] In addition, as used herein, "computer-readable medium 1720" may be singular or plural. For example, program instructions 1718 may be located in a computer-readable medium 1720 in the form of a single storage device or system. In another example, program instructions 1718 may be located in a computer-readable medium 1720 distributed in multiple data processing systems. In other words, some instructions in program instructions 1718 may be located in one data processing system, while other instructions in program instructions 1718 may be located in one data processing system. For example, a portion of program instructions 1718 may be located in a computer-readable medium 1720 in a server computer, while another portion of program instructions 1718 may be located in a computer-readable medium 1720 in a group of client computers.

[0164] The different components shown for data processing system 1700 are not meant to provide architectural limitations to the manner in which the different embodiments may be implemented. In some illustrative examples, one or more of the components may be incorporated into or otherwise form a part of another component. For example, in some illustrative examples, memory 1706, or portions thereof, may be incorporated into processor unit 1704. The different illustrative embodiments may be implemented in a data processing system including components in addition to or in place of those shown for data processing system 1700. Fig.17 Other components shown in the illustrative examples may be varied from those shown. The different embodiments may be implemented using any hardware device or system capable of executing program instructions 1718.

[0165] In these examples, program instructions 1718 may be program instructions for at least one of model generator 214 or intrusion assessment system 250. These program instructions may be executed on a computer system to perform one or more of the different operations described in the illustrative examples. The program instructions on computer-readable storage medium 1724 in computer program product 1720 may be loaded onto one or more processor units in a computer system to perform one or more of the operations described for the different illustrative examples.

[0166] Therefore, an illustrative example provides a method, apparatus, system, and computer program product for object identification. The subject identifies budget intrusions on an operational surface at an airport. In an illustrative example, an airport object location system includes a plurality of vehicle location units, a sensor system, and a model generator. The plurality of vehicle location units are connected to a plurality of vehicles. The plurality of vehicle location units generate vehicle location information for a plurality of vehicles in an area including the operational surface at the airport and a vehicle timestamp for the vehicle location information. The sensor system is connected to a reference vehicle. The sensor system is configured to generate sensor data for the area, wherein a reference timestamp and reference location information are associated with the sensor data. The model generator is configured to correlate the vehicle location information of the vehicle with the sensor data using the vehicle timestamp, the reference location information, and the reference timestamp to form a data set.

[0167] The various illustrative examples provide the ability to increase the amount of training data used to train a machine learning model to form an intrusion model that can detect or predict intrusions into an operating surface at an airport. The operating surface can be a runway, a taxiway, or other area of ​​an airport where aircraft may move. In addition, the illustrative examples can increase the amount of data used to test and validate a computer vision model created to form an intrusion model compared to current techniques for collecting data.

[0168] This application also includes the following examples:

[0169] Example 1. An airport object location system (202), comprising:

[0170] a plurality of vehicle location units (208) connected to a plurality of vehicles (220), wherein the plurality of vehicle location units (208) generate vehicle location information (222) for the plurality of vehicles (220) in an area (105, 226) including an operating surface (228) at an airport (107, 230) and a vehicle timestamp (224) for the vehicle location information (222);

[0171] an electro-optical sensor system (300) (301) coupled to an aircraft, wherein the electro-optical sensor system (300) (301) is configured to generate a video (310) comprising frames (311) of the region (105, 226), wherein reference timestamps (235) and reference position information (236) are associated with the frames (311) in the video (310); and

[0172] A model generator (214) is configured to correlate the vehicle position information (222) of the vehicle (220) with the frame (311) in the video (310) using the vehicle timestamp (224), the reference position information (236) and the reference timestamp (235) to form a data set (238).

[0173] Example 2. The airport object location system (202) of Example 1, wherein the model generator (214) is configured to:

[0174] The data set (238) is used to train a machine learning model (240) to detect intrusions into a selected operating surface (242), thereby forming an intrusion model (244), wherein the intrusion model (244) detects intrusions in a selected area (243) including the selected operating surface (242).

[0175] Example 3. The airport object location system (202) of Example 1, wherein the model generator (214) is configured to:

[0176] creating a computer vision model (241) to detect intrusions into a selected operating surface (242); and

[0177] The computer vision model (241) is tested and validated using the dataset (238).

[0178] Example 4. The airport object location system (202) of Example 2, further comprising:

[0179] An intrusion assessment system (250) configured to:

[0180] receiving a current video of the selected area (243); and

[0181] Intrusion of the selected operating surface (242) by a vehicle (420) is detected using the current video and the intrusion model (244).

[0182] Example 5. The airport object location system (202) of Example 4, wherein the intrusion assessment system (250) is configured to:

[0183] displaying the current video on a display system (255); and

[0184] In response to detecting the intrusion, a graphical indicator is associated with the vehicle (420) in the current video displayed on the display system (255).

[0185] Example 6. An airport object location system (202) according to Example 2, wherein the selected area (243) is one of the area at the airport (107, 230), a different area at the airport (107, 230), or another area at another airport.

[0186] Example 7. An airport object location system (202) according to Example 1, wherein a vehicle location unit (400) among the plurality of vehicle location units (208) has visibility in a video (310) that prevents the vehicle location unit from being identified by an intrusion model (244) created using the data set (238).

[0187] Example 8. The airport object location system (202) of Example 1, wherein a vehicle location unit (400) of the plurality of vehicle location units (208) comprises:

[0188] an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a vehicle, wherein the antenna system (410, 502) is 8 inches or less in height from a surface on a top of the vehicle; and

[0189] A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

[0190] Example 9. The airport object location system (202) of Example 8, wherein the antenna system (410, 502) includes a plurality of antennas configured to be removably attached to the top of the vehicle.

[0191] Example 10. The airport object location system (202) of Example 1, wherein a vehicle location unit (400) of the plurality of vehicle location units (208) comprises:

[0192] an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a dashboard of a vehicle; and

[0193] A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

[0194] Example 11. An airport object location system (202) according to Example 1, wherein the aircraft is selected from the group consisting of: commercial aircraft, passenger aircraft, cargo aircraft, tilt-rotor aircraft, tilt-wing aircraft, vertical take-off and landing aircraft, unmanned aerial vehicles, drones, electric vertical take-off and landing aircraft, and personal aircraft.

[0195] Example 12. The airport object location system (202) of Example 1, wherein the operating surface (228) is selected from the group consisting of: a runway, a taxiway, a ramp, and a helipad.

[0196] Example 13. An airport object location system (202), comprising:

[0197] a plurality of vehicle location units (208) connected to a plurality of vehicles (220), wherein the plurality of vehicle location units (208) generate vehicle location information (222) for the plurality of vehicles (220) in an area (105, 226) including an operating surface (228) at an airport (107, 230) and a vehicle timestamp (224) for the vehicle location information (222);

[0198] a sensor system (300) connected to a reference vehicle (232), wherein the sensor system (300) is configured to generate sensor data (234) of the area (105, 226), wherein a reference timestamp (235) and reference location information (236) are associated with the sensor data (234); and

[0199] A model generator (214) is configured to correlate the vehicle position information (222) of the vehicle (220) with the sensor data (234) using the vehicle timestamp (224), the reference position information (236) and the reference timestamp (235) to form a data set (238).

[0200] Example 14. The airport object location system (202) of Example 13, wherein the model generator (214) is configured to:

[0201] A machine learning model (240) is trained using the dataset (238) to assess intrusion into a selected operating surface (242), thereby forming an intrusion model (244), wherein the intrusion model (244) assesses intrusion in a selected area (243) including the selected operating surface (242).

[0202] Example 15. The airport object location system (202) of Example 13, wherein the model generator (214) is configured to:

[0203] creating a computer vision model (241) to detect intrusions into a selected operating surface (242); and

[0204] The computer vision model (241) is tested and validated using the dataset (238).

[0205] Example 16. The airport object location system (202) of Example 14, further comprising:

[0206] An intrusion assessment system (250) configured to:

[0207] receiving current sensor data (234) for the selected area (243); and

[0208] Intrusion of the selected operating surface (242) by a vehicle (420) is assessed using the current sensor data (234) and an intrusion model (244) created using the data set (238).

[0209] Example 17. The airport object location system (202) of Example 16, wherein when evaluating the intrusion of the vehicle (420) into the selected operating surface (242), the intrusion assessment system (250) is configured to:

[0210] The intrusion of the selected operating surface (242) by the vehicle (420) is detected using the current sensor data (234) and the intrusion model (244).

[0211] Example 18. The airport object location system (202) of Example 16, wherein when evaluating the intrusion of the vehicle (420) into the selected operating surface (242), the intrusion model (244) is configured to:

[0212] The intrusion of the selected operating surface (242) by the vehicle (420) is predicted to occur using the current sensor data (234) and the intrusion model (244).

[0213] Example 19. An airport object location system (202) according to Example 13, wherein the intrusion model (244) is selected from at least one of a machine learning model (240) or a computer vision model (241) trained using the data set (238).

[0214] Example 20. The airport object location system (202) of Example 16, wherein the intrusion assessment system (250) is configured to:

[0215] displaying the current sensor data (234) on a display system (255); and

[0216] In response to detecting the intrusion, a graphical indicator is associated with the vehicle in the current sensor data (234) displayed on the display system (255).

[0217] Example 21. An airport object location system (202) according to Example 14, wherein the selected area (243) is one of the areas (105, 226) at the airport (107, 230), a different area at the airport (107, 230), or another area at another airport.

[0218] Example 22. An airport object location system (202) according to Example 13, wherein the sensor data (234) is selected from at least one of a video (310) comprising frames (311), light detection and ranging data (320), or radar data (324).

[0219] Example 23. An airport object location system (202) according to Example 13, wherein the sensor system (300) is selected from at least one of an electro-optical sensor system (301), a light detection and ranging system (302), or a radar system (304).

[0220] Example 24. An airport object location system (202) according to Example 13, wherein a vehicle location unit (400) among the plurality of vehicle location units (208) has visibility in the sensor data (234) that prevents an intrusion model (244) trained using the data set (238) from identifying the vehicle location unit (400).

[0221] Example 25. The airport object location system (202) of Example 13, wherein a vehicle location unit (400) of the plurality of vehicle location units (208) is a portable location data collection system (401).

[0222] Example 26. The airport object location system (202) of Example 13, wherein a location generation unit in the plurality of vehicle location units (208) comprises:

[0223] an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a vehicle (420), wherein the antenna system (410, 502) is 8 inches or less in height from a surface (421) of a top (422) of the vehicle (420); and

[0224] A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

[0225] Example 27. An airport object location system (202) according to Example 24, wherein the antenna system (410, 502) includes a plurality of antennas (423) configured to be removably attached to the top (422) of the vehicle (420).

[0226] Example 28. The airport object location system (202) of Example 13, wherein a location generation unit in the plurality of vehicle location units (208) comprises:

[0227] an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a dashboard (425) of a vehicle (420); and

[0228] A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

[0229] Example 29. A method for detecting an object at an airport (107, 230), the method comprising:

[0230] Generating (1000) vehicle location information (222) for the plurality of vehicles (220) in an area (105, 226) including an operating surface (228) at an airport (107, 230) and a vehicle timestamp (224) for the vehicle location information (222) using a plurality of vehicle location units (208) connected to the plurality of vehicles (220);

[0231] generating (1002) sensor data (234) of the area (105, 226) using a sensor system (300) coupled to a reference vehicle (232);

[0232] associating (1004) a reference timestamp (235) and reference position information (236) with the sensor data (234), wherein the reference position information (236) is for the reference vehicle (232); and

[0233] The vehicle position information (222) of the vehicle (220) is correlated (1006) with the sensor data (234) using the vehicle timestamp (224), the reference position information (236), and the reference timestamp (235) to form a data set (238).

[0234] Example 30. The method according to Example 29, further comprising:

[0235] A machine learning model (240) is trained (1100) using the data set (238) to evaluate intrusion into a selected operating surface (242), thereby forming an intrusion model (244), wherein the intrusion model (244) evaluates intrusion of a vehicle (220) in a selected area (243) including the selected operating surface (242).

[0236] Example 31. The method according to Example 30, further comprising:

[0237] receiving (1200) current sensor data (234) for the selected area (243) from a new sensor system (300); and

[0238] Intrusion of the selected operating surface (242) by a vehicle (420) is evaluated (1202) using the current sensor data (234) and the intrusion model (244).

[0239] Example 32. The method of example 31, wherein evaluating (1202) the intrusion of the vehicle (420) into the selected operating surface (242) comprises:

[0240] The intrusion of the selected operating surface (242) by the vehicle (420) is detected (1300) using the current sensor data (234) and the intrusion model (244).

[0241] Example 33. The method of example 31, wherein evaluating (1202) the vehicle's intrusion into the selected operating surface (242), the intrusion model (244) comprising:

[0242] The intrusion of the selected operating surface (242) by the vehicle (420) is predicted (1400) to occur using the current sensor data (234) and the intrusion model (244).

[0243] Example 34. The method according to Example 31, further comprising:

[0244] The movement of the intruding vehicle is altered (1600) in response to detecting the intrusion.

[0245] Example 35. The method according to Example 31, further comprising:

[0246] The movement of another reference vehicle is altered in response to detecting the intrusion.

[0247] The description of different illustrative embodiments is presented for the purpose of illustration and description, and is not meant to be exhaustive, nor is it limited to the embodiments of the disclosed forms. Different illustrative examples describe components that perform actions or operations. In the illustrative embodiments, the components can be configured to perform the described actions or operations. For example, the components can have a configuration or design for the structure that provides the component with the ability to perform the actions or operations described as being performed by the component in the illustrative examples. In addition, in the context of using the terms "includes", "including", "having", "including" and their variations in this article, these terms are intended to be included in a manner similar to the term "comprises" as an open transition word, without excluding any additional or other elements.

[0248] Many modifications and variations will be apparent to those of ordinary skill in the art. In addition, different illustrative embodiments may provide different features than other desired embodiments. The selected embodiment or embodiments are chosen and described in order to best explain the principles of the embodiments, the practical application, and to enable others of ordinary skill in the art to understand the disclosure of various embodiments with various modifications suitable for the intended specific use.

Claims

1. An airport object location system (202), comprising: a plurality of vehicle location units (208) connected to a plurality of vehicles (220), wherein the plurality of vehicle location units (208) generate vehicle location information (222) for the plurality of vehicles (220) in an area (105, 226) including an operating surface (228) at an airport (107, 230) and a vehicle timestamp (224) for the vehicle location information (222); an electro-optical sensor system (300) (301) coupled to an aircraft, wherein the electro-optical sensor system (300) (301) is configured to generate a video (310) comprising frames (311) of the region (105, 226), wherein reference timestamps (235) and reference position information (236) are associated with the frames (311) in the video (310); and A model generator (214) is configured to correlate the vehicle position information (222) of the vehicle (220) with the frame (311) in the video (310) using the vehicle timestamp (224), the reference position information (236) and the reference timestamp (235) to form a data set (238).

2. The airport object location system (202) of claim 1, wherein the model generator (214) is configured to: The data set (238) is used to train a machine learning model (240) to detect intrusions into a selected operating surface (242), thereby forming an intrusion model (244), wherein the intrusion model (244) detects intrusions in a selected area (243) including the selected operating surface (242).

3. The airport object location system (202) of claim 1, wherein the model generator (214) is configured to: creating a computer vision model (241) to detect intrusions into a selected operating surface (242); and The computer vision model (241) is tested and validated using the dataset (238).

4. The airport object location system (202) of claim 2, further comprising: An intrusion assessment system (250) configured to: Receiving a current video of the selected area (243); and Intrusion of the selected operating surface (242) by a vehicle (420) is detected using the current video and the intrusion model (244).

5. The airport object location system (202) of claim 4, wherein the intrusion assessment system (250) is configured to: displaying the current video on a display system (255); and In response to detecting the intrusion, a graphical indicator is associated with the vehicle (420) in the current video displayed on the display system (255).

6. The airport object location system (202) of claim 2, wherein the selected area (243) is one of the area at the airport (107, 230), a different area at the airport (107, 230), or another area at another airport.

7. The airport object location system (202) of claim 1, wherein a vehicle location unit (400) among the plurality of vehicle location units (208) has visibility in a video (310) that prevents the vehicle location unit from being identified by an intrusion model (244) created using the data set (238).

8. The airport object location system (202) of claim 1, wherein a vehicle location unit (400) in the plurality of vehicle location units (208) comprises: an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a vehicle, wherein the antenna system (410, 502) is 8 inches or less in height from a surface on a top of the vehicle; and A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

9. The airport object location system (202) of claim 8, wherein the antenna system (410, 502) comprises a plurality of antennas configured to be removably attached to the top of the vehicle.

10. The airport object location system (202) of claim 1, wherein a vehicle location unit (400) in the plurality of vehicle location units (208) comprises: an antenna system (410, 502) that receives global positioning system signals (509), wherein the antenna system (410, 502) is configured to be removably attached to a dashboard of a vehicle; and A portable receiver (412) connected to the antenna system (410, 502), wherein the portable receiver (412) processes the GPS signal (509) received by the antenna system (410, 502) to generate the vehicle position information (222) and a timestamp (432) in the form of GPS data (431).

Images