Method and system for unmanned vehicle inspection and analysis of infrastructure and assets

Abstract

Methods and systems for enabling automated asset management are provided herein. In an embodiment, an asset data collector is configured to automatically collect asset data related to assets in an intermodal facility according to a task plan. The automated data collection configurator is configured to configure a task plan based on one or more parameters, and instantiate the asset data collector to initiate collection of asset data according to the task plan. The automated asset data analyzer is configured to identify data in which one or more assets occur, identify one or more assets detected in the dataset, check the one or more assets based on the dataset in which the one or more assets are detected to determine a condition of the one or more assets, and output the condition of the one or more assets.

Description

Technical Field

[0001] The present disclosure relates generally to automated asset management and, more particularly, to asset detection and identification techniques and systems for automated asset inspection. Background Art

[0002] Many current operations require extensive infrastructure. A typical infrastructure operation can include numerous and diverse assets, spread across a large area for operational use. For example, in current rail-based transportation systems, terminals or intermodal facilities may be used to facilitate the transportation and / or storage of freight. In these examples, these intermodal facilities may serve as rail freight hubs, allowing freight to be transported to and from the facility via ground vehicles, and also allowing freight to be loaded onto and unloaded from rail cars.

[0003] To support transportation operations, these intermodal facilities can include a wide range of infrastructure assets, such as inventory, storage containers, transport vehicles, rail cars, trailers, tracks, warehouses, buildings, security fences, storage spaces, roads, and signs. These assets may be in use during operations, stored at the intermodal facility during transportation, transferred through the intermodal facility, permanently attached to the intermodal facility, or otherwise operationally associated with the facility. Because these assets are critical to operations, and in some cases, even essential to them, their management is crucial. Asset management can include inspecting assets to determine / confirm their condition, location, and quantity.

[0004] Current asset management solutions involve manual inspections of these assets. For example, users can manually approach assets for inspection and determine / confirm their condition, location, quantity, and so on. However, given the sheer number of assets involved in operations, this manual approach can be tedious, time-consuming, and prone to error. Furthermore, different personnel with varying skills may be required depending on the inspection activity, potentially increasing the required human resources. For example, inspecting an asset to determine its location within an intermodal facility may not require specialized skills, but inspecting an item to determine whether it requires maintenance or is safe may require personnel with knowledge of certain safety principles. Therefore, current asset management systems suffer from a technical issue: a lack of automation for asset management and inspection operations. Summary of the Invention

[0005] The present disclosure achieves technical advantages as a system, method, and computer-readable storage medium that provide automated asset management and inspection capabilities. The present disclosure provides a system that is integrated into a practical application and has meaningful limitations for performing automatic collection and automatic analysis of asset-related data (e.g., image data, metadata, etc.) to detect, identify, and / or inspect assets that appear or are represented in the collected asset data. Embodiments of the present disclosure may include an automated data collection configurator with a mission plan configuration function, and / or an asset data collector that can perform automated data collection according to a mission plan. In an embodiment, the automated asset data analyzer can be configured to perform automated analysis on the collected asset data to detect (e.g., using a machine learning (ML) model), identify (e.g., using an ML model), and / or inspect (e.g., including determining the condition and / or location of the identified objects) assets that appear or are represented in the collected asset data according to the mission plan configured by the automated data collection configurator.

[0006] The present disclosure addresses the technical problem of a lack of automated functionality in asset management and inspection operations by at least providing a novel and inventive mechanism for automating asset management and inspection operations. The technical solutions provided herein are missing from traditional systems and are not simply the application of manual processes to a computer environment, but also include the functionality of implementing technical processes to replace or supplement current manual solutions for asset management and inspection. Therefore, the present disclosure goes far beyond the scope of simply applying manual processes to computers. For example, the solutions provided by the present disclosure include configuration and instantiation functionality to enable automated asset data collection, which may include image data, and perform automatic analysis on the collected asset image data to detect (e.g., using a machine learning (ML) model), identify (e.g., using an ML model) and / or inspect (e.g., including determining the condition and / or location of the identified objects) assets that appear or are represented in the collected asset image data. Therefore, the claims herein are necessarily rooted in computer technology because they can overcome problems arising in the fields of machine learning and image processing.

[0007] Thus, the present disclosure discloses concepts that are inseparable from computer technology, thereby providing technical advantages that can realize the functions of an asset management system for automating the collection of asset data and automating the analysis of the collected asset data. The systems and techniques of the embodiments provide an improved asset management system that is capable of automatically collecting data related to assets in an intermodal facility more accurately and with less lag time, and automatically analyzing the collected asset data.

[0008] An object of the present invention is to provide a system for automated asset management and inspection. Another object of the present invention is to provide a method for automated asset management and inspection. The present disclosure provides the above and other objects, including at least the following embodiments.

[0009] In one particular embodiment, an automated asset management system is provided. The system includes an asset data collector configured to automatically collect asset data related to one or more assets in an intermodal facility according to a mission plan. The system also includes an automated data collection configurator configured to configure the mission plan based on one or more parameters and instantiate the asset data collector. In an embodiment, instantiating the asset data collector causes the asset data collector to start according to the mission plan to automatically collect asset data related to one or more assets in the intermodal facility. The system also includes an automated asset data analyzer configured to detect one or more assets in an asset data set collected by the asset data collector. In an embodiment, detecting one or more assets in the data set includes identifying data in which one or more assets appear. The automated asset data analyzer may also be configured to identify the one or more assets detected in the data set, examine the one or more assets based on the data set in which the one or more assets were detected to determine a condition of the one or more assets, and output the condition of the one or more assets.

[0010] In another embodiment, a method for automated asset management is provided. The method includes configuring a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on one or more parameters, and instantiating an asset data collector to automatically collect asset data associated with the one or more assets. In an embodiment, instantiating the asset data collector causes the asset data collector to be started according to the mission plan to automatically collect asset data associated with the one or more assets in the intermodal facility. The method also includes automatically collecting, by the asset data collector, asset data associated with the one or more assets in the intermodal facility according to the mission plan, and detecting one or more assets in an asset data set collected by the asset data collector. In an embodiment, detecting the one or more assets in the data set includes identifying data in which the one or more assets appear. The method also includes identifying the one or more assets detected in the data set, inspecting the one or more assets based on the data set in which the one or more assets were detected to determine a status of the one or more assets, and outputting the status of the one or more assets.

[0011] In another embodiment, a computer-based automated asset management tool is provided. This computer-based tool may include a non-transitory computer-readable medium storing computer code that, when executed by a processor, causes a computing device to perform operations. The operations include configuring a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on one or more parameters, and instantiating an asset data collector to automatically collect asset data associated with the one or more assets. In an embodiment, instantiating the asset data collector causes the asset data collector to be initiated according to the mission plan to automatically collect asset data associated with the one or more assets in the intermodal facility. The operations also include automatically collecting asset data associated with the one or more assets in the intermodal facility by the asset data collector according to the mission plan, and detecting the one or more assets in an asset data set collected by the asset data collector. In an embodiment, detecting the one or more assets in the data set includes identifying data in which the one or more assets appear. The operations also include identifying the one or more assets detected in the data set, inspecting the one or more assets based on the data set in which the one or more assets are detected to determine a condition of the one or more assets, and outputting the condition of the one or more assets.

[0012] The features and technical advantages of the present disclosure have been broadly outlined above so that the detailed description of the present disclosure below can be better understood. Additional features and advantages of the present disclosure will be described below, which form the subject of the claims of the present disclosure. It should be understood by those skilled in the art that the disclosed concepts and specific embodiments can be easily used as a basis for modifying or designing other structures to achieve the same purposes of the present disclosure. It should also be recognized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the present disclosure as described in the appended claims. The novel features of the organization and method of operation that are believed to be characteristic of the present disclosure, as well as further objects and advantages, can be better understood from the following description in conjunction with the accompanying drawings. It should be expressly understood that each of the figures is for illustration and description purposes only and does not represent a definition of the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more fully understand the present disclosure, the following description is given in conjunction with the accompanying drawings:

[0014] Figure 1 is a block diagram of an exemplary system configured for providing automated asset management and inspection capabilities and functionality according to an embodiment of the present disclosure.

[0015] Figure 2 is an example block diagram of configuration operations for an automated asset management system implemented according to an embodiment of the present disclosure.

[0016] Figure 3is a block diagram illustrating an example of the operation of an automated asset management system in an intermodal facility implemented according to an embodiment of the present disclosure.

[0017] Figure 4 A high-level operational flow diagram of a system configured to provide automated asset management according to an embodiment of the present disclosure is shown.

[0018] It should be understood that the drawings are not necessarily drawn to scale and that the disclosed embodiments are sometimes shown in diagrammatic form and in fragmentary view. In some cases, details that are not necessary for an understanding of the disclosed methods and apparatuses or that render other details difficult to understand may have been omitted. It should be understood, of course, that the present disclosure is not limited to the particular embodiments shown herein. DETAILED DESCRIPTION

[0019] The disclosure and its various features and advantageous details set forth in the following written description will be more fully explained with reference to the non-limiting examples included in the accompanying drawings and the content detailed in the specification. Descriptions of well-known components are omitted herein so as not to unnecessarily obscure the main features described herein. The following examples are intended to facilitate an understanding of how the disclosure may be implemented and practiced. One of ordinary skill in the art will understand that the disclosure means that any suitable combination of the following functions or exemplary embodiments may be combined to achieve the claimed subject matter. The disclosure includes a representative number of species within the genus, or structural features common to members of the genus, so that one of ordinary skill in the art can identify members of the genus. Therefore, these examples should not be construed as limiting the scope of the claims.

[0020] A person of ordinary skill in the art will understand that any system claim set forth herein encompasses all elements and limitations disclosed therein, and therefore each system claim is required to be considered as a whole. Any reasonably foreseeable item that has a functional relationship with the claim also falls within the relevant scope. After thoroughly understanding the disclosure and claims in the application as filed, the examiner searched for prior art disclosed in patents and other published documents (i.e., non-patent literature). Therefore, as evidenced by the issuance of this patent, the prior art fails to disclose or teach the elements and limitations set forth in the claims supported by the specification and drawings, and therefore the claims as filed are patentable under the applicable laws and rules of this jurisdiction.

[0021] Various embodiments of the present disclosure are directed to systems and techniques that can provide automated asset management and inspection capabilities. In certain embodiments, a system implemented in accordance with the present disclosure may include functionality for automatically collecting asset-related data (e.g., image data, metadata, etc.) and for performing automated analysis of the collected data to detect, identify, and / or inspect assets that appear or are represented in the collected asset data. Embodiments of the present disclosure may include an automated data collection configurator having a mission plan configuration capability, and / or an asset data collector that can perform automated data collection according to a mission plan. In embodiments, the automated asset data analyzer can be configured to perform automated analysis of the collected asset data to detect (e.g., using a machine learning (ML) model), identify (e.g., using an ML model), and / or inspect (e.g., including determining the condition and / or location of the identified objects) assets that appear or are represented in the collected asset data according to a mission plan configured by the automated data collection configurator.

[0022] Figure 1 is a block diagram of an exemplary system 100 configured to provide automated asset management and inspection capabilities and functions according to an embodiment of the present disclosure. Figure 1 As shown, system 100 may include a server 110, a user terminal 140, an asset data collector 130, and a network 145. These components, and their individual components, may operate in conjunction to provide the functionality discussed herein. For example, in operation consistent with an embodiment, a mission plan may be configured through the coordinated functionality of server 110 and user terminal 140. The mission plan may define one or more parameters that may configure asset data collector 130 to perform automated data collection of assets in accordance with the mission plan. The functionality of server 110 may be used to perform automated analysis of the collected asset data to detect, identify, and / or inspect assets that appear or are represented in the collected asset data.

[0023] It is worth noting that the current discussion focuses on a specific application of asset management, specifically the management of assets in intermodal facilities. However, it should be recognized that the techniques disclosed herein are also applicable to other asset management applications. For example, the techniques disclosed herein can also be applied to managing retail inventory in a warehouse, or to managing vehicle inventory in a parking lot, or to managing any number of assets in any number of environments. Therefore, the discussion herein of intermodal facility assets should not be construed as limiting in any way.

[0024] like Figure 1 The discussion of the illustrated system 100 will be further based on Figure 2 and 3 Examples in . Figure 2 is an example block diagram of configuration operations for an automated asset management system implemented according to an embodiment of the present disclosure. Figure 3 is a block diagram illustrating an example of the operation of an automated asset management system in an intermodal facility implemented according to an embodiment of the present disclosure.

[0025] It is worth noting that the functional blocks and components of the system 100 of the embodiments of the present invention can be implemented using a processor, an electronic device, a hardware device, an electronic component, a logic circuit, a memory, a software code, a firmware code, etc., or any combination thereof. For example, one or more functional modules, or portions thereof, can be implemented as discrete gate or transistor logic, discrete hardware components, or a combination thereof, configured to provide logic for performing the functions described herein. Additionally or alternatively, when implemented in software, one or more functional blocks or portions thereof can include code segments that can be operated on a processor to provide logic for performing the functions described herein.

[0026] It should also be noted that the various components of system 100 are shown as individual and independent components. However, it is understood that the various components shown can be implemented as a single component (e.g., a single application, server module, etc.), can be functional components of a single component, or the functionality of these different components can be distributed across multiple devices / components. In such embodiments, the functionality of each corresponding component can be aggregated from the functionality of multiple modules located in a single or multiple devices.

[0027] It should also be noted that the functionality described with reference to each different functional block of system 100 described herein is for illustrative purposes only and not for purposes of limitation, and that the functionality provided by the different functional blocks may be combined into a single component or may be provided by computing resources located in a cloud-based environment accessible via a network (e.g., one of networks 145).

[0028] The user terminal 140 can be implemented as a mobile device, a smartphone, a tablet computing device, a personal computing device, a laptop computing device, a desktop computing device, a vehicle computer system, a personal digital assistant (PDA), a smartwatch, other types of wired and / or wireless computing devices, or any portion thereof. In an embodiment, the user terminal 140 can be configured to provide an interface (e.g., a graphical user interface (GUI)) that is configured to facilitate user interaction with the system 100 (e.g., via the network 145) to execute and utilize the functionality provided by the server 110. In an embodiment, for example, the functionality of the user terminal 140 can enable a user to specify one or more parameters that can be used by the automated data collection configurator 120 of the server 110 to configure a usable or executable mission plan to implement automated data collection. For example, the mission plan can be used or executed by the asset data collector 130 to collect asset data according to the mission plan.

[0029] In some embodiments, the mission plan may include a flight path or route that the asset data collector 130 will follow to collect data related to the assets of the intermodal facility. In embodiments, a user may specify parameters that may be used to determine (e.g., via the automated data collection configurator 120) the mission plan flight path. For example, a user may specify an actual path or mode, an operating area, a set of waypoints, or any combination thereof. For example, Figure 2 As shown in the example in FIG, GUI 220 enables a user to specify a flight pattern 241 (e.g., by drawing the pattern by representing at least a portion of intermodal facility 225). During the automated asset data collection operation, asset data collector 130 may follow flight pattern 241 starting and ending at start / end point 240 to collect asset data according to the mission plan. In some embodiments, as described below, flight pattern 241 may be automatically determined by system 10 (e.g., by automated data collection configurator 120) based on parameters specified for the mission plan, or determined by asset data collector 130, rather than being manually drawn or otherwise manually drawn by the user.

[0030] In an embodiment, the GUI 220 enables the user to specify an operating area (e.g., Figure 2 As shown in the example box 243, a box of a specific size and / or shape is drawn around the desired operating area, or a location (e.g., Lot A, Rows 1 and 2) containing the desired operating area is designated by representing at least a portion of the intermodal facility 225. In embodiments, the system 100 (e.g., via functionality of the automated data collection configurator 120 or functionality of the asset data collector 130) may determine a flight pattern (e.g., flight pattern 241) for the asset data collector 130 to follow during automated asset data collection based on the mission plan parameters.

[0031] In an embodiment, GUI 220 enables a user to specify a set of waypoints (e.g., waypoints 245-247, such as Figure 2 In some embodiments, the flight pattern (e.g., flight pattern 242) may be defined by the asset data collector 130 to collect asset data in the intermodal facility 225 during automated data collection. In some embodiments, the asset data collector 130 may fly between waypoints during automated data collection. In some embodiments, a flight pattern (e.g., flight pattern 242) may be determined based on a set of waypoints (e.g., by functionality of the automated data collection configurator 120 or functionality of the asset data collector 130). In these embodiments, the asset data collector 130 may follow the flight pattern, starting and ending at the start / end point 240, to collect asset data according to the mission plan.

[0032] In some embodiments, the GUI 220 allows the user to further specify the parameters of the mission plan. For example, the user can specify a mission plan title 201 to identify the mission plan. This functionality can be important because it allows the user to reload a previously configured mission plan, allowing the user to reuse a mission plan without having to specify all of its parameters, or to reload the parameters of a previous mission plan and modify some of them without having to specify all of them again. The GUI 220 allows the user to specify a flight altitude 206 at which the asset data collector 130 will operate when collecting asset data. In some embodiments, the GUI 220 can display a warning when the altitude selected or specified by the user is too low, too high, or determined to be inappropriate for the mission based on one or more other parameters. For example, a flight mode may specify that the asset data collector 130 fly over cables. The system 100 (e.g., via functionality of the automated data collection configurator 120) may determine that a particular altitude is too low for this mode because flying at that altitude may create a risk of the asset data collector 130 colliding with the cables. In this case, the GUI 220 may issue a warning to the user, or may recommend a more appropriate altitude (eg, an altitude determined to avoid the risk of collision).

[0033] In an embodiment, the GUI 220 may enable a user to specify a focus height 207, which may indicate the height at which the asset data collector 130 is to focus on the ground to collect asset data. For example, a focus height of 5 feet may indicate that the asset data collector 130 is to focus at a height of 5 feet above the ground when collecting asset data, in which case the asset data collector 130 may tilt one or more of its sensors (e.g., cameras) so that the sensors focus at the 5-foot focus height. It should be understood that the focus height may indicate the focal point of data collection, and not necessarily a single data collection point. Thus, the focus does not indicate a unique point of data collection, but rather the point at which the data sensor is focused. The field of view of the sensor may define the collection area, which may include the focal point at which the sensor is focused.

[0034] In embodiments, the GUI 220 allows the user to specify a video fly-through speed 208, which may indicate the speed at which the asset data collector 130 collects data. In embodiments, the video fly-through speed 208 may be used to configure the speed at which the asset data collector 130 collects asset data. A faster video fly-through speed 208 may result in a smaller amount of collected asset data. For example, when the collected asset data is a video, a faster video fly-through speed may result in a relatively shorter video because the asset data collector 130 may be moving quickly while collecting the video. Therefore, in this case, and depending on the video quality and frame rate of the collected video, the file size of the collected video may be smaller. On the other hand, a slower video fly-through speed 208 may result in a larger amount of collected asset data. For example, when the collected asset data is a video, a slower video fly-through speed may result in a relatively longer video because the asset data collector 130 may be moving slowly while collecting the video. Therefore, in this case, and depending on the video quality and frame rate of the collected video, the file size of the collected video may be larger.

[0035] In embodiments, the GUI 220 may allow a user to specify a collection quality 209, which may indicate the quality of the asset data collected by the asset data collector 130. For example, when the collected asset data includes video, the collection quality 209 may specify the quality of the video to be collected. Notably, the size of the collected data may depend on the collection quality 209. For example, a higher quality may result in a larger amount of collected data, as higher quality generally requires more information to be collected, while a lower quality may result in a relatively smaller amount of collected data. In some embodiments, the GUI 220 may automatically adjust the options displayed for selecting the collection quality 209 based on the type of sensor that the asset data collector 130 may be equipped with. For example, the asset data collector 130 may be equipped with a camera that can collect 4K video. In this case, the options available to the user when selecting the collection quality 209 may include qualities up to 4K.

[0036] In an embodiment, the GUI 220 may allow a user to specify high-level parameters 210. Although not shown, the high-level parameters may include the frame rate of the asset data to be collected (e.g., frames per second (fps)), the type of asset data to be collected (e.g., video, still images, infrared data, thermal data, audio data, etc.), the metadata sampling rate (e.g., the rate at which metadata associated with the asset data collector 130 is collected), the number of pattern repetitions (e.g., indicating the number of times the flight pattern is to be repeated in the mission plan), the stop condition (e.g., if this condition is found, the mission plan is terminated and the asset data collector may be returned to the starting / end point), the mission type (e.g., inspection mission, positioning mission, surveillance mission, security mission, maintenance mission, etc.), etc. In an embodiment, the GUI 220 may allow a user to specify the regulatory parameters 210, which may include airspace and Low Altitude Authorization and Notification Capability (LAANC) parameters.

[0037] In an embodiment, GUI 220 may present detailed information about the mission plan based on specified parameters. For example, altitude 202 may display the altitude at which asset data collector 130 will operate during execution of the mission plan, which may correspond to the altitude specified by flight altitude 206. Mission duration 203 may indicate the duration of the mission plan based on the parameters of the mission plan (e.g., flight path, flight speed, flight altitude, etc.). Depending on the parameters of the mission plan, video duration 204 may indicate the length or size of the collected asset data. For example, when the collected asset data includes video, video duration 204 may indicate the length of the collected video, which may depend on the mission duration, flight speed, etc. Battery indicator 205 may indicate the number of batteries required for asset data collector 130 to complete the mission plan, or the number of charges required for asset data collector 130 to complete the mission plan (e.g., the number of times the battery may need to be charged).

[0038] In an embodiment, user terminal 140 may be communicatively coupled to server 110 (e.g., via network 145), and one or more parameters of the mission plan may be sent to server 110 (e.g., to an automated data collection configurator) to configure the mission plan. In some embodiments, one or more parameters of the mission plan may be specified using terminal 140, may be stored on a removable storage device (not shown), and may then be uploaded from the removable storage device to server 110. In this manner, one or more parameters may be flexibly provided to server 110 to configure the mission plan.

[0039] In some embodiments, the user terminal 140 can be configured to present (e.g., display) the results output by the server 110, which may include the identification, location, and / or condition of the assets in the intermodal facility 225. In some embodiments, the results can be output to an external system, such as an asset management system.

[0040] Reference again Figure 1 , the asset data collector 130 can be configured to perform automated collection of asset data associated with assets in the intermodal facility based on a mission plan. For example, in an embodiment, the asset data collector 130 can include sensors (e.g., cameras, detectors, microphones, etc.) configured to collect sensor data associated with the assets. The sensor data may include static images, videos, infrared data, thermal data, etc. In an embodiment, the asset data collector 130 can be a vehicle on which sensors can be mounted. For example, the vehicle can be a ground vehicle (e.g., a car or truck) or an aircraft (e.g., a manned aircraft, an airborne or remotely operated aircraft, or a UAS (unmanned aerial vehicle)). The asset data collector 130 can be configured to avoid obstacles and / or traverse the intermodal storage facility at a low speed (e.g., a speed not exceeding ten miles per hour) or a speed that may exceed ten miles per hour.

[0041] It is worth noting that, throughout this disclosure, the discussion focuses on the use of cameras to collect image data (e.g., video), but it should be understood that this discussion is for illustrative purposes only and is not intended to be limiting in any way. The techniques disclosed herein are applicable to the collection of any sensor data that can be used to detect, identify, and inspect assets in an intermodal facility. It is also important to note that, while the discussion of this disclosure focuses on embodiments in which the asset data collector 130 comprises a UAS (e.g., a UAS with a camera mounted thereon), the techniques disclosed herein are also applicable to embodiments in which the asset data collector 130 comprises a manned aerial system or a ground vehicle. For example, in some embodiments, a ground vehicle may be equipped with a camera, and the camera may be configured to collect image data of assets in the intermodal facility as the ground vehicle travels through the intermodal facility according to a mission plan (e.g., following a "flight" plan, in which a ground route for the ground vehicle to follow is specified). Therefore, the discussion of UAS should not be construed as limiting in any way. As used herein, UAS may refer to an unmanned aerial vehicle, and may also include autonomous or ground-controlled aircraft, such as drones, multirotors, quadrotors, fixed-wing aircraft, and the like.

[0042] In an embodiment, the asset data collector 130 may be configured to receive a mission plan (e.g., a mission plan configured by the automated data collection configurator 120). For example, in an embodiment, the mission plan (which may include one or more parameters specified for the mission plan) may be uploaded to the asset data collector 130 via the network 145 (e.g., from the automated data collection configurator 120). In some embodiments, the mission plan may be uploaded to the asset data collector 130 manually, such as by storing the mission plan on a mobile storage device, connecting the mobile storage device to the asset data collector 130, and uploading the mission plan from the mobile storage device to the asset data collector 130.

[0043] The asset data collector 130 may be configured to execute the mission plan according to the configuration of the mission plan. For example, the asset data collector 130 may execute the mission plan by traversing the intermodal facility according to the configuration of the mission plan (e.g., flight mode, flight altitude, flight speed, focus altitude, acquisition quality, etc.) while collecting asset data (e.g., image data related to the assets). The asset data may include data related to assets in the intermodal facility, and may include assets such as inventory, storage containers, transport vehicles, rail cars, trailers, tracks, warehouses, buildings, security fences, storage spaces, roads, signs, etc. For example, Figure 3 As shown, intermodal facility 301 may include various assets of various types, such as rail cars 310 , tracks 315 , buildings 320 , containers 325 , security fences 330 , and other assets.

[0044] In an embodiment, the asset data collector 130 may be configured to periodically collect or record metadata related to the asset data collector 130. The metadata may include the location of the asset data collector 130 at the time the metadata is recorded, the orientation of one or more sensors of the asset data collector 130 (e.g., the orientation of a camera of the asset data collector 130 and / or the orientation of a gimbal on which the camera is mounted), the height of the asset data collector 130, the speed of the asset data collector 130, a timestamp, etc.

[0045] In embodiments, the asset data collector 130 may include onboard memory (not shown) configured to store collected asset data, which may include image data associated with assets in the intermodal facility and / or metadata associated with the asset data collector 130. In these embodiments, the collected asset data may be stored in the onboard memory for subsequent transfer to the server 110. For example, in some embodiments, the collected asset data may be transferred from the asset data collector 130 to a mobile storage device (not shown). The mobile storage device may then be connected to the server 110, and the collected asset data may be uploaded from the mobile storage device to the server 110. In some embodiments, the asset data collector 130 may be communicatively coupled to the server 110 (e.g., via a network 145), and the collected asset data may be transferred to the server 110 via the network 145. In some embodiments, the asset data collector 130 may be configured to transmit the collected asset data to the server 110 via real-time streaming (e.g., via the network 145 or other network not shown). In these embodiments, the asset data collector 130 may store, transmit, or upload the asset data to the server 110 in real time or near real time as the asset data is collected. The real-time streaming capability of the asset data collector 130 may be in addition to or in lieu of storing the collected image data in the onboard memory of the asset data collector 130 for subsequent transfer to the server 110.

[0046] In some embodiments, the asset data collector 130 can be configured to collaborate with other asset data collectors. In these embodiments, the system 100 can include multiple asset data collectors, and these asset data collectors can operate in various collaborative modes. In these embodiments, the mission plan of an asset data collector can be configured based on the mission plans configured for other asset data collectors. In these cases, the mission plan of each asset data collector will take into account the mission plans of these asset data collectors. For example, Figure 3 As shown, asset data collectors 130a-130d may operate in intermodal facility 301 to provide automated asset management functionality consistent with embodiments of the present disclosure. Figure 3In this example, asset data collectors 130a-130d can operate in a non-redundant mode, in which case each asset data collector can be configured with a separate mission plan to independently collect asset data. For example, asset data collector 130a can be configured with a mission plan that includes a maintenance inspection mission to inspect railcar 310 and track 315. In this case, asset data collector 130a can perform automated data collection related to railcar 310 and track 315, which can be processed by server 110 to perform automated analysis of the asset data. In this case, asset data collector 130a can perform automated data collection without the cooperation of asset data collectors 130b-130d. In this example, asset data collector 130b can be configured with a mission plan that includes a location mission to determine / verify the location of container 325. In this case, asset data collector 130b can perform automated data collection related to container 325, which can be processed by server 110 to perform automated analysis of the asset data. In this case, asset data collector 130b can perform automated data collection without the cooperation of asset data collectors 130a, 130c, or 130d. Furthermore, in this example, asset data collector 130d can configure a mission plan that includes a security mission to provide surveillance and perimeter security for intermodal facility 301. In this case, asset data collector 130d can perform automated data collection associated with fence 330 that may surround intermodal facility 301 and buildings 320 and 321 that may be adjacent to fence 330. The asset data collected by asset data collector 130d can be processed by server 110 to perform automated analysis of the asset data. In this case, asset data collector 130d can perform automated data collection without the cooperation of asset data collectors 130a-130c.

[0047] In some embodiments, the asset data collectors can operate in a redundant mode. In these embodiments, some asset data collectors can be used to review results associated with other asset data collectors. For example, asset data collectors 130b and 130c can be configured to check the mission plan of container 325. In the case of this redundant mode, asset data collectors 130b and 130c can be used to collect data associated with the same container (i.e., container 325). The results associated with the data collected by one of the asset data collectors 130b and 130c can be used to verify, confirm or otherwise validate the results associated with the asset data collected by the other of the asset data collectors 130b and 130c, as the results are expected to be similar. In some embodiments, the redundant mode can include designating some asset data collectors to be on standby to replace other asset data collectors in the event of a problem to ensure continuity of operation.

[0048] As described above, in some embodiments, different asset data collectors can be assigned different types of tasks. Task types may include maintenance tasks, inspection tasks, security tasks, etc. For example, a task plan can configure an asset data collector for a maintenance task. In this case, the asset data collector can collect asset data for analysis to determine whether the asset requires maintenance, and determine whether the maintenance performed on the asset is performed correctly, etc. In another example, a task plan can configure an asset data collector for a security task. In this case, the asset data collector can collect asset data for analysis to determine the security posture of the asset, monitor to prevent security issues (e.g., illegal intrusion, theft, unauthorized entry, etc.), ensure that the asset is not stolen, mishandled, or otherwise interfered with, etc. In another example, a task plan can configure an asset data collector for an inspection task. In this case, the asset data collector can collect asset data that needs to be analyzed in order to inspect the asset to determine the condition or location of one or more assets.

[0049] Reference again Figure 1 The server 110, the asset data locator 130, and the user terminal 140 may be communicatively coupled via a network 145. The network 145 may include a wired network, a wireless communication network, a cellular network, a cable transmission system, a local area network (LAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, a public switched telephone network (PSTN), and the like.

[0050] Server 110 may be configured to facilitate configuration of automatic data collection and perform automatic analysis of collected asset data according to embodiments of the present disclosure. In embodiments, server 110's functionality for facilitating configuration of automatic data collection may include functionality for generating a task plan configuration for execution by an asset data collector to implement automatic data collection. In embodiments, server 110's functionality for performing automatic analysis of asset data may include functionality for detecting assets in the asset data, identifying the detected assets based on the asset data, inspecting the assets based on the asset data, and generating output content including automated analysis results, according to embodiments of the present disclosure.

[0051] The functionality of the server 110 may be provided by various components of the server 110 in cooperation with each other, as described in detail below. Figure 1 A single server 110 is shown, but it will be appreciated that the server 110 and its various functional blocks may be implemented as a single device, or may be distributed across multiple devices with their own processing resources, the overall functionality of which may be configured to perform operations in accordance with the present disclosure. Figure 1The components of server 110 are shown as single and independent blocks, but the various components of server 110 can be single components (e.g., single applications, server modules, etc.), can be functional components of the same component, or the functionality can be distributed across multiple devices / components. In such embodiments, the functionality of each respective component can be aggregated from the functionality of multiple modules located in a single or multiple devices. In addition, specific functionality described with respect to a particular component of server 110 may actually be part of a different component of server 110, and thus, descriptions of specific functionality with respect to a particular component of server 110 are for illustrative purposes only and are not intended to be limiting in any way.

[0052] like Figure 1 As shown, the server 110 includes a processor 111, a memory 112, a database 124, an automated data collection configurator 120, and an automated asset data analyzer 121. The processor 111 may include a processor, a microprocessor, a controller, a microcontroller, multiple microprocessors, an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), or any combination thereof, and may be configured to execute instructions to perform operations according to the disclosure herein. In some embodiments, the implementation of the processor 111 may include code segments (e.g., software, firmware, and / or hardware logic) that can be executed in hardware (e.g., a processor) to perform the tasks and functions described herein. In other embodiments, the processor 111 may be implemented as a combination of hardware and software. The processor 111 may be communicatively coupled to the memory 112.

[0053] The memory 112 may include one or more semiconductor memory devices, read-only memory (ROM) devices, random access memory (RAM) devices, one or more hard disk drives (HDDs), flash memory devices, solid-state drives (SSDs), erasable ROMs (EROMs), compact disk ROMs (CD-ROMs), optical disks, other devices configured to store data in a persistent or non-persistent state, network storage, cloud storage, local storage, or a combination of different memory devices. The memory 112 may include a processor-readable medium configured to store one or more sets of instructions (e.g., software, firmware, etc.) that, when executed by a processor (e.g., one or more processors of the processor 111), will perform the tasks and functions described herein.

[0054] The memory 112 may also be configured to facilitate storage operations. For example, the memory 112 may include a database 124 for storing various information related to the operation of the system 100. For example, the database 124 may store ML models, mathematical models, rule models, and / or other models that may be used by components of the server 110 to analyze and process asset data to detect assets in the asset data, identify detected assets based on the asset data, and inspect detected assets based on the asset data according to embodiments of the present disclosure. In embodiments, the database 124 may store asset data collected by the asset data collector 130. For example, the asset data collected by the asset data collector 130 may be ingested by the server 110 and stored in the database 124 for further analysis (e.g., by the automated asset data analyzer 121). In embodiments, the database 124 may store configuration data. For example, the database 124 may store one or more of the aforementioned parameters and / or a mission plan configuration generated (e.g., by the automated data collection configurator 120) so that the asset data collector 130 performs automated data collection based on one or more parameters (e.g., including flight route, flight mode, flight range, flight altitude, flight speed, video acquisition speed, focus altitude, acquisition video quality, image data type to be collected, etc.). The database 124 is shown as being integrated into the memory 112, but in some embodiments, the database 124 may be provided as a separate storage module or as a cloud-based storage module. Additionally or alternatively, the database 124 may be a single database or a distributed database implemented in multiple database modules.

[0055] The automated data collection configurator 120 can be configured to facilitate automated data collection, including functionality for configuring a mission plan that can be executed by the asset data collector 130 based on one or more parameters, and / or functionality for instantiating (e.g., launching or causing the launching of) at least one asset data collector for executing a mission. For example, one or more parameters can be specified (e.g., by a user through a GUI or in a file uploaded to the server 110), and the automated data collection configurator 120 can compile, construct, and / or process the one or more parameters to generate a mission plan to be executed by the asset data collector 130. In embodiments, the mission plan includes configurations that the asset data collector 130 can use to execute the mission plan, as described in detail above.

[0056] In embodiments, the mission plan of the asset data collector 130 may include operating parameters that the asset data collector 130 may follow during execution of the mission plan. In some embodiments, the automated data collection configurator 120 may generate operating parameters based on the specified parameters of the mission. For example, the mission plan may include flight mode, speed, altitude, etc. In embodiments, the operating parameters generated by the automated data collection configurator 120 may be based on the type of mission for which the mission plan was generated. For example, different types of missions may generate different operating parameters. Figure 2 As shown, the automated data collection configurator 120 may generate a pattern 241 and a first altitude (possibly a relatively low altitude) for an inspection mission. In this case, pattern 241 and the first altitude may be particularly suitable for an inspection mission because the pattern is typically a zigzag pattern and is relatively low in altitude, allowing the asset data collector to collect image data from both the front and back of the asset being inspected (e.g., a container). Alternatively, the automated data collection configurator 120 may generate a pattern 242 and a second altitude for a security mission, which may be higher than the first altitude. In this case, pattern 242 and the higher second altitude may be particularly suitable for a security mission because the pattern is typically a linear pattern and is relatively high in altitude, sufficient to collect data useful in determining the security status of the asset.

[0057] In some embodiments, the automated data collection configurator 120 may determine that a mission may require multiple asset data collectors. In these embodiments, the automated data collection configurator 120 may generate multiple mission plans, each of which may be configured with a specific asset data collector, or may generate a single mission plan that is common to multiple asset data collectors. In each case, the automated data collection configurator 120 may generate one or more mission plans based on one or more parameters associated with the mission.

[0058] In some embodiments, the automated data collection configurator 120 collaborates with the automated asset data analyzer 121 and can be configured to utilize the real-time streaming capabilities of the asset data collector 130 to dynamically determine and / or modify mission plans based on operational conditions. For example, asset data can be collected by the asset data collector 130 and transmitted to the server 110 via real-time streaming in near real time. The automated asset data analyzer 121 can analyze the collected asset data and determine operational conditions (e.g., asset conditions, environmental conditions, safety conditions, maintenance conditions, etc.). The analysis results of the automated asset data analyzer 121 can be provided to the automated data collection configurator 120, which can determine, based on the operational conditions, that the mission plan currently configured by the asset data collector 130 is no longer suitable for the mission. In this case, the automated data collection configurator 120 can determine modifications to the mission plan based on the operational conditions, thereby optimizing the mission plan. The modified mission plan can be transmitted to the asset data collector 130, which can then continue executing the mission according to the modified mission plan. For example, during the execution of a mission, the automated asset data analyzer 121 may determine that the data collected from a particular area may be incomplete and that there may be some missing expected data. In this case, the data collection configurator 120 may generate a modified mission plan that includes issuing instructions to the asset data collector 130, or may directly instruct the asset data collector 130 to fly over the area where asset data is missing to ensure that asset data for the area is collected. In another example, the automated asset data analyzer 121 may determine anomalies in the collected asset data set. The automated data collection configurator 120 may determine whether further inspection is necessary, and may instruct the asset data collector 130 to fly over the area where the anomaly occurs to further collect data, and in some cases, the modified mission plan may include improving the acquisition quality, reducing the video acquisition flight speed to ensure complete collection of asset data, increasing the acquisition frame rate, changing the altitude, etc.

[0059] In some embodiments, the asset data collector 130 can be configured to modify the mission plan. For example, in some embodiments, the flight pattern of the mission plan can be modified during the execution of the mission plan. The asset data collector 130 can be configured with obstacle avoidance functionality. In this case, during the execution of the mission plan, the asset data collector 130 may determine that following the flight pattern may result in a collision with an object. In this case, the asset data collector 130 may modify the flight pattern to avoid the collision or may notify the automated data collection configurator 120 of the potential collision, in which case the automated data collection configurator 120 may generate a modified mission plan.

[0060] The automated asset data analyzer 121 can be configured to perform automated analysis on the collected asset data to detect, identify, and / or inspect assets appearing or represented in the asset data collected according to the mission plan configured by the automated data collection configurator 120. In embodiments, the automated data analysis functionality for detecting, identifying, and / or inspecting assets appearing or represented in the collected asset data may include applying an ML module and rule set to the collected asset data. Automated data analysis techniques for detecting, identifying, and / or inspecting objects appearing in acquired image data are disclosed in co-pending and commonly owned U.S. patent application Ser. No. 17 / 822,999 (filed Aug. 29, 2022, and entitled “UAV-Based Automated Yard Inspection”), the disclosure of which is incorporated herein by reference in its entirety for all purposes. In embodiments, the automated asset data analyzer 121 may include an asset detector / identifier 122 and an asset inspector 123, which may collaborate to provide the functionality of the automated asset data analyzer 121 for performing automated analysis on the collected asset data.

[0061] In an embodiment, the asset detector / identifier 122 may be configured to analyze the collected asset data to detect and identify assets that appear or are represented in the asset data. For example (and with additional reference to Figure 3 ), the collected asset data may include image data related to assets in the intermodal facility 301. The image data may include a video or a series of pictures of the assets in the intermodal facility 301. For example, the collected asset data may include a video of the fence 330 and buildings 320 and 321 collected by the asset data collector 130d according to the mission plan. In another example, the collected asset data may include a video of the container 325 collected by the asset data collectors 130b and / or 130c according to the mission plan, or may include a video of the train car 310 and / or track 315 collected by the asset data collector 130a according to the mission plan. In these examples, the collected asset data may include video data of the assets in the intermodal facility 301.

[0062] In embodiments, asset detector / identifier 122 can be configured to analyze the collected asset data frame by frame using an ML / computer vision (CV) model. The ML / CV model can be trained to detect and identify assets present in the asset data (e.g., assets appearing in image data). In these embodiments, asset detector / identifier 122 can apply the ML / CV model to the video data frame by frame to detect and identify assets appearing in different frames. For example, the ML / CV model can detect assets appearing in a frame and can determine a set of frames in which at least one asset appears. In this manner, asset detector / identifier 122 can detect assets in the asset data.

[0063] In an embodiment, the asset detector / identifier 122 may apply an ML / CV model to a set of frames in which at least one asset appears to identify the asset appearing in the frames. For example, the ML / CV model may be applied to each frame to identify the asset appearing in each frame. In an embodiment, each asset appearing in each frame is assigned an identifier, allowing the system 100 to track a specific asset frame by frame.

[0064] In embodiments, the asset inspector 123 may be configured to inspect each asset detected in each frame of the frame set. In embodiments, the asset inspection may include determining the condition of the asset. The condition of the asset may include the physical condition of the asset, and may also include the asset's location, any damage to the asset, whether the asset is open or closed, whether the asset is in the correct location, whether the asset is vacant, whether the asset's appearance is as expected, whether the asset is complete or partially missing, whether the asset has been vandalized, whether the asset contains unexpected attachments, and / or any other condition that may be determined to be unexpected or may require further inspection.

[0065] In embodiments, the condition of an asset may be relevant in the context of the type of task associated with collecting asset data. For example, a security task may cause a fence, such as fence 330, to appear in the asset data. In this example, asset inspector 123 may determine that the condition of fence 330 is damaged, which may necessitate dispatching personnel for further investigation or remedial action. On the other hand, asset inspector 123 may determine that the condition of fence 330 is normal, and no further action may be taken. In another example, an inspection task to determine or verify the location of an asset (e.g., container 325) may cause container 325 to appear in the asset data. In this example, the condition of container 325 may include the location of each container 325. As described elsewhere herein, techniques for determining the location of an object based on collected image data are disclosed in co-pending and commonly owned U.S. patent application Ser. No. 17 / 822,999 (filed Aug. 29, 2022, and entitled "UAV-Based Automated Yard Inspection"), the disclosure of which is incorporated herein by reference in its entirety for all purposes. In yet another example, a maintenance task to determine whether an asset (e.g., track 315) requires maintenance may cause track 315 to appear in the asset data. In this example, the condition of track 315 determined by asset inspector 123 may include whether track 315 requires maintenance, as determined by the physical condition of track 315 (e.g., whether there is any damage or the potential for damage).

[0066] In some embodiments, the analysis results of the automated asset data analyzer 121 can be provided as output content, for example, to the user terminal 140 and / or an external system (such as an asset management system).

[0067] Figure 4 A high-level operational flow chart 400 of a system configured to provide automated asset management according to an embodiment of the present disclosure is shown. For example, Figure 4 According to the embodiments herein, Figure 1 In an embodiment, the operations of the method 400 may be stored as instructions and executed by one or more processors to cause the one or more processors to perform the operations of the method 400.

[0068] At block 402, a mission plan is configured based on one or more parameters to collect asset data associated with one or more assets in the intermodal facility. In some embodiments, the collection of asset data associated with one or more assets in the intermodal facility may be performed by an asset data collector (e.g., asset data collector 130, located at Figure 1 In embodiments, functionality of an automated data collection configurator (e.g., automated data collection configurator 120) may be used to configure a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on one or more parameters. In embodiments, the automated data collection configurator may perform operations to configure a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on one or more parameters, as described above with reference to the automated data collection configurator 120 and as described above with reference to the automated data collection configurator 120. Figure 1-3 shown.

[0069] At block 404, an asset data collector (e.g., the asset data collector 130) may be instantiated to automatically collect asset data associated with one or more assets. In embodiments, instantiating the asset data collector causes the asset data collector to be initiated according to a mission plan to automatically collect asset data associated with one or more assets in the intermodal facility. In embodiments, functionality of an automated data collection configurator (e.g., the automated data collection configurator 120) may be used to instantiate the asset data collector to automatically collect asset data associated with one or more assets. In embodiments, the automated data collection configurator may perform operations to instantiate the asset data collector to automatically collect asset data associated with one or more assets, the specific operations and functionality being as described above with reference to the automated data collection configurator 120 and as described above. Figure 1-3 shown.

[0070] At block 406, asset data associated with one or more assets in the intermodal facility is automatically collected according to the mission plan. In embodiments, functionality of an asset data collector (e.g., asset data collector 130) may be used to automatically collect asset data associated with one or more assets in the intermodal facility according to the mission plan. In embodiments, the asset data collector may perform operations to automatically collect asset data associated with one or more assets in the intermodal facility according to the mission plan, with the specific operations and functionality described above with reference to the asset data collector 130 and as described above. Figure 1-3 shown.

[0071] At block 408, one or more assets are detected in the asset data set collected by the asset data collector. In embodiments, detecting the one or more assets in the data set includes identifying data in which the one or more assets are present. In embodiments, functionality of an asset detector / identifier (e.g., asset detector / identifier 122) may be used to detect the one or more assets in the asset data set collected by the asset data collector. In embodiments, the asset detector / identifier may perform operations to detect the one or more assets in the asset data set collected by the asset data collector, the specific operations and functionality of which are as described above with reference to the asset detector / identifier 122 and as described above with reference to the asset detector / identifier 122. Figure 1-3 shown.

[0072] At block 410, one or more assets detected in the data set are identified. In embodiments, functionality of an asset detector / identifier (e.g., asset detector / identifier 122) may be used to identify one or more assets detected in the data set. In embodiments, the asset detector / identifier may perform operations to detect one or more assets detected in the data set, with the specific operations and functionality described above with reference to asset detector / identifier 122 and as described above. Figure 1-3 shown.

[0073] At block 412, the one or more assets are inspected based on the dataset of the one or more assets detected to determine a condition of the one or more assets. In embodiments, functionality of an asset inspector (e.g., asset inspector 123) may be used to inspect the one or more assets based on the dataset of the one or more assets detected to determine a condition of the one or more assets. In embodiments, the asset inspector may perform operations to inspect the one or more assets based on the dataset of the one or more assets detected to determine a condition of the one or more assets, the specific operations and functionality of which are as described above with reference to asset inspector 123 and as described above with reference to asset inspector 123. Figure 1-3 shown.

[0074] At block 414, the status of the one or more assets is output. In embodiments, functionality of an automated asset data analyzer (e.g., automated asset data analyzer 121) may be used to output the status of the one or more assets. In embodiments, the automated asset data analyzer may perform operations to output the status of the one or more assets, with the specific operations and functionality described above with reference to the asset data analyzer 121 and as described above. Figure 1-3 shown.

[0075] Those skilled in the art will readily appreciate that the above advantages and purposes would not be possible without the specific combination of computer hardware and other structural components and mechanisms assembled in the system of the present invention and described herein. In addition, the algorithms, methods, and processes disclosed herein improve and convert any general-purpose computer or processor disclosed in this specification and the accompanying drawings into a special-purpose computer that is programmed to execute the disclosed algorithms, methods, and processes to achieve the above functions, advantages, and goals. It should also be understood that for those skilled in the art, there are a variety of programming tools available for generating and implementing the functions and operations described above. In addition, the selection of a particular programming tool may be determined by the specific goals and constraints of implementing these concepts, which are selected to implement the concepts set forth herein and in the appended claims.

[0076] The description in this patent document should not be construed as implying that any particular element, step, or function is an essential or critical element that must be included within the scope of the claims. Furthermore, unless the exact words "means for" or "step for" are expressly used in a particular claim and followed by a participle phrase identifying the function, no claim is intended to invoke 35 U.S.C. §112(f) with respect to any additional claim or claim element. Terms used in the claims (such as (but not limited to) "mechanism," "module," "device," "unit," "component," "element," "member," "means," "machine," "system," "processor," "processing device," or "controller") should be understood and intended to refer to structures known to one skilled in the relevant art, and to be further modified or enhanced by the features of the claims themselves, and are not intended to invoke 35 U.S.C. §112(f). Even under the broadest reasonable interpretation, in the absence of the specific language described above, the claims are not intended to invoke 35 U.S.C. §112(f) in accordance with this paragraph of the specification.

[0077] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. For example, each new structure described herein may be modified to accommodate specific local changes or requirements while retaining their basic configuration or structural relationship to each other, or while performing the same or similar functions described herein. Therefore, the present embodiments should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the invention should be determined by the appended claims rather than by the foregoing description. Therefore, all changes that come within the meaning and equivalent range of the claims should be included in the claims. In addition, the various elements in the claims are not well-known, conventional or traditional. Instead, the claims are directed to the unconventional inventive concepts described in the specification.

[0078] It will also be understood by those skilled in the art that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above from a functional perspective. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. Those skilled in the art can implement the described functions in different ways for each specific application, but such implementation decisions should not be interpreted as causing deviations from the scope of the present disclosure. Those skilled in the art will also readily recognize that the components, methods, or interactive order or combination described herein are merely examples, and the components, methods, or interactives of the various embodiments of the present disclosure can be combined or performed in a manner other than that shown and described herein.

[0079] Figure 1-4 The functional blocks and modules in the can include processors, electronic devices, hardware devices, electronic components, logic circuits, memories, software codes, firmware codes, etc., or any combination thereof. Consistent with the foregoing, the various illustrative logic blocks, modules, and circuits described in connection with the disclosure herein can be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but, alternatively, the processor can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors with a DSP core, or any other such configuration.

[0080] The steps of the method or algorithm described in conjunction with the content disclosed herein can be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC may reside in a user terminal, a base station, a sensor or any other communication device. Alternatively, the processor and storage medium can reside in the user terminal as discrete components.

[0081] In one or more exemplary designs, the functions described can be implemented using hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored as one or more instructions or codes in a computer-readable medium or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. A computer-readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, but not limited to, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store the required program code means in the form of instructions or data structures and can be accessed by a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. In addition, a connection can also be appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL), the coaxial cable, fiber optic cable, twisted pair, or DSL are all included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser video disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0082] Those skilled in the art will readily appreciate that the above advantages and purposes would not be possible without the specific combination of computer hardware and other structural components and mechanisms assembled in the system of the present invention and described herein. In addition, the algorithms, methods, and processes disclosed herein improve and convert any general-purpose computer or processor disclosed in this specification and the accompanying drawings into a special-purpose computer that is programmed to execute the disclosed algorithms, methods, and processes to achieve the above functions, advantages, and goals. It should also be understood that for those skilled in the art, there are a variety of programming tools that can be used to generate and implement the functions and operations described above. In addition, the specific choice of programming tool may depend on the specific goals and constraints of the implementation scheme selected to implement the concepts set forth herein and in the appended claims.

[0083] The description in this patent document should not be construed as implying that any particular element, step, or function is an essential or critical element that must be included within the scope of the claims. Furthermore, unless the exact words "means for" or "step for" are expressly used in a particular claim and followed by a participle phrase identifying the function, no claim is intended to invoke 35 U.S.C. §112(f) with respect to any additional claim or claim element. Terms used in the claims (such as (but not limited to) "mechanism," "module," "device," "unit," "component," "element," "member," "means," "machine," "system," "processor," "processing device," or "controller") should be understood and intended to refer to structures known to one skilled in the relevant art, and to be further modified or enhanced by the features of the claims themselves, and are not intended to invoke 35 U.S.C. §112(f). Even under the broadest reasonable interpretation, in the absence of the specific language described above, the claims are not intended to invoke 35 U.S.C. §112(f) in accordance with this paragraph of the specification.

[0084] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. For example, each of the new structures described herein may be modified to accommodate specific local changes or requirements while retaining their basic configuration or structural relationship to each other, or while performing the same or similar functions described herein. Therefore, the present embodiments should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the invention should be determined by the appended claims, not by the foregoing description. Therefore, all changes within the meaning and range of equivalence of the claims should be included in the claims. In addition, the various elements in the claims are not well-known, conventional or traditional. Instead, the claims are directed to the unconventional inventive concepts described in the specification.

Claims

1. Automated asset management system, including: an asset data collector configured to automatically collect asset data associated with one or more assets in the intermodal facility in accordance with the mission plan; Automated data collection configurator, configured to: configuring the mission plan based on one or more parameters; and instantiating an asset data collector, wherein instantiating the asset data collector causes the asset data collector to be initiated according to the task plan to automatically collect asset data associated with one or more assets in the intermodal facility; and Automated asset data analyzer, configured to: Detecting one or more assets in an asset data set collected by an asset data collector, wherein detecting the one or more assets in the data set includes identifying data in which the one or more assets appear; identifying one or more assets detected in the data set; Inspecting the one or more assets to determine a condition of the one or more assets based on a data set in which the one or more assets are detected; and Outputs the status of one or more assets.

2. The system of claim 1, wherein the one or more parameters include: The task type of the mission plan; A flight route defined by one or more flight modes, an operating area, and a set of waypoints; The title of the mission plan; Flights run by asset data collectors while automatically collecting asset data; A focus height, which indicates the height at which the asset data collector is to focus on the ground when automatically collecting asset data; Video collection flight speed, which indicates the speed at which the asset data collector operates when automatically collecting asset data; The collection quality of asset data collected by the asset data collector; one or more advanced parameters; and One or more regulatory parameters.

3. The system of claim 2, wherein the task type comprises one or more of the following: Maintenance tasks; Inspection tasks; and Security mission.

4. The system of claim 2, wherein the flight paths of the mission plans are based on mission types, and wherein a first flight path for a first mission of a first type is different than a second flight path for a second mission of a second type.

5. The system of claim 1 , wherein to detect one or more assets in the data set, identify one or more assets, and determine a condition of one or more assets, configuring the automated asset data analyzer comprises configuring the automated asset data analyzer to: A machine learning (ML) model is applied to the asset data and configured to detect one or more assets in the data set and identify the one or more assets detected in the data set.

6. The system of claim 1 , wherein the condition of the one or more assets comprises one or more of the following: the physical condition of one or more assets; the location of one or more assets; damage to one or more assets; One or more assets are open or closed; the location of one or more assets; One or more of the properties is vacant or occupied; the appearance of one or more assets relative to expectations; Asset conditions that are determined to be unexpected or require further inspection.

7. The system of claim 1, wherein the asset data collector comprises an unmanned aerial system (UAS).

8. The system of claim 7, wherein the asset data includes image data associated with the one or more assets, and wherein the UAS includes a camera for acquiring the image data associated with the one or more assets.

9. The system of claim 1 , wherein the asset data collector comprises a plurality of data collection vehicles, and wherein the plurality of data collection vehicles automatically collect asset data associated with the one or more assets in one of the following ways: Redundancy mode; and Non-redundant mode.

10. The system according to claim 9, wherein: At least one of the multiple data collection carriers is in a standby state so as to replace another data collection carrier in the multiple data collection carriers to automatically collect asset data when the other data collection carrier is unable to continue to automatically collect asset data.

11. An automated asset management method, comprising: configuring a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on the one or more parameters; instantiating an asset data collector to automatically collect asset data associated with one or more assets, wherein instantiating the asset data collector causes the asset data collector to be initiated according to the task plan to automatically collect asset data associated with one or more assets in the intermodal facility; automatically collecting, by an asset data collector, asset data associated with one or more assets in an intermodal facility in accordance with a mission plan; detecting one or more assets in the asset data set collected by the asset data collector, wherein detecting the one or more assets in the data set includes data identifying the presence of the one or more assets; identifying one or more assets detected in the data set; Inspecting the one or more assets to determine a condition of the one or more assets based on a data set in which the one or more assets are detected; and Outputs the status of one or more assets.

12. The method according to claim 11, wherein: The one or more parameters include: The task type of the mission plan; A flight route defined by one or more flight modes, an operating area, and a set of waypoints; The title of the mission plan; Flights run by asset data collectors while automatically collecting asset data; A focus height, which indicates the height at which the asset data collector is to focus on the ground when automatically collecting asset data; Video collection flight speed, which indicates the speed at which the asset data collector operates when automatically collecting asset data; The collection quality of asset data collected by the asset data collector; one or more advanced parameters; and One or more regulatory parameters.

13. The method according to claim 12, wherein the task type comprises one or more of the following: Maintenance tasks; Inspection tasks; and Security mission.

14. The method of claim 12, wherein flight paths of mission plans are based on mission types, and wherein a first flight path for a first mission of a first type is different than a second flight path for a second mission of a second type.

15. The method according to claim 11, wherein: Detecting one or more assets in a data set and identifying one or more assets include: A machine learning (ML) model is applied to the asset data and configured to detect one or more assets in the data set and identify the one or more assets detected in the data set.

16. The system according to claim 11, wherein: The condition of one or more assets includes one or more of the following: the physical condition of one or more assets; the location of one or more assets; damage to one or more assets; One or more assets are open or closed; the location of one or more assets; One or more of the properties is vacant or occupied; the appearance of one or more assets relative to expectations; Asset conditions that are determined to be unexpected or require further inspection.

17. The system of claim 11, wherein the asset data collector comprises an unmanned aerial system (UAS).

18. The system of claim 7, wherein the asset data includes image data associated with the one or more assets, and wherein the UAS includes a camera for acquiring the image data associated with the one or more assets.

19. The system of claim 1, wherein the asset data collector comprises a plurality of data collection vehicles, and wherein the plurality of data collection vehicles automatically collect asset data associated with the one or more assets in one of the following ways: Redundancy mode; and Non-redundant mode.

20. A computer-based automated asset management tool, the computer-based tool comprising a non-transitory computer-readable medium storing computer code which, when executed by a processor, causes a computing device to perform operations comprising: configuring a mission plan for collecting asset data associated with one or more assets in an intermodal facility based on the one or more parameters; Instantiating an asset data collector to automatically collect asset data associated with one or more assets, wherein: Instantiating the asset data collector causes the asset data collector to be initiated according to the task schedule to automatically collect asset data associated with one or more assets in the intermodal facility; automatically collecting, by an asset data collector, asset data associated with one or more assets in an intermodal facility in accordance with a mission plan; detecting one or more assets in the asset data set collected by the asset data collector, wherein detecting the one or more assets in the data set includes data identifying the presence of the one or more assets; identifying one or more assets detected in the data set; Inspecting the one or more assets to determine a condition of the one or more assets based on a data set in which the one or more assets are detected; and Outputs the status of one or more assets.

Images