Multimodal inventory visualization system and method

Abstract

According to some embodiments, a method for visualizing inventory of a multimodal container yard includes accessing a unit inventory stored in an inventory database. The unit inventory includes information about a plurality of transport containers physically located in the multimodal container yard. The method further includes providing one or more instructions to display, on an electronic display of a user computing system, a graphical user interface including a three-dimensional representation of the multimodal container yard. The method further includes updating, in real-time, the three-dimensional representation of the multimodal container yard based on the information about the plurality of transport containers in the unit inventory. The method further includes providing one or more instructions to display, in the graphical user interface, metadata about a particular transport container in response to a user selection of a particular visual representation of the particular transport container.

Description

[0001] Cross-reference to related applications This application is a continuation-in-part of U.S. Patent Application No. 18 / 435,667, filed on February 7, 2024, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0002] This disclosure generally relates to intermodal logistics, and more specifically, to intermodal inventory visualization systems and methods. Background Technology

[0003] A multimodal container yard is a facility that transfers shipping containers between trucks and trains. Most multimodal container yards include large storage areas for storing shipping containers. For example, trucks can enter a multimodal container yard and be instructed to park shipping containers in designated parking areas. However, for various reasons, truck drivers may not park the shipping containers in the requested locations. Therefore, multimodal container yards can utilize "yard checks" to ensure that shipping containers can be located when needed for loading onto trains. Yard checks are typically performed by workers who visually observe units within the yard (e.g., trailers, containers, etc.) and their locations (e.g., lot / row / spot) and manually enter the observed information into an inventory database. However, manual yard checks are time-consuming, monotonous, and prone to frequent errors. Summary of the Invention

[0004] This disclosure achieves technical advantages as a system, method, and computer-readable storage medium that provides functionality for imaging, identifying, and visualizing an inventory of transport containers, such as those in a rail yard. This disclosure provides a system that can be integrated into practical applications, having meaningful defining features that may include: accessing a unit inventory stored in an inventory database, the unit inventory including information about multiple transport containers physically located in a multimodal container yard; and providing one or more instructions to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard. Other meaningful defining features of the system integrated into practical applications include: updating the three-dimensional representation of the multimodal container yard based on information about the multiple transport containers in the unit inventory; and providing one or more instructions to display metadata about a particular transport container in the graphical user interface in response to a user's selection of a particular visualization representation of a particular transport container.

[0005] This disclosure addresses the technical problem of a lack of technical functionality for railway yard verification in current systems by providing a method and system for automatically determining the identification and location of transport containers within a railway yard and then providing a real-time three-dimensional graphical representation of the railway yard and its inventory. The technical solution provided herein, which is lacking in conventional systems, goes beyond simply applying manual processes to a computerized environment. It includes the functionality to supplement current manual methods for performing yard verification and enhances current inventory systems by providing a mechanism for automatically determining the identification and location of transport containers within a railway yard and then providing a real-time three-dimensional graphical representation of the railway yard and its inventory. In doing so, this disclosure goes far beyond simply applying manual processes to a computer.

[0006] Unlike existing solutions that may require manual traversal of railway yards and manual input of the identification and location of transport containers parked within them, embodiments of this disclosure provide systems and methods for automatically determining the identification and location of transport containers within railway yards (e.g., using aircraft or ground vehicles). By providing automated determination and verification of transport container inventory and location, facilities such as railway yards can be monitored more efficiently and effectively. This can improve the efficiency of the facility and reduce or eliminate the need for manual yard checks. This can reduce the resource usage (e.g., fuel and electricity) that would otherwise be required for manual yard checks. Furthermore, by providing a system that transmits data only in response to certain events (e.g., the detection of transport containers in an image), network bandwidth and computer resources (e.g., computer memory and processing power) can be optimized or reduced relative to existing systems. Further, by providing a real-time three-dimensional graphical representation of the multimodal container yard and its inventory (e.g., transport containers), personnel can gain a more accurate understanding of the inventory within the yard, thereby improving the efficiency of the facility. Other technical advantages will be readily apparent to those skilled in the art from the following figures, description, and claims. Furthermore, while specific advantages have been listed above, multiple implementation schemes may include all, some, or none of the listed advantages.

[0007] Therefore, this disclosure discloses concepts inextricably linked to computer technology, enabling the provision of technical benefits for realizing functions for automating the management and visualization of multimodal transport inventory, such as that in railway yards. The systems and techniques of the various embodiments provide improved systems by providing the ability to perform functions currently performed manually, as well as functions currently impossible.

[0008] One object of this disclosure is to provide a system for automatically managing and visualizing an inventory of physical objects such as shipping containers in a rail yard. Another object of this disclosure is to provide a method for automatically managing and visualizing an inventory of physical objects such as shipping containers in a rail yard. These and other objects are provided by this disclosure, which includes at least the following embodiments.

[0009] In one embodiment, a system includes an inventory database storing unit inventory, one or more memory units, and one or more computer processors communicatively coupled to the inventory database and the one or more memory units. The unit inventory includes information about a plurality of transport containers physically located in a multimodal container yard. The one or more computer processors are configured to access the unit inventory stored in the inventory database and provide one or more instructions to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard. The one or more computer processors are further configured to update the three-dimensional representation of the multimodal container yard in real time based on information about the plurality of transport containers in the unit inventory. Updating the three-dimensional representation of the multimodal container yard includes providing a visual representation in the three-dimensional representation of the multimodal container yard of each transport container physically located in the multimodal container yard, wherein the visual representation of each particular transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the particular transport container in the multimodal container yard. Updating the three-dimensional representation of the multimodal container yard further includes providing a visual representation of the current location of vehicles traveling within the multimodal container yard. The one or more computer processors are also configured to provide one or more instructions to display metadata about a particular transport container in the graphical user interface in response to a user's selection of a specific visual representation for that particular transport container.

[0010] In another embodiment, a method for visualizing inventory in a multimodal container yard includes: accessing unit inventory stored in an inventory database, and providing one or more instructions to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard. The method further includes: updating the three-dimensional representation of the multimodal container yard based on information in the unit inventory regarding the plurality of transport containers. Updating the three-dimensional representation of the multimodal container yard includes: providing a visual representation in the three-dimensional representation of the multimodal container yard of each transport container physically located within the multimodal container yard, wherein the visual representation of each particular transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the particular transport container in the multimodal container yard. Updating the three-dimensional representation of the multimodal container yard further includes: providing a visual representation in the three-dimensional representation of the multimodal container yard of the current location of vehicles traveling within the multimodal container yard. The method further includes providing one or more instructions to display metadata about the specific shipping container in the graphical user interface in response to a user's selection of a specific visual representation of the specific shipping container.

[0011] In another embodiment, one or more computer-readable non-transitory storage media contain instructions that, when executed by a processor, cause the processor to perform operations including: accessing a unit inventory stored in an inventory database; and providing one or more instructions to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard. The instructions further include: updating the three-dimensional representation of the multimodal container yard based on information in the unit inventory regarding the plurality of transport containers. Updating the three-dimensional representation of the multimodal container yard includes: providing a visual representation in the three-dimensional representation of the multimodal container yard of each transport container physically located within the multimodal container yard, wherein the visual representation of each particular transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the particular transport container in the multimodal container yard. Updating the three-dimensional representation of the multimodal container yard further includes providing a visual representation of the current location of vehicles traveling within the multimodal container yard. The instructions also include providing one or more instructions to display metadata about a specific transport container in the graphical user interface in response to a user's selection of a specific visual representation for that particular transport container.

[0012] The foregoing has provided a fairly broad overview of the features and technical advantages of this disclosure in order to facilitate a better understanding of the subsequent detailed description of this disclosure. Additional features and advantages of this disclosure that constitute the subject matter of the claims will be described below. Those skilled in the art will understand that the disclosed concepts and specific embodiments can be readily utilized as the basis for modifying or designing other structures for achieving the same purpose as this disclosure. Those skilled in the art will also recognize that such equivalent constructions do not depart from the spirit and scope of this disclosure as set forth in the appended claims. Novel features considered characteristic of this disclosure, whether relating to its organization or method of operation, together with further objects and advantages, will be better understood in conjunction with the accompanying drawings based on the following description. However, it should be clearly understood that each drawing is provided for illustrative and descriptive purposes only and is not intended to be a definition of limitation of this disclosure. Attached Figure Description

[0013] To gain a more complete understanding of this disclosure, reference is now made to the following description taken in conjunction with the accompanying drawings, in which: Figure 1 This is a schematic diagram illustrating a top view of a container yard according to a specific implementation plan.

[0014] Figure 2 This illustrates the management based on a specific implementation plan. Figure 1 A schematic diagram of a multimodal transport inventory management system for container yard inventory.

[0015] Figure 3 This is a schematic diagram illustrating an inventory imaging system for determining the parking location of transport containers according to a specific implementation plan.

[0016] Figure 4 This is a flowchart illustrating an example method for managing multimodal transport inventory according to a specific implementation plan.

[0017] Figures 5A-5D An example of a virtual container yard is shown according to a specific implementation scheme. In some implementation schemes, the virtual container yard can be displayed in a graphical user interface on a user's computing system to visualize the physical inventory of the physical container yard.

[0018] Figure 6 This illustrates the provision of services according to a specific implementation plan. Figures 5A-5D A flowchart illustrating an example method for visualizing multimodal transport inventory using a virtual container yard.

[0019] Figure 7 These are example computer systems that, according to specific implementation schemes, can be used to implement various aspects of the various technologies presented herein.

[0020] It should be understood that the accompanying drawings are not necessarily drawn to scale, and the disclosed embodiments are sometimes illustrated schematically and in partial views. In some cases, details that are not essential for understanding the disclosed methods and apparatus or that make other details imperceptible may have been omitted. Of course, it should be understood that this disclosure is not limited to the specific embodiments illustrated herein. Detailed Implementation

[0021] The disclosure and its various features and advantageous details presented in the following written description are explained more fully with reference to the accompanying drawings and the non-limiting embodiments as detailed in the specification. Descriptions of well-known components have been omitted to avoid unnecessarily obscuring the key features described herein. The embodiments used in the following description are intended to facilitate an understanding of how the present disclosure can be implemented and practiced. This disclosure will be interpreted by those skilled in the art as meaning that any suitable combination of the following functions or exemplary embodiments can be combined to achieve the claimed subject matter. This disclosure includes a representative number of species falling within the scope of a genus, or structural features common to members of that genus, enabling those skilled in the art to identify members of that genus. Therefore, these embodiments should not be construed as limiting the scope of the claims.

[0022] Those skilled in the art will understand that any system claims presented herein encompass all elements and limitations disclosed therein, and therefore each system claim should be considered as a whole. Any reasonably foreseeable items functionally related to the claims are also relevant. Having thoroughly understood the disclosure and claims of this application, the examiner has searched for prior art disclosed in patents and other publications (i.e., non-patent documents). Therefore, the grant of this patent demonstrates that the elements and limitations presented in the claims are supported by the specification and drawings, the granted claims address patentable subject matter, and the prior art fails to disclose or teach the claims as a whole, making the granted claims patentable under applicable national laws and regulations.

[0023] Multimodal container yards are facilities that transfer transport containers between trucks and trains. Most multimodal container yards include extensive storage areas for storing transport containers. For example, trucks can enter a multimodal container yard and are instructed to park the transport containers in designated parking areas. However, for various reasons, truck drivers may not park the transport containers in the requested parking locations. Therefore, multimodal container yards can utilize "yard checks" to ensure that transport containers can be located when needed for loading onto trains. Yard checks are typically performed by workers who visually observe units (e.g., trailers, containers, etc.) within the yard and their locations (e.g., zones / rows / positions) and manually enter the observed information into an inventory database. However, manual yard checks are time-consuming, monotonous, and prone to frequent errors.

[0024] To address these and other issues related to managing the inventory of transport containers, such as those stored in intermodal container yards, the disclosed embodiments provide systems and methods for imaging and identifying intermodal inventory. In some embodiments, the disclosed systems and methods utilize one or more inventory imaging systems that can move within the container yard to capture information about the yard. For example, the inventory imaging system can be attached to a container transport vehicle (e.g., a rail yard tractor), and as the container transport vehicle moves containers between trains and parking spaces within the container yard, it can capture information about a subset of the containers in the container yard. As another embodiment, the inventory imaging system can be attached to a dedicated imaging vehicle (e.g., an aircraft such as a drone, car, truck, golf cart, all-terrain vehicle (ATV), autonomous vehicle, motorcycle, remote-controlled vehicle, etc.), and as the dedicated imaging vehicle traverses a dedicated route throughout the container yard, it can capture information about the complete inventory of containers in the container yard. The inventory imaging system may include a computing system and imaging equipment (e.g., a camera). The inventory imaging system can be configured to continuously analyze images captured by the imaging device and identify containers within those images using an onboard machine learning module. Once a container is identified, the inventory imaging system can send information about the identified container (e.g., parking location, unit identification markings, etc.) to a remote computing system. By automatically performing image analysis (e.g., machine learning inference) in real time by the inventory imaging system located directly on the imaging vehicle, transport containers and their associated information (e.g., parking location and identification markings) can be updated in the inventory much faster (e.g., in real-time or near real-time) than other systems that rely on backend servers for image processing. Furthermore, some embodiments of the disclosed inventory imaging system save network bandwidth by electronically transmitting information only when an event of interest occurs (e.g., in response to the identification of a container within a captured image). These features and advantages of the disclosed embodiments, along with other features and advantages, are discussed in more detail below.

[0025] Figure 1This is a schematic top view illustrating a container yard 100 according to a specific embodiment. Typically, the container yard 100 can be an area within a multimodal rail yard for storing transport containers 110. Transport containers 110 can be parked in specific locations (e.g., parking location 300), which can be identified by zone 101, row 102, and position 103. For example, transport container 110A can be parked in zone 101A (“A”), row 102A (“1”), and position 103A (“1”). As another embodiment, transport container 110B can be parked in zone 101B (“B”), row 102D (“2”), and position 103B (“2”). Transport containers 110 can be any suitable container that can be transported and parked within the container yard 100. In some embodiments, transport container 110 is a multimodal transport container capable of being transported by both rail and truck.

[0026] In addition to transporting container 110, container yard 100 includes one or more imaging vehicles 120. Typically, imaging vehicle 120 has an attached imaging system (e.g., inventory imaging system 220) that captures and analyzes images of container yard 100 and transmits messages 125 regarding transporting container 110 to a remote computing system. In the illustrated embodiment, imaging vehicle 120 includes imaging vehicle 120A, imaging vehicle 120B, and imaging vehicle 120C. Imaging vehicle 120A may be an aircraft (e.g., a drone, a remotely controlled (RC) aircraft, etc.) having one or more attached inventory imaging systems. Imaging vehicle 120B may be a container transport vehicle (e.g., a site tractor, tractor, truck, etc.) having one or more attached inventory imaging systems. Imaging vehicle 120C may be any other ground vehicle (e.g., a car, multi-purpose vehicle, RC vehicle, etc.) having one or more attached inventory imaging systems.

[0027] Typically, an inventory imaging system attached to imaging vehicle 120 is a system and apparatus that captures and analyzes images of container yard 100 to identify transport containers 110 and their locations within the container yard 100. In some embodiments, imaging vehicle 120 (e.g., imaging vehicles 120A-120C) moves within container yard 100 and captures images or videos of transport containers 110. The inventory imaging system then uses a machine learning model to process the captured images or videos in real time to identify transport containers 110 within the captured images or videos. Once a transport container 110 is identified, the inventory imaging system may additionally determine one or more unit identifiers and parking locations of the identified transport container 110. The determined unit identifiers and parking locations of the transport container 110 are then electronically transmitted to a remote computer system for updating the inventory database of container yard 100. Further details regarding inventory imaging systems that can be attached to imaging vehicle 120 are discussed in more detail below.

[0028] Figure 2 This is a schematic diagram illustrating a multimodal transport inventory management system 200 for imaging and identifying inventory in a container yard 100, according to a specific implementation scheme. The multimodal transport inventory management system 200 includes a remote computing system 210, one or more inventory imaging systems 220, and a user computing system 240, all communicatively coupled via a network 230. The remote computing system 210 includes a memory 215 and an inventory database 214. The memory 215 stores a message processing module 211, and the inventory database 214 stores unit inventory 216. The inventory imaging computing system 250 is attached to an imaging vehicle 120 and includes a memory 255, an imaging device 251 (which stores a machine learning module 212), a GPS module 252, and a communication interface 258. The memory 255 stores a map 213 and images 217.

[0029] Typically, the inventory imaging system 220 captures and analyzes images 217 as the imaging vehicle 120 moves within the container yard 100 to determine the parking location of the transport container 110. To this end, the inventory imaging system 220 may first use a machine learning module 212 to analyze the images 217 captured by the imaging device 251 to determine that the transport container 110 is depicted in at least one image 217. The inventory imaging system 220 then uses a map 213 of the container yard 100 to determine the parking location of the transport container 110 identified in image 217. In response to determining that the transport container 110 is depicted in at least one image 217, the inventory imaging system 220 may electronically transmit (e.g., via network 230 using communication interface 258) a message 125 about the identified transport container 110 to a telecomputing system 210. The message 125 sent to the telecomputing system 210 may include data such as the determined parking location of the identified transport container 110 and one or more identification tags of the identified transport container 110. To conserve network bandwidth, some implementations of the inventory imaging system 220 send message 125 only when the machine learning module 212 identifies the shipping container 110 within image 217. Once message 125 is received by the remote computing system 210, the message processing module 211 uses the data within message 125 to update the unit inventory 216. Further details regarding the operation of the inventory imaging system 220 are discussed below.

[0030] Message 125 may be information provided by the inventory imaging system 220 to the multimodal transport inventory management system 200. Typically, each message 125 provides data / information about transport containers 110 within the container yard 100. In some embodiments, message 125 includes the unit identification number of a specific transport container 110 and the parking location of that specific transport container 110 within the container yard 100. For example, when the inventory imaging system 220 determines that transport container 110A has the unit identification number “BICU123456” and is parked in zone / row / position A-1-1 (e.g., ...), the message may be part of the multimodal transport inventory management system 200. Figure 1 When (as illustrated in the example), the inventory imaging system 220 can send message 125 to the multimodal transport inventory management system 200, which includes the unit identification number "BICU 123456" and the location of the zone / row / position A-1-1. As another embodiment, when the inventory imaging system 220 determines that transport container 110B has the unit identification number "BICU 654321" and is parked in zone / row / position B-2-2 (as illustrated in the example), the inventory imaging system 220 can send message 125 to the multimodal transport inventory management system 200. Message 125 includes the unit identification number "BICU 654321" and the location of the zone / row / position B-2-2 (as illustrated in the example). Figure 1 When (as illustrated in the example), the yard inspection source 120B can send message 125 to the multimodal transport inventory management system 200. Message 125 includes the unit identification number "BICU 654321" and the location of the zone / row / position of B-2-2.

[0031] Message 125 can be in any format, including JavaScript Object Notation (JSON), TCP / IP, XML, HTML, ASCII, SMS, CSV, API, or other suitable formats. Each message 125 can consist of a message header, header attributes, and a message body, or can be encapsulated and grouped in any suitable format. In some implementations, message 125 is sent via a distributed event streaming platform such as Apache Kafka.

[0032] In some implementations, the multimodal transport inventory management system 200 includes a message processing module 211. Typically, the message processing module 211 processes messages 125 received from the inventory imaging system 220 and stores data within message 125 regarding the transport container 110 in the unit inventory 216. The message processing module 211 represents any suitable set of instructions, logic, or code contained in a computer-readable storage medium. For example, the message processing module 211 may be contained in a memory 215, a disk, a CD, or a flash drive. In specific implementations, the message processing module 211 may include processor-executable instructions (e.g., software applications) to perform one or more of the functions described herein.

[0033] The machine learning module 212 can be any hardware, software, or a combination of hardware and software capable of analyzing image 217 to identify shipping containers 110 within image 217 in real-time or near real-time. In some embodiments, the machine learning module 212 includes one or more neural networks that have been trained to perform object detection, reasoning, and recognition on image 217 to identify shipping containers 110.

[0034] Map 213 stores information about the physical properties of container yard 100. In some embodiments, map 213 includes information about the parking locations within container yard 100. For example, map 213 may include GPS coordinates for each parking location. As a specific embodiment, map 213 may include coverage of each parking location within container yard 100 (e.g., such as...). Figure 1 The GPS coordinates of the four corners of the rectangle (the location of the zone / row / position) illustrated in A-1-1. In some implementations, map 213 is a Geographic Information System (GIS) map.

[0035] The inventory database 214 of the remote computing system 210 can store certain types of information for the remote computing system 210 and the message processing module 211. The inventory database 214 can be any or a combination of volatile or non-volatile local or remote devices suitable for storing information. The inventory database 214 may include RAM, ROM, magnetic storage devices, optical storage devices, or any other suitable information storage devices or combinations thereof. The inventory database 214 can be an external or internal component of the remote computing system 210. The inventory database 214 can be located in any location suitable for storing information for the remote computing system 210. For example, the inventory database 214 may be located in a cloud environment. Typically, the inventory database 214 stores a unit inventory 216, which includes data from message 125 (e.g., the parking location of the shipping container 110 and the unit identifier of the shipping container 110).

[0036] Image 217 is captured by imaging device 251 and can be stored in memory 255. Image 217 can be a still image or it can be video. Image 217 can be in a suitable data format and can be any suitable resolution and size.

[0037] Imaging device 251 is any suitable system or device for capturing images 217 of transport containers 110 within container yard 100. In some embodiments, imaging device 251 is a high-resolution camera from LUXONIS that includes built-in capabilities for machine learning (e.g., via machine learning module 212) and depth vision. In some embodiments, imaging device 251 includes a field of view 260 that can be pointed in a known direction away from imaging vehicle 120, such that it captures transport containers 110 as imaging vehicle 120 moves along row 102 of container yard 100.

[0038] GPS module 252 can be any suitable system, device, or application used to communicate with one or more satellites to determine the Global Positioning System (GPS) coordinates of inventory imaging system 220. GPS coordinates can be the current latitude, longitude, and altitude of inventory imaging system 220. In some embodiments, GPS module 252 is any typical GPS receiver.

[0039] Communication interface 258 can be any hardware, software, or a combination of hardware and software that provides one or more wireless interfaces (e.g., Wi-Fi, cellular, Bluetooth, etc.) between the stock imaging system 220 and the network 230. In some embodiments, communication interface 258 provides a Wi-Fi interface via LTE (or any other cellular communication standard). In some embodiments, communication interface 258 can be communication interface 610, which is discussed below. Figure 6Let's have a more detailed discussion.

[0040] In some embodiments, the inventory imaging system 220 may additionally include one or more lighting devices (not illustrated) configured to illuminate the transport container 110 as the imaging vehicle 120 moves within the container yard 100. For example, the inventory imaging system 220 may include one or more LED lights attached to the imaging vehicle 120 and pointed at the transport container 110. As another embodiment, the inventory imaging system 220 may include one or more infrared lights attached to the imaging vehicle 120 and pointed at the transport container 110.

[0041] As discussed above, the inventory imaging system 220 is a system and apparatus configured to capture and analyze images of the container yard 100 to identify the transport container 110 and its location within the container yard 100. In some embodiments, the inventory imaging system 220 is coupled to an imaging vehicle 120 moving within the container yard 100. However, in other embodiments, the inventory imaging system 220 may be attached to other objects within the container yard 100. For example, the inventory imaging system 220 may be attached to a stationary object closely adjacent to the entrance / exit of the container yard 100 to analyze the transport container 110 as it enters / leaves the container yard 100. As another embodiment, the inventory imaging system 220 may be attached to a crane loading / unloading the transport container 110 from a train. Typically, the inventory imaging system 220 may be located on any moving or stationary object to capture and analyze images of the container yard 100 to identify the transport container 110 and its location within the container yard 100.

[0042] Network 230 allows communication between the various components of the multimodal transport inventory management system 200. This disclosure contemplates network 230 as any suitable network operable to facilitate communication between the components of the multimodal transport inventory management system 200. Network 230 may include any interconnected system capable of transmitting audio, video, signals, data, messages, or any combination thereof. Network 230 may include all or part of a local area network (LAN), wide area network (WAN), overlay network, software-defined network (SDN), virtual private network (VPN), packet data network (e.g., the Internet), mobile phone network (e.g., cellular network, such as 4G or 5G), common legacy telephone (POT) network, wireless data network (e.g., WiFi, WiGig, WiMax, etc.), Long Term Evolution (LTE) network, Universal Mobile Telecommunications System (UMTS) network, peer-to-peer (P2P) network, Bluetooth network, near field communication network, Zigbee network, and / or any other suitable network.

[0043] User computing system 240 can be any suitable system or device that allows users to interact with multimodal transport inventory management system 200. User computing system 240 can be a mobile device, smartphone, tablet computing device, personal computing device, laptop computing device, desktop computing device, vehicle computer system, personal digital assistant (PDA), smartwatch, other types of wired and / or wireless computing device, or any part thereof. In some embodiments, user computing system 240 includes electronic display 242, such as a computer or smartphone screen. In some embodiments, user computing system 240 can be configured to provide an interface (e.g., GUI 244) that can be structured to facilitate interaction between an operator, for example via network 230, and the multimodal transport inventory management system 200 to perform and utilize the features provided by the multimodal transport inventory management system 200. In some embodiments, GUI 244 can be used to display a real-time three-dimensional representation of transport container 110 and container yard 100, as referenced below. Figures 5C-5D The subject of discussion.

[0044] In operation, the multimodal transport inventory management system 200 utilizes one or more inventory imaging systems 220 to capture and analyze images 217 as one or more imaging vehicles 120 move within the container yard 100 to determine the parking location of transport containers 110. For this purpose, the inventory imaging system 220 may first capture images 217 using imaging equipment 251 as the imaging vehicles 120 move within the container yard 100. After capturing image 217, the inventory imaging system 220 analyzes image 217 using a machine learning module 212 to determine in real-time (or near real-time) that at least one transport container 110 is depicted in image 217. The inventory imaging system 220 can then use a map 213 of the container yard 100 to determine the parking location of the transport containers 110 identified in image 217. Further details regarding how certain embodiments determine the parking location of transport containers 110 are referenced below. Figure 3 discuss.

[0045] In response to determining that the transport container 110 is depicted within at least one image 217, the inventory imaging system 220 may electronically transmit (e.g., via network 230 using communication interface 258) a message 125 about the identified transport container 110 to the telecomputing system 210. The message 125 sent to the telecomputing system 210 may include data such as the determined parking location of the identified transport container 110 and one or more identification marks of the identified transport container 110. Further details regarding how the identification marks of the transport container 110 are determined in certain embodiments are referenced below. Figure 3Discussion. Once message 125 is received by remote computing system 210, message processing module 211 uses the data within message 125 to update unit inventory 216.

[0046] To conserve network bandwidth, battery power, and computer processing / memory resources, some implementations of the inventory imaging system 220 only send message 125 when an object of interest is found in the captured image 217. For example, some implementations only send message 125 in response to the machine learning module 212 identifying a shipping container 110 within image 217 (i.e., message 125 is sent by the inventory imaging system 220 after each new shipping container 110 is identified within image 217, but in other cases, the inventory imaging system 220 does not use communication interface 258 to conserve resources).

[0047] By performing object detection directly within the machine learning module 212 of the imaging device 251 (rather than within a backend server system, such as the remote computing system 210), the implementation of this disclosure offers significant advantages over other systems. Because the inventory imaging system 220 can utilize the machine learning module 212 and the imaging device 251 to detect transport containers 110 in real time as the imaging vehicle 120 traverses the container yard 100, the unit inventory 216 can be updated more quickly and efficiently. Therefore, operators can gain a more accurate understanding of the inventory within the container yard 100.

[0048] In some implementations, the inventory imaging system 220 can determine whether it is currently located within the geofence 104 of the container yard 100 (e.g., using a GPS module 252). Figure 1(Examples are provided) to save additional resources. In these embodiments, when the inventory imaging system 220 is currently outside the geofence 104 of the container yard 100, the inventory imaging system 220 can prevent the machine learning module 212 from analyzing image 217. To determine whether the inventory imaging system 220 is currently inside or outside the geofence 104 of the container yard 100, some embodiments compare the current GPS coordinates of the inventory imaging system 220 from the GPS module 252 with the GPS coordinates of the geofence 104. For example, the geofence 104 can be square or rectangular in shape and can include the GPS coordinates of the four corners of the geofence 104. If the inventory imaging system 220 determines that its current GPS coordinates are within the GPS coordinates of the four corners of the geofence 104, then the inventory imaging system 220 determines that it is currently inside the geofence 104, and thus enables the machine learning module 212 to analyze image 217 to identify the shipping container 110. On the other hand, if the inventory imaging system 220 determines that the current GPS coordinates of the inventory imaging system 220 are not within the GPS coordinates of the four corners of the geofence 104, the inventory imaging system 220 determines that the inventory imaging system 220 is currently outside the geofence 104, and therefore disables and prevents the machine learning module 212 from analyzing the image 217 in order to save resources.

[0049] Figure 3 This is a schematic diagram illustrating how an inventory imaging system 220 determines the parking position 300 (e.g., 300A-300C) of a shipping container 110. Typically, once the machine learning module 212 detects the shipping container 110 within one or more images 217 captured by the imaging device 251, the inventory imaging system 220 then attempts to determine the exact parking position 300 of the detected shipping container 110. For this purpose, some implementations first determine the current field of view 360 of the imaging device 251 of the inventory imaging system 220. The inventory imaging system 220 can determine the current field of view 360 of the imaging device 251 by analyzing the system's current GPS coordinates from the GPS module 252. For example, if the current GPS coordinates of the system are (N, W, m) and the imaging device 251 has a fixed field of view 260 of 120 degrees facing 90 degrees from the direction of travel of the imaging vehicle 120, then the stock imaging system 220 can determine that the current field of view 360 of the imaging device 251 is located at (N, W, m) and facing 120 degrees facing 90 degrees from the direction of travel of the imaging vehicle 120.

[0050] Once the inventory imaging system 220 has determined the current field of view 360 of the imaging device 251, the inventory imaging system 220 can then determine possible parking locations 300 within the current field of view 360 of the imaging device 251. In some embodiments, the inventory imaging system 220 compares the determined current field of view 360 of the imaging device 251 with a map 213 of the stored container yard 100 to determine possible parking locations 300 of the detected transport container 110. Using the above embodiments, the inventory imaging system 220 can overlay the current field of view 360 located at (N, W, m) and facing 90 degrees to 120 degrees from the direction of travel of the imaging vehicle 120 onto the map 213 and determine that parking locations 300A-300C are within the current field of view 360 of the imaging device 251 (e.g., Figure 3 exemplified in ).

[0051] Once the inventory imaging system 220 has determined the possible parking locations 300 within the current field of view 360 of the imaging device 251, the inventory imaging system 220 determines the GPS coordinates of each of the possible parking locations. In some embodiments, the inventory imaging system 220 can do this by accessing map 213 and determining the GPS coordinates stored in map 213 for each of the possible parking locations. For example, each parking location in map 213 could be a rectangular area that includes the GPS coordinates of each of the four corners of the rectangular area.

[0052] Next, the inventory imaging system 220 can use one or more settings of the imaging device 251 and the current GPS coordinates of the inventory imaging system 220 to calculate the GPS coordinates 370 of the detected transport container 110. In this step, the inventory imaging system 220 can first determine its current GPS coordinates based on the GPS module 252. Then, the inventory imaging system 220 can utilize the imaging device 251 to determine the distance from the inventory imaging system 220 to the transport container 110. For example, in some embodiments, the inventory imaging system 220 can utilize one or more settings of the imaging device (e.g., the focal length and resolution of the imaging device 251) to determine the distance from the inventory imaging system 220 to the transport container 110. In other embodiments, the imaging device 251 may include one or more depth sensors within the imaging device 251 to determine the distance from the inventory imaging system 220 to the transport container 110. Finally, the inventory imaging system 220 can calculate the GPS coordinates 370 of the detected transport container 110 by adjusting the current GPS coordinates of the inventory imaging system 220 with the determined distance from the inventory imaging system 220 to the transport container 110.

[0053] After the inventory imaging system 220 determines the GPS coordinates 370 of the detected transport container 110, the inventory imaging system 220 can then determine the most likely parking location 300 of the transport container 110 by comparing the GPS coordinates of each of the possible parking locations with the GPS coordinates of the transport container. For example, if the inventory imaging system 220 determines that the GPS coordinates 370 of the detected transport container 110 are within the GPS coordinates of the four corners of the parking location 300C (e.g., ... Figure 3 (As illustrated in the example), the inventory imaging system 220 determines that the detected transport container 110 is currently located at parking position 300C.

[0054] In some implementations, the inventory imaging system 220 can determine the most likely parking location 300 of the shipping container 110 across multiple images 217, and then select the parking location 300 that is selected most frequently. For example, consider a scenario where an imaging vehicle 120 is moving through a container yard 100 and capturing ten images 217 in which a particular shipping container 110 is detected by the machine learning module 212. Following the steps discussed above, the inventory imaging system 220 determines that the particular shipping container 110 is located twice at parking location 300A, once at parking location 300B, and seven times at parking location 300C. In this scenario, the inventory imaging system 220 can ultimately determine that the particular shipping container 110 is located at parking location 300C because parking location 300C is selected most frequently across the ten images.

[0055] In some implementations, the inventory imaging system 220 may utilize any suitable method or algorithm for determining the unit identification number of the shipping container 110 to be included in message 125. For example, Figure 3 An inventory imaging system 220 is illustrated to determine unit identifier 310 based on image 320. In this embodiment, the inventory imaging system 220 captures image 320 of shipping container 110 and then uses optical character recognition (OCR) or machine learning algorithms to analyze image 320 and determine that unit identifier 310 of shipping container 110 is “BICU 123456”. In other embodiments, the inventory imaging system 220 may utilize any other suitable technology or technical means to determine unit identifier 310 (e.g., radio frequency identification (RFID), Bluetooth, Wi-Fi, etc.).

[0056] The functional blocks and components of the multimodal transport inventory management system 200 of this disclosure can be implemented using processors, electronic devices, hardware devices, electronic components, logic circuits, memory, software code, firmware code, and any combination thereof. For example, one or more functional blocks or portions thereof can be implemented as discrete gate or transistor logic, discrete hardware components, or combinations thereof configured to provide logic for performing the functions described herein. Additionally or alternatively, when implemented in software, one or more functional blocks or portions thereof may include code segments operable on a processor to provide logic for performing the functions described herein.

[0057] The various components of the multimodal transport inventory management system 200 can be exemplified as single and individual components. However, it should be understood that each of the exemplified components can be implemented as a single component (e.g., a single application, server module, etc.), can be a functional component of a single component, or the functionality of these various components can be distributed across multiple devices / components. In such an implementation, the functionality of each corresponding component can be aggregated from the functionality of multiple modules residing in a single device or multiple devices.

[0058] Furthermore, the functions described in each of the different functional blocks of the multimodal transport inventory management system 200 described herein are provided for illustrative purposes only and not in a restrictive manner, and the functions described as being provided by different functional blocks may be combined into a single component or may be provided via computing resources set up in a cloud-based environment accessible via a network (such as one of the networks 230).

[0059] Figure 4 This is a flowchart illustrating an example method 400 for managing multimodal transport inventory according to certain implementation schemes. In some implementation schemes, in Figure 4 The functionality illustrated in the example block can be performed by the stock imaging system 220 according to an embodiment of this document. In some embodiments, the operation of method 400 can be stored as instructions that, when executed by one or more processors, cause one or more processors to perform the operation of method 400.

[0060] Method 400 can be executed by a computing system and can begin in step 410. In step 410, method 400 accesses multiple images captured by an imaging device. In some embodiments, the multiple images are image 217 and the imaging device is imaging device 251. In some embodiments, the imaging device and the computing system are coupled to a vehicle such as imaging vehicle 120, which moves within a multimodal container yard. The vehicle can be a container truck, aircraft, car, truck, golf cart, ATV, autonomous vehicle, motorcycle, or remote-controlled vehicle.

[0061] In step 420, method 400 uses a machine learning module to analyze multiple images to determine that the transport container is depicted within at least one of the multiple images. In some embodiments, the machine learning module is machine learning module 212 located within imaging device 251. In some embodiments, step 420 is performed only if the computing system is physically located within the geofence of the multimodal container yard.

[0062] In step 430, method 400 determines the parking location of the transport container. In some embodiments, method 400 uses a map of the multimodal container yard in step 430 to determine the parking location of the transport container. In some embodiments, the map is map 213. In some embodiments, the parking location is parking location 300.

[0063] In some implementations, step 430 includes determining the current field of view of the imaging device by analyzing the current GPS coordinates of the system. The current GPS coordinates can be obtained from a GPS receiver (such as GPS module 252). Step 430 may also include determining multiple possible parking locations of the multimodal container yard within the field of view of the imaging device using the determined current field of view of the imaging device and a stored map of the multimodal container yard. Step 430 may also include determining the GPS coordinates of each of the possible parking locations within the field of view of the imaging device based on the stored map of the multimodal container yard. Step 430 may also include calculating the GPS coordinates of the transport container using one or more settings of the imaging device and the current GPS coordinates of the system. The one or more settings of the imaging device may include the focal length and resolution of the imaging device. Step 430 may also include determining the parking location of the transport container by comparing the GPS coordinates of each of the possible parking locations with the GPS coordinates of the transport container.

[0064] In step 440, in response to determining that the transport container is depicted in at least one of a plurality of images, method 400 electronically transmits a message including data about the transport container via a communication network. In some embodiments, the message is message 125. In some embodiments, the data about the transport container includes the determined parking location of the transport container and one or more identification marks on the transport container. After step 440, method 400 may terminate.

[0065] Figures 5A-5D Various views of a virtual container yard 500, which can be displayed in a GUI 244 on a user computing system 240 according to certain embodiments, are illustrated. Typically, the virtual container yard 500 is a three-dimensional representation of the container yard 100, allowing the user to view the real-time (or near real-time) status of the container yard 100 (e.g., the location of inventory and vehicles, such as imaging vehicle 120). For example, a user can view the virtual container yard 500 in a GUI 244 on an electronic display 242 using a user computing system 240 (such as a smartphone or computer). The user can manipulate (e.g., move, rotate, zoom in, zoom out, etc.) the virtual container yard 500 to view the current status of the transport containers 110 and vehicles (such as imaging vehicle 120) within the container yard 100, as described in more detail below.

[0066] In some implementations, a computer system (such as remote computing system 210) can continuously update the virtual container yard 500 using information from unit inventories 216 stored in inventory database 214 from multimodal transport inventory management system 200, as described above. For example, remote computing system 210 can access unit inventories 216 at predetermined intervals (e.g., per second, per minute, per hour, etc.) and update the virtual container yard 500 accordingly. In other implementations, remote computing system 210 can access unit inventories 216 when a change is detected in unit inventories 216, and then update the virtual container yard 500 accordingly. In still other implementations, remote computing system 210 can access unit inventories 216 after receiving message 125 from inventory management system 200, and then update the virtual container yard 500 accordingly. Thus, the virtual container yard 500 can be updated in real time (or near real time) to provide an accurate depiction of the current state of container yard 100.

[0067] In some implementations, the virtual container yard 500 is a three-dimensional model of the container yard 100. In some implementations, a map of the container yard 100 (such as map 213) is used to generate the virtual container yard 500. Generally, any suitable computer-based 3D modeling software or application can be used to generate the virtual container yard 500.

[0068] like Figures 5A-5C As illustrated, some embodiments of the virtual container yard 500 include a visual representation 510 of the transport container 110, a virtual parking location 520, and a visual representation 530 of vehicles (e.g., imaging vehicle 120 or site tractor). The visual representation 510 of the transport container 110 is typically a three-dimensional model of the transport container 110 placed at the virtual parking location 520, which corresponds to the physical parking location (e.g., parking location 300) of the corresponding transport container 110 in the container yard 100, as determined by the inventory imaging system 220. In some embodiments, the appearance of the visual representation 510 of the transport container 110 is based on the carrier identification of the transport container 110 (e.g., current carrier identification 543). For example, all shipping containers 110 identified as belonging to carrier "ABC" may have a visual representation 510A, which has a first appearance matching that of typical shipping containers of carrier "ABC", while all shipping containers 110 identified as belonging to carrier "XYZ" may have a visual representation 510B, which has a second appearance matching that of typical shipping containers of carrier "XYX", such as... Figure 5B As illustrated in the example. In other embodiments, different colors, shapes, shadows, etc., can be used on the visual representation 510 to distinguish between transport containers 110 identified as belonging to carrier "ABC" and transport containers 110 identified as belonging to carrier "XYZ".

[0069] In some embodiments, the virtual container yard 500 includes visual representations 530 of vehicles (such as imaging vehicle 120 or site tractor). The visual representation 530 is typically a three-dimensional model of the vehicle, placed within the virtual container yard 500 at a location corresponding to the physical location of the actual vehicle within the container yard 100. In some embodiments, the visual representation 530 of the vehicle is placed within the virtual container yard 500 using GPS coordinates from the vehicle (e.g., from GPS module 252). In some embodiments, the appearance of each visual representation 530 is based on the type of vehicle represented. For example, such as... Figure 5A and Figure 5C As illustrated, when the vehicle being represented is a site towing vehicle, the visualization 530 can be presented as a site towing vehicle.

[0070] In some implementations, the virtual container yard 500 can display an instruction 541 for each virtual parking location 520. For example, as Figure 5CAs illustrated, some embodiments may display indication 541 when a user selects a virtual parking location 520 or otherwise interacts with it (e.g., by hovering a pointer over it). In some embodiments, indication 541 may include a zone identifier (e.g., "Zone: C"), a row identifier (e.g., "Row: 1"), and a bit identifier (e.g., "Bit: 20") for the virtual parking location 520, such as... Figure 5C exemplified in .

[0071] like Figure 5D As illustrated, some embodiments of the virtual container yard 500 include metadata 540 about the transport container 110. For example, the metadata 540 about the transport container 110 may be displayed in response to a user's selection of a specific visual representation 510 or virtual parking location 520 for the transport container 110. In some embodiments, the metadata 540 about the transport container 110 includes an indication 541 of parking location 300 (e.g., "Area C - Row 1 - Position 20") and a current image 542 of parking location 300. The current image 542 of parking location 300 may be captured, for example, by imaging device 251 as imaging vehicle 120 moves within container yard 100, as described above. Typically, the current image 542 of parking location 300 is the most recent (i.e., most recent) image available for the selected parking location.

[0072] In some implementations, the metadata 540 regarding the transport container 110 includes the current carrier identifier 543 (e.g., carrier / owner “ABC”) and current identification number 544 (e.g., “27431”) of the transport container 110 located at the selected parking location. Typically, the current carrier identifier 543 and current identification number 544 are physical markings on the transport container 110 that provide a unique identifier for the transport container 110. In some implementations, the current carrier identifier 543 and current identification number 544 are determined by the inventory imaging system 220, as referenced above. Figure 3 As described (e.g., using OCR or machine learning algorithms to analyze image 320).

[0073] In some implementations, metadata 540 regarding the transport container 110 includes a timestamp 545, chassis initials 546, chassis number 547, URL 548, and camera ID 549. The timestamp 545 may indicate the time when the current image 542 of parking location 300 was captured. The chassis initials 546 may include one or more characters associated with the chassis, the chassis number 547 may include one or more characters associated with the chassis, and the URL 548 may provide a hyperlink to media associated with the chassis or container. The camera ID 549 may be an identifier for the camera (e.g., imaging device 251) used to capture the current image 542 of parking location 300.

[0074] In some implementations, the metadata 540 regarding the transport container 110 includes a history of information captured about the selected parking location. For example, such as... Figure 5D As illustrated, the metadata 540 regarding the transport container 110 may include a previous image 552 of the parking location 300 of the transport container 110, a previous carrier identifier 553, a previous identification number 554, a timestamp 555 of the previous image 552, and a camera ID 549 of the camera used to capture the previous image 552. While the metadata 540 regarding the transport container 110 illustrates a single previous image 552 and associated data, other embodiments may include more than one previous image 552 and its associated data.

[0075] In some implementations, the metadata 540 regarding the transport container 110 includes a confidence value 560 associated with the current carrier identifier 543 and current identification number 544 of the selected transport container. Typically, the confidence value 560 can be provided by the inventory imaging system 220 and can indicate how reliable the current carrier identifier 543 and current identification number 544 of the selected transport container can be. In some scenarios, it may be difficult to distinguish the characters of the unit identifier 310 with absolute certainty, as in reference... Figure 3 The aforementioned uncertainty may be due to, for example, low-quality image 320 (e.g., due to weather or a dirty camera lens) or the presence of characters that are difficult to determine using OCR (e.g., "3" vs. "8", "8" vs. "B", etc.). To account for this uncertainty, the inventory imaging system 220 may, in some embodiments, use any suitable technique to calculate a confidence level 560 and then send the confidence level 560 for display in the metadata 540 regarding the shipping container 110. In some embodiments, the confidence level 560 may be represented as a value between 0 and 100.

[0076] Figure 6 This is a flowchart illustrating an example method 600 for visualizing multimodal transport inventory according to certain implementation schemes. In some implementation schemes, in Figure 6 The functionality illustrated in the example blocks can be performed by one or both of the stock imaging system 220 and the remote computing system 210 according to the embodiments herein. In some embodiments, the operation of method 600 can be stored as instructions that, when executed by one or more processors, cause one or more processors to perform the operation of method 600.

[0077] Method 600 can be executed by a computing system and can begin in step 610. In step 610, method 600 accesses a unit inventory stored in an inventory database. This unit inventory includes information about multiple transport containers physically located in a multimodal container yard. In some embodiments, the unit inventory is unit inventory 216 and the inventory database is inventory database 214. In some embodiments, the computing system is remote computing system 210. In some embodiments, the transport container is transport container 110 and the multimodal container yard is container yard 100.

[0078] At step 620, method 600 provides one or more instructions to display a graphical user interface (GUI) on an electronic display of a user computing system, the GUI including a three-dimensional representation of the multimodal container yard. In some embodiments, the electronic display is electronic display 242, and the user computing system is user computing system 240. In some embodiments, the GUI is GUI 244, and the three-dimensional representation of the multimodal container yard is a virtual container yard 500.

[0079] At step 630, method 600 updates the three-dimensional representation of the multimodal container yard based on information about multiple transport containers in the unit inventory. In some embodiments, method 600 updates the three-dimensional representation of the multimodal container yard in real-time or near real-time in step 630. In some embodiments, method 600 updates the three-dimensional representation of the multimodal container yard at predetermined intervals (e.g., per second, per minute, per hour, etc.) in step 630. In other embodiments, method 600 updates the three-dimensional representation of the multimodal container yard in step 630 when a change is detected in unit inventory 216. In still other embodiments, method 600 updates the three-dimensional representation of the multimodal container yard in step 630 after receiving message 125 from inventory imaging system 220. In some embodiments, step 630 includes: providing a visual representation (e.g., visualization 510) of each transport container physically located within the multimodal container yard in the three-dimensional representation of the multimodal container yard, wherein the visual representation of each particular transport container is placed at a virtual parking location (e.g., virtual parking location 520) in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the particular transport container in the multimodal container yard (e.g., parking location 300). In some embodiments, step 630 includes: providing a visual representation (e.g., visualization 530) of the current location of vehicles traveling within the multimodal container yard in the three-dimensional representation of the multimodal container yard.

[0080] At step 640, method 600 provides one or more instructions to display metadata about the specific shipping container in the graphical user interface in response to a user's selection of a specific visual representation of the specific shipping container. In some embodiments, the metadata is metadata 540 about shipping container 110. In some embodiments, the metadata includes one or more of the following: an indication of a specific physical parking location of the specific shipping container (e.g., indication 541 of parking location 300); a current image of the specific physical parking location (e.g., current image 542); a current carrier identifier of the specific shipping container (e.g., current carrier identifier 543); a current identification number of the specific shipping container (e.g., current identification number 544); and a timestamp of the current image (e.g., timestamp 545). In some implementations, the metadata includes one or more of the following: a previous image of the particular physical parking location (e.g., previous image 552); a previous carrier identifier of the particular shipping container (e.g., previous carrier identifier 553); a previous identification number of the particular shipping container (e.g., previous identification number 554); and a timestamp of the previous image (e.g., timestamp 555). After step 640, method 600 may end.

[0081] Figure 7 An example computer system 700 is illustrated, which can be used as a remote computing system 210 and a stock imaging computing system 250. In specific embodiments, one or more computer systems 700 perform one or more steps of one or more methods described or illustrated herein. In specific embodiments, one or more computer systems 700 provide the functionality described or illustrated herein. In specific embodiments, software running on one or more computer systems 700 performs one or more steps of one or more methods described or illustrated herein, or provides the functionality described or illustrated herein. Specific embodiments include one or more portions of one or more computer systems 700. In this document, references to computer systems may cover computing devices and vice versa, where appropriate. Furthermore, references to computer systems may cover one or more computer systems, where appropriate.

[0082] This disclosure contemplates any suitable number of computer systems 700. This disclosure contemplates that the computer systems 700 may take any suitable physical form. As an example and not a limitation, the computer system 700 may be an embedded computer system, a system-on-a-chip (SOC), a single-board computer system (SBC) (such as, for example, a modular computer (COM) or a modular system (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a computer system grid, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, the computer system 700 may: comprise one or more computer systems 700; be single or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 700 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. By way of example and not limitation, one or more computer systems 700 may execute one or more steps of one or more methods described or illustrated herein in real time or in batch mode. Where appropriate, one or more computer systems 700 may execute one or more steps of one or more methods described or illustrated herein at different times or at different locations.

[0083] In a specific embodiment, computer system 700 includes a processor 702, a memory 704, a storage device 706, an input / output (I / O) interface 708, a communication interface 710, and a bus 712. Although this disclosure describes and exemplifies a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0084] In a specific embodiment, processor 702 includes hardware for executing instructions, such as those constituting a computer program. By way of example and not limitation, to execute instructions, processor 702 may: retrieve (or fetch) instructions from internal registers, internal caches, memory 704, or storage device 706; decode and execute them; and then write one or more results to internal registers, internal caches, memory 704, or storage device 706. In a specific embodiment, processor 702 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates that processor 702 may include any suitable number of suitable internal caches where appropriate. By way of example and not limitation, processor 702 may include one or more instruction caches, one or more data caches, and one or more translation back buffers (TLBs). Instructions in the instruction cache may be copies of instructions in memory 704 or storage device 706, and the instruction cache may accelerate the retrieval of these instructions by processor 702. The data in the data cache may be: a copy of data in memory 704 or storage device 706 for operation by instructions executed at processor 702; the result of a previous instruction executed at processor 702 for access by subsequent instructions executed at processor 702 or for writing to memory 704 or storage device 706; or other suitable data. The data cache can accelerate read or write operations of processor 702. The TLB can accelerate virtual address translation of processor 702. In specific embodiments, processor 702 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates that processor 702 may include any suitable number of suitable internal registers where appropriate. Where appropriate, processor 702 may: include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 702. Although this disclosure describes and exemplifies a particular processor, this disclosure contemplates any suitable processor.

[0085] In a specific embodiment, memory 704 includes main memory for storing instructions to be executed by processor 702 or data to be operated by processor 702. By way of example and not limitation, computer system 700 may load instructions from storage device 706 or other sources (such as, for example, another computer system 700) into memory 704. Processor 702 may then load the instructions from memory 704 into internal registers or internal caches. To execute the instructions, processor 702 may retrieve and decode the instructions from internal registers or internal caches. During or after instruction execution, processor 702 may write one or more results (which may be intermediate or final results) into internal registers or internal caches. Processor 702 may then write one or more of these results into memory 704. In a specific embodiment, processor 702 executes only instructions that are in one or more internal registers or internal caches or in memory 704 (not in storage device 706 or elsewhere), and operates only on data that is in one or more internal registers or internal caches or in memory 704 (not in storage device 706 or elsewhere). One or more memory buses (each of which may include an address bus and a data bus) couple processor 702 to memory 704. Bus 712 may include one or more memory buses, as described below. In a specific embodiment, one or more memory management units (MMUs) are located between processor 702 and memory 704 and facilitate access to memory 704 requested by processor 702. In a specific embodiment, memory 704 includes random access memory (RAM). Where appropriate, this RAM may be volatile memory. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Furthermore, where appropriate, this RAM may be single-port or multi-port RAM. This disclosure contemplates any suitable RAM. Where appropriate, memory 704 may include one or more memories 704. Although this disclosure describes and exemplifies specific memories, this disclosure contemplates any suitable memory.

[0086] In a specific embodiment, storage device 706 includes a mass storage device for data or instructions. By way of example and not limitation, storage device 706 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, storage device 706 may include removable or non-removable (or fixed) media. Where appropriate, storage device 706 may be internal or external to computer system 700. In a specific embodiment, storage device 706 may be a non-volatile solid-state memory. In a specific embodiment, storage device 706 includes read-only memory (ROM). Where appropriate, this ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically changeable ROM (EAROM), or flash memory, or a combination of two or more of these. This disclosure contemplates any suitable physical form for mass storage device 706. Where appropriate, storage device 706 may include one or more storage control units that facilitate communication between processor 702 and storage device 706. Where appropriate, storage device 706 may include one or more storage devices 706. Although this disclosure describes and exemplifies particular storage devices, any suitable storage device is contemplated herein.

[0087] In specific embodiments, I / O interface 708 includes hardware, software, or both that provide one or more interfaces for communication between computer system 700 and one or more I / O devices. Where appropriate, computer system 700 may include one or more of these I / O devices. One or more of these I / O devices can enable communication between a person and computer system 700. By way of example and not limitation, I / O devices may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet computer, touchscreen, trackball, video camera, another suitable I / O device, or a combination of two or more of these. I / O devices may include one or more sensors. This disclosure contemplates any suitable I / O device and any suitable I / O interface 708 for them. Where appropriate, I / O interface 708 may include one or more device drivers or software drivers that enable processor 702 to drive one or more of these I / O devices. Where appropriate, I / O interface 708 may include one or more I / O interfaces 708. Although this disclosure describes and exemplifies specific I / O interfaces, this disclosure considers any suitable I / O interface.

[0088] In specific embodiments, communication interface 710 includes hardware, software, or both of providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 700 and one or more other computer systems 700 or one or more networks. By way of example and not limitation, communication interface 710 may include a network interface controller (NIC) or network adapter for communicating with Ethernet or other wired-based networks, or a wireless NIC (WNIC) or wireless adapter for communicating with wireless networks (such as Wi-Fi networks). This disclosure contemplates any suitable network and any suitable communication interface 710 for it. By way of example and not limitation, computer system 700 may communicate with one or more portions of an ad hoc network, personal area network (PAN), local area network (LAN), wide area network (WAN), metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an embodiment, computer system 700 may communicate with wireless PAN (WPAN) (such as, for example, Bluetooth WPAN), Wi-Fi network, Wi-Fi Max network, cellular telephone network (such as, for example, GSM network, LTE network, or 5G network), or other suitable wireless networks, or combinations of two or more of these. Where appropriate, computer system 700 may include any suitable communication interface 710 for any of these networks. Where appropriate, communication interface 710 may include one or more communication interfaces 710. Although specific communication interfaces are described and illustrated in this disclosure, any suitable communication interface is contemplated in this disclosure.

[0089] In a specific implementation, bus 712 includes hardware, software, or both, that couple components of computer system 700 to each other. By way of example and not limitation, bus 712 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 712 may include one or more buses 712. While this disclosure describes and exemplifies specific buses, this disclosure contemplates any suitable bus or interconnect.

[0090] In this document, where appropriate, a computer-readable non-transitory storage medium may include one or more semiconductor-based or other integrated circuits (ICs) (such as, for example, field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tape, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage medium, or any suitable combination of two or more of these. Where appropriate, a computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0091] Those skilled in the art will readily understand that the advantages and objectives described above would be impossible without the assembly in the system of the present invention and the specific combination of the computer hardware described herein with other structural components and mechanisms. Furthermore, the algorithms, methods, and processes disclosed herein improve upon any general-purpose computer or processor disclosed in this specification and accompanying drawings, converting it into a special-purpose computer programmed to execute the disclosed algorithms, methods, and processes to achieve the foregoing functions, advantages, and objectives. It should also be understood that those skilled in the art know that various programming tools can be used to generate and implement the features and operations described above. Moreover, the specific choice of programming tools may depend on the specific objectives and constraints imposed on the implementation chosen to achieve the concepts set forth herein and in the appended claims.

[0092] The descriptions in this patent document should not be construed as implying that any particular element, step, or function may be a necessary or critical element that must be included within the scope of the claims. Furthermore, unless the exact words “means for” or “step for” are explicitly used in a particular claim, followed by a participle phrase identifying the function, no claim is intended to invoke 35 USC § 112(f) with respect to any appended claim or claim element. The use of terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “building block,” “device,” “machine,” “system,” “processor,” “processing device,” or “controller” within the claims may be understood and intended to refer to structures known to a person skilled in the art as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 USC § 112(f). For example, the terms “processor” and “controller” may be a class of structures rather than a specific structure, and may be defined in functional terms, but this does not make them means-plus-function. In view of this paragraph of the specification, even under the broadest reasonable interpretation, in the absence of the specific language described above, the claims are not intended to invoke 35 USC § 112(f).

[0093] This disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. For example, each new structure described herein may be modified to accommodate specific local changes or requirements while retaining its basic configuration or structural relationships with each other, or performing the same or similar functions described herein. Therefore, this embodiment should be considered illustrative rather than restrictive in all respects. Consequently, the scope of this disclosure may be established by the appended claims rather than by the foregoing description. Therefore, all changes falling within the meaning and equivalent scope of the claims are intended to be included therein. Furthermore, the elements of the claims are not to be fully understood, conventional, or routine. Rather, the claims target unconventional inventive concepts described in the specification.

[0094] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally according to their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure. Those skilled in the art will also readily recognize that the order or combination of components, methods, or interactions described herein are merely embodiments, and that components, methods, or interactions of various embodiments of this disclosure may be combined or performed in ways different from those illustrated and described herein.

[0095] The functional blocks and modules disclosed herein may include processors, electronic devices, hardware devices, electronic components, logic circuits, memories, software code, firmware code, and any combination thereof. Consistent with the foregoing, the various exemplary logic blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration.

[0096] The steps of the methods or algorithms described in conjunction with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor, allowing the processor to read information from and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal, base station, sensor, or any other communication device. Alternatively, the processor and storage medium may reside as discrete components in the user terminal.

[0097] In one or more exemplary designs, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored on or transmitted through a computer-readable medium as one or more instructions or code. A computer-readable medium includes both computer storage media and communication media, with communication media including any medium that facilitates the transfer of a computer program from one place to another. A computer-readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. By way of example and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. Furthermore, a connection can be appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL), then that coaxial cable, fiber optic cable, twisted pair, or DSL is included in the definition of a medium. The terms "disk" and "disc" can include compact optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while discs typically reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0098] Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions, and modifications may be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Furthermore, the scope of this application is not intended to be limited to the specific embodiments of the processes, machines, manufactures, compositions of matter, apparatuses, methods, and steps described in the specification. As will be readily understood by those skilled in the art from the disclosure of this publication, processes, machines, manufactures, compositions of matter, apparatuses, methods, or steps that currently exist or will be developed thereafter perform substantially the same function, in substantially the same manner, or achieve substantially the same effect as the corresponding embodiments described herein can be utilized based on the present disclosure. Therefore, the appended claims are intended to include such processes, machines, manufactures, compositions of matter, apparatuses, methods, or steps within their scope.

Claims

1. A system for visualizing inventory in a multimodal container yard, the system comprising: An inventory database that stores unit inventory, which includes information about multiple transport containers physically located in the multimodal container yard; One or more memory units; as well as One or more computer processors, communicatively coupled to the inventory database and the one or more memory units, and configured to: Access the unit inventory stored in the inventory database; One or more instructions are provided to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard; Based on the information about the multiple transport containers in the unit inventory, the three-dimensional representation of the multimodal container yard is updated in real time, wherein updating the three-dimensional representation of the multimodal container yard includes: The three-dimensional representation of the multimodal container yard provides a visual representation of each transport container physically located within the multimodal container yard, wherein the visual representation of each specific transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the specific transport container in the multimodal container yard; and The three-dimensional representation of the multimodal container yard provides a visual representation of the current position of vehicles traveling within the multimodal container yard; and One or more instructions are provided to display metadata about a particular shipping container in the graphical user interface in response to a user's selection of a particular visual representation of a particular shipping container.

2. The system for visualizing inventory in a multimodal container yard according to claim 1, wherein the metadata regarding the specific transport container includes one or more of the following: Indication of the specific physical location of the particular transport container; The current image of the specific physical parking location; The current carrier identifier for the specific transport container; The current identification number of the specific transport container; The timestamp of the current image.

3. The system for visualizing inventory in a multimodal container yard according to claim 2, wherein the metadata regarding the specific transport container further includes one or more of the following: Previous images of the specific physical parking location; The previous carrier identification of the specific transport container; The previous identification number of the specific transport container; and The timestamp of the previous image.

4. The system for visualizing inventory in a multimodal container yard according to claim 2, wherein the indication of the specific physical location of the specific transport container includes: District signage; Row markings; as well as Bit identifier.

5. The system for visualizing inventory in a multimodal container yard according to claim 2, wherein the metadata about the particular transport container further includes a confidence value associated with the current carrier identifier and the current identification number of the particular transport container.

6. The system for visualizing inventory in a multimodal container yard according to claim 1, wherein the visual representation of each transport container physically located in the multimodal container yard is based on the carrier identifier of the particular transport container.

7. The system for visualizing inventory in a multimodal container yard according to claim 1, wherein the vehicles comprise: Container transport vehicles; Aircraft; car; truck; Golf cart; All-terrain vehicles (ATVs); Autonomous vehicles; motorcycle; or Remote-controlled vehicles.

8. A method for visualizing inventory in a multimodal container yard, executed by a computing system, the method comprising: Access the unit inventory stored in the inventory database, the unit inventory including information about multiple transport containers physically located in the multimodal container yard; One or more instructions are provided to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard; Based on the information about the multiple transport containers in the unit inventory, the three-dimensional representation of the multimodal container yard is updated, wherein updating the three-dimensional representation of the multimodal container yard includes: The three-dimensional representation of the multimodal container yard provides a visual representation of each transport container physically located within the multimodal container yard, wherein the visual representation of each specific transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the specific transport container in the multimodal container yard; and The three-dimensional representation of the multimodal container yard provides a visual representation of the current position of vehicles traveling within the multimodal container yard; and One or more instructions are provided to display metadata about a particular shipping container in the graphical user interface in response to a user's selection of a particular visual representation of a particular shipping container.

9. The method for visualizing inventory in a multimodal container yard according to claim 8, wherein the metadata regarding the specific transport container includes one or more of the following: Indication of the specific physical location of the particular transport container; The current image of the specific physical parking location; The current carrier identifier for the specific transport container; The current identification number of the specific transport container; The timestamp of the current image.

10. The method for visualizing inventory in a multimodal container yard according to claim 9, wherein the metadata regarding the specific transport container further includes one or more of the following: Previous images of the specific physical parking location; The previous carrier identification of the specific transport container; The previous identification number of the specific transport container; and The timestamp of the previous image.

11. The method for visualizing inventory in a multimodal container yard according to claim 9, wherein the indication of the specific physical location of the particular transport container includes: District signage; Row markings; as well as Bit identifier.

12. The method for visualizing inventory in a multimodal container yard according to claim 9, wherein the metadata about the particular transport container further includes a confidence value associated with the current carrier identifier and the current identification number of the particular transport container.

13. The method for visualizing inventory in a multimodal container yard according to claim 8, wherein the visualization of each transport container physically located in the multimodal container yard is based on the carrier identifier of the particular transport container.

14. The method for visualizing inventory in a multimodal container yard according to claim 8, wherein the vehicles comprise: Container transport vehicles; Aircraft; car; truck; Golf cart; All-terrain vehicles (ATVs); Autonomous vehicles; motorcycle; or Remote-controlled vehicles.

15. One or more computer-readable non-transitory storage media comprising instructions that, when executed by a processor, cause the processor to perform operations including: Access the unit inventory stored in the inventory database, which includes information about multiple transport containers physically located in a multimodal container yard; One or more instructions are provided to display a graphical user interface on an electronic display of a user computing system, the graphical user interface including a three-dimensional representation of the multimodal container yard; Based on the information about the multiple transport containers in the unit inventory, the three-dimensional representation of the multimodal container yard is updated, wherein updating the three-dimensional representation of the multimodal container yard includes: The three-dimensional representation of the multimodal container yard provides a visual representation of each transport container physically located within the multimodal container yard, wherein the visual representation of each specific transport container is placed at a virtual parking location in the three-dimensional representation of the multimodal container yard, the virtual parking location corresponding to the physical parking location of the specific transport container in the multimodal container yard; and The three-dimensional representation of the multimodal container yard provides a visual representation of the current position of vehicles traveling within the multimodal container yard; and One or more instructions are provided to display metadata about a particular shipping container in the graphical user interface in response to a user's selection of a particular visual representation of a particular shipping container.

16. One or more computer-readable non-transitory storage media according to claim 15, wherein the metadata concerning the particular shipping container includes one or more of the following: Indication of the specific physical location of the particular transport container; The current image of the specific physical parking location; The current carrier identifier for the specific transport container; The current identification number of the specific transport container; The timestamp of the current image.

17. One or more computer-readable non-transitory storage media according to claim 16, wherein the metadata concerning the particular shipping container further comprises one or more of the following: Previous images of the specific physical parking location; The previous carrier identification of the specific transport container; The previous identification number of the specific transport container; and The timestamp of the previous image.

18. One or more computer-readable non-transitory storage media according to claim 16, wherein the indication of the particular physical parking location of the particular shipping container includes: District signage; Row markings; as well as Bit identifier.

19. One or more computer-readable non-transitory storage media according to claim 16, wherein the metadata regarding the particular shipping container further includes a confidence value associated with the current carrier identifier and the current identification number of the particular shipping container.

20. One or more computer-readable non-transitory storage media according to claim 15, wherein the visual representation of each transport container physically located in the multimodal container yard is based on the carrier identifier of the particular transport container.

Images