System for monitoring goods

Abstract

The invention relates to a system for monitoring goods. Systems and methods of tracking cargo. The tracking system includes a wireless system having a tag configured to connect to a cargo and transmit identification data, and a locator configured to connect to a vehicle and receive the identification data transmitted from the tag. The vision system includes a camera positioned in the vehicle and configured to capture an image of the cargo. The control unit includes processing circuitry configured to identify the cargo and track the location of the cargo based on signals transmitted from the tag and received by the locator, and track the location of the cargo based on images captured by the vision system.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Serial No. 63 / 684,939, filed August 20, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to the field of cargo handling, and more specifically to a system that uses both wireless and vision systems to identify cargo and monitor its movement. Background Technology

[0004] A wide variety of vehicles are used to transport goods. Examples include, but are not limited to, airplanes, ocean-going vessels, and trucks. The transportation process typically involves loading goods onto a vehicle, positioning the goods within the vehicle, transporting the goods from a first location to a second location, and then unloading the goods. It is necessary to identify and monitor the goods during transportation.

[0005] Existing systems offer various ways to identify cargo loaded onto vehicles. However, these systems are often inaccurate because they cannot identify cargo during handling, cannot accurately monitor its location during handling, and / or cannot determine its position on the vehicle. This can lead to cargo being misidentified and / or improperly loaded onto the vehicle. In some cases, such as with airplanes, improperly loaded cargo requires unloading before flight and then reloading it in the correct location to ensure proper weight balance.

[0006] Some existing systems require operators to visually inspect and identify goods. However, visual identification of goods has been found to be inaccurate because operators often cannot accurately identify goods or properly input goods identification tags into the monitoring software. Furthermore, this can be costly, as it requires one or more operators to identify the tags and input them into the monitoring software. This process can also be time-consuming, slowing down the loading process and potentially causing transportation delays. Summary of the Invention

[0007] One aspect relates to a method for tracking goods. This method includes: receiving wireless signals from tags on the goods; identifying the goods based on the wireless signals; determining the location of the goods based on the wireless signals; and capturing images of the goods over a period of time, and monitoring the location of the goods based on the images as the goods move through an area.

[0008] On the other hand, the method also includes attaching a tag to the cargo before receiving wireless signals from the tag on the cargo.

[0009] On the other hand, the method also includes capturing an image of the cargo after determining its location based on wireless signals.

[0010] In another aspect, the method also includes identifying goods based on identification data contained in the wireless signal.

[0011] In another aspect, the method also includes simultaneously monitoring the location of the cargo based on wireless signals and images.

[0012] In another aspect, the method further includes: determining, based on the image, that the goods have stopped moving within the area; and when the goods stop moving, determining the final position of the goods as the point where the goods are located.

[0013] In another aspect, the method also includes: identifying the cargo and determining its location when the cargo is loaded onto the aircraft.

[0014] In another aspect, the method further includes receiving wireless signals at multiple locators and determining the location of the goods based on the signal strength of the wireless signals received at the multiple locators.

[0015] One aspect relates to a method for tracking goods loaded onto a vehicle. The method includes: receiving wireless signals at one or more of a plurality of locators, wherein the wireless signals are transmitted from a tag attached to the goods; identifying the goods based on the wireless signals; receiving the wireless signals at the plurality of locators and tracking the position of the goods based on the wireless signals as the goods move within the vehicle; capturing images of the goods as they move within the vehicle; and tracking the position of the goods within the vehicle based on the images.

[0016] In another aspect, the method further includes receiving wireless signals at multiple locators and determining the location of the goods based on the signal strength of the wireless signals received at the multiple locators.

[0017] On the other hand, the method also includes receiving wireless signals at multiple locators installed at fixed locations on the vehicle.

[0018] On the other hand, receiving wireless signals at one or more of the multiple locators includes receiving Bluetooth Low Energy signals transmitted from a BLE tag attached to the cargo.

[0019] In another aspect, the method further includes: identifying points on the cargo based on the image, and tracking the position of the cargo based on the points identified in the image.

[0020] In another aspect, the method also includes: identifying the leading edge of the cargo based on an image; and tracking the leading edge of the cargo as it moves within the vehicle.

[0021] One aspect relates to a cargo tracking system, including a wireless system comprising: tags configured to connect to cargo and transmit identification data; and locators configured to connect to a vehicle and receive identification data transmitted from the tags. A vision system includes multiple cameras positioned within the vehicle and configured to capture images of the cargo. A control unit includes processing circuitry configured to: identify the cargo and track its position based on signals transmitted from the tags and received by the locators; and track the cargo's position based on images captured by the vision system.

[0022] On the other hand, the wireless system is Bluetooth Low Energy.

[0023] On the other hand, the control unit is configured to initially identify the cargo and its location based on identification data received by one or more locators, and then track the cargo's location based on images captured by a vision system.

[0024] The features, functions, and advantages already discussed can be realized independently in each aspect or in combination in other aspects, as can be seen in further details with reference to the description and figures below. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a cargo tracking system that includes wireless and vision systems.

[0026] Figure 2 It is an isometric view of an aircraft equipped with a cargo tracking system.

[0027] Figure 3 It is an isometric view of cargo loaded into the cargo hold of a vehicle through an opening.

[0028] Figure 4 It is a schematic diagram including the alignment area and the cargo hold along the driveway extending along that length.

[0029] Figure 5 It is a schematic diagram of a cargo hold that includes a passageway extending along its length.

[0030] Figure 6 This is a schematic diagram of a control unit that receives signals from locators and tags and is configured to output calculated information.

[0031] Figure 7 This is a flowchart of a method for identifying and tracking goods.

[0032] Figure 8 This is a flowchart of a method for identifying and tracking goods.

[0033] Figure 9 This is a flowchart of a method for identifying and tracking goods.

[0034] Figure 10 This is a schematic diagram of the control unit. Detailed Implementation

[0035] This disclosure relates to a cargo tracking system that identifies and tracks the location of cargo, including determining the final location of the cargo on a vehicle. In some embodiments, the final location may be a location within the cargo hold of the vehicle or a location within a warehouse. Figure 1 As shown, the cargo tracking system 15 includes a wireless system 20 and a vision system 40. The wireless system 20 is configured to identify cargo 90 based on a tag 21, which is configured to be attached to cargo 90. The tag 21 also enables the wireless system 20 to track the location of cargo 90. The vision system 40 visually tracks the location of cargo 90, such as during handling during loading and unloading. The vision system 40 determines the precise location of cargo 90, such as its location within a cargo hold during transport. The control unit 50 identifies cargo 90 and determines its location based on identification data and / or images. In some embodiments, the wireless system 20 identifies cargo 90 and determines its approximate location. Once the approximate location is known, the vision system 40 tracks the location of cargo 90 more precisely. The wireless system 20 and the vision system 40 can operate continuously and / or in parallel.

[0036] Figure 2 An application of a cargo tracking system 15 for goods transported by a vehicle 100 is illustrated. In this embodiment, the vehicle 100 is an aircraft configured to transport cargo 90. The vehicle 100 typically includes a fuselage 101 having one or more doors 102 leading to a cargo hold 103 inside the fuselage 101. The cargo hold 103 includes a floor, ceiling, and sidewalls and is configured to accommodate cargo 90 during transport. The cargo tracking system 15 is integrated with the vehicle 100 and is configured to identify and track the location of cargo 90 on the vehicle 100.

[0037] Figure 3 Cargo 90 is shown positioned on platform 110 for loading into vehicle 100. Door 102 in vehicle fuselage 101 is in the open position to allow cargo 90 to move through opening 104 and into cargo hold 103. Cargo 90 can be unloaded in a similar manner, wherein cargo 90 is removed from cargo hold 103 through opening 104 and loaded onto platform 110.

[0038] Cargo 90 may have various shapes and sizes. In one embodiment, cargo 90 includes a unit loading device (ULD). The ULD may include different configurations, including but not limited to pallets supporting smaller packages and containers for holding contents on wide-body and certain narrow-body aircraft, and its shape and size are formed to conform to the dimensions of cargo hold 103. In another embodiment, cargo 90 includes smaller containers (e.g., boxes, crates) positioned on pallets and held together with packaging materials (e.g., mesh, plastic packaging materials). Figure 3 An embodiment is shown having a label 21 attached to a pallet and one or more individual packages located on the pallet.

[0039] During loading, the cargo moves through door 102 and into cargo hold 103. Figure 4 The diagram shows an opening 104 in the cargo hold 103 and fuselage 101, through which cargo 90 is loaded onto the vehicle 100. The cargo hold 103 includes an alignment area 106 extending inward from the opening 104. The alignment area 106 is used to align the cargo 90 along a passageway 105 that extends along the length of the cargo hold 103. During loading, the cargo 90 moves into the alignment area 106, aligns with one of the passageways 105, and then moves downward along the selected passageway 105. The passageway 105 is divided along its length into compartments 109, each compartment 109 being sized to accommodate one or more pieces of cargo 90. The cargo 90 moves along the passageway 105 to one of the compartments 109. In some embodiments, this includes moving the cargo 90 along the passageway 105 until it reaches the end of the passageway 105, or to other cargo 90 already loaded in the passageway 105. Figure 4 This includes an embodiment where cargo hold 103 includes six passageways 105, wherein Figure 5 An embodiment includes two passageways 105. The layout may include different numbers of passageways 105 extending along the cargo hold 103.

[0040] In some embodiments, cargo 90 has an allocation location within cargo hold 103, such as in a specific compartment 109 of a specific passageway 105. In some embodiments, cargo 90 is loaded onto vehicle 100 according to a Loading Instruction Report (LIR). The LIR is used by the operator loading vehicle 100 and is used to set the location of cargo 90 on vehicle 100 to comply with instructions regarding weight and balance limits. In some embodiments, allocation locations are determined to distribute the weight of cargo 90. When vehicle 100 is an aircraft, weight distribution is important for balancing the aircraft to ensure safe flight. In another embodiment where vehicle 100 is an ocean-going vessel, weight distribution prevents vehicle 100 from remaining stable in the water and reduces the risk of uncontrollable capsizing or swaying. Allocation locations also facilitate loading and unloading of cargo 90, as the passageway arrangement of cargo hold 103 uses a Last-In-First-Out (FILO) loading command. Access to a specific cargo block 90 requires the movement of other cargo 90 positioned inward along the corresponding passageway 105 (i.e., positioned between the desired cargo 90 and opening 104). Allocation locations of cargo 90 are also important for monitoring exposure during transport. One or more environmental factors (e.g., temperature, humidity) are monitored in cargo hold 103 and can be used to determine the exposure of cargo 90.

[0041] The wireless system 20 includes an identification tag 21 and a locator 25. The tag 21 transmits unique identification data configured to be picked up by the locator 25. In some embodiments, the tag 21 is battery powered to transmit identification information detected by the locator 25.

[0042] Tag 21 is configured to be attached to cargo 90, with each cargo 90 having a separate tag 21. Tag 21 can be attached to cargo 90 in various ways, including but not limited to one or more fasteners, adhesives, and wires. In one embodiment, cargo 90 is equipped with a receiver for receiving tag 21. Tag 21 includes identification data identifying cargo 90. In some embodiments, the data includes an alphanumeric code (such as a serial number) identifying cargo 90. In addition, or as an alternative, data 90 includes other identification information and / or information, including but not limited to a written description, cargo owner, destination, and cargo identification (e.g., cargo ID code).

[0043] The locator 25 is configured to receive identification data from the tag 21. The locator 25 is positioned near the cargo 90 such that the identification data can be read as the cargo 90 passes by the locator 25. In some embodiments, the locator 25 is configured to be mounted to the vehicle 100. Alternatively, the locator 25 is configured to be mounted near the vehicle 100, for example, on a shelf or support, for positioning near the opening 104 for loading and unloading the cargo 90.

[0044] In such Figure 3 and Figure 4 In some embodiments shown, one or more locators 25 are positioned at the opening 104 of the cargo hold 103. This positioning enables the wireless system 20 to identify and locate the cargo 90 before and / or while it is being loaded onto the vehicle 100. The locators 25 are also positioned along the cargo hold 103. Figure 4 and Figure 5 As shown, the positioners 25 are spaced apart along the length of the passage 105.

[0045] The control unit 50 receives data from the locator 25 and identifies and determines the location of the cargo 90. The control unit 50 can be located in various locations, including in the cargo hold 103, inside the vehicle 100, and at a remote location outside the vehicle 100. In some embodiments, the control unit 50 is dedicated to the cargo tracking system 15. In other embodiments, the control unit 50 is a component of another data processing system of the vehicle 100.

[0046] Cargo tracking system 15 can use different technologies to identify and track cargo 90. In some embodiments, cargo tracking system 15 uses Bluetooth Low Energy (BLE). Tag 21 is a hardware transmitter that broadcasts identification data. Locator 25 is configured to receive identification data from tag 21. In some embodiments, multiple locators 25 are located on vehicle 100 and arranged to receive identification data. In the case where multiple locators 25 are located in cargo hold 103, trilateration or polygonal positioning is used to determine the location of cargo 90. Locator 25 receives signals from tag 21 and determines a Received Signal Strength Indicator (RSSI). RSSI is determined based on the known signal strength at a known distance and the signal strength of the received signal from tag 21. RSSI is transmitted to control unit 50, which uses strength values ​​from multiple different locators 25 (e.g., trilateration for three different locators; multi-point positioning for four or more locators) to determine the location of tag 21.

[0047] In other embodiments, the control unit 50 uses the Stigmergic method, which uses an intensity map to estimate the location of the tag 21.

[0048] The location can be identified and tracked by the wireless system 20 using other network protocols. Examples of wireless network protocols include, but are not limited to, ZigBee and Wi-Fi. Each of these protocols enables communication between the tag 21 and the locator 25 to transmit identification data. The location of the goods 90 is determined using calculations of signal strength. Other embodiments use an RFID wireless system, in which the locator 25 transmits radio waves and receives signals returned from the RFID tag 21.

[0049] The cargo tracking system 15 includes a vision system 40 for tracking the position of cargo 90. The vision system 40 includes an electro-optical sensor 41 positioned on the vehicle 100 to capture images of cargo 90. The following disclosure will include the electro-optical sensor 41 as a camera, but other types of electro-optical sensors may also be used to capture images of cargo 90.

[0050] Camera 41 is configured to capture individual discrete images and / or video of cargo 90. Camera 41 is configured to capture two-dimensional and / or three-dimensional images. In some embodiments, camera 41 includes a fixed field of view. This provides a sequence of images to be captured, which includes cargo 90 moving across the field of view. For example, a first image in the sequence captures cargo 90 on a first side of the image, a second image captures cargo 90 at the center of the image, and a third image captures cargo 90 on a opposite second side of the field of view.

[0051] In some embodiments, each of the cameras 41 has an independent field of view different from any other camera 41. In other embodiments, the cameras 41 are arranged to have overlapping fields of view. This facilitates tracking the movement of cargo 90 as it moves within cargo hold 103 and through the different fields of view of the different cameras 41.

[0052] Camera 41 is mounted to vehicle 100 in various known locations, including on one or more doors 102, on the fuselage wall at opening 104, and inside cargo hold 103. In some embodiments, camera 41 is positioned in an elevated position, particularly inside cargo hold 103, to prevent and / or reduce obstruction of camera 41 by cargo 90. Specific embodiments include positioning camera 41 in the ceiling of cargo hold 103 or along a side wall spaced upwards from the floor (e.g., 75 inches above the floor).

[0053] Figure 5 A network of cameras 41 positioned between the front end 107 and the rear end 108 of cargo hold 103 is shown. In this embodiment, six cameras 41 are positioned along the length of cargo hold 103. The cameras 41 are spaced apart along passageway 105 and located in different compartments 109 along passageway 105. In some embodiments, the cameras 41 are spaced apart along passageway 105, with a maximum gap between the cameras 41 of approximately five compartments 109. This allows different cameras 41 to capture images in the same area even if one or more cameras 41 are obstructed by cargo 90.

[0054] Camera 41 can face in different directions to cover cargo compartment 103. In some embodiments, camera 41, spaced apart from front end 107, faces forward. In another embodiment, camera 41, spaced apart from rear end 108, faces rearward.

[0055] In some embodiments, camera 41 flows at a minimum frame rate of 2 Hz or higher. This speed ensures that transition events of cargo 90 are not missed in the images. In other embodiments, when using a Real-Time Streaming Protocol (RTSP) stream to simulate a live streaming scene, the rate is one image per second. This setup does not have any significant frame latency between different camera streams and therefore has a lower chance of missing transition events of cargo 90.

[0056] Control unit 50 uses images to track the position of cargo 90. In some embodiments, control unit 50 determines a specific point on cargo 90 for tracking position. This point allows cargo 90 to be tracked in different images, such as when cargo 90 moves through the field of view of multiple cameras 41. For example, control unit 50 determines and tracks the position of the centroid of cargo 90. Centroid tracking can be performed using different methods, including but not limited to K-means clustering. Other embodiments include selecting different points on cargo 90, such as, but not limited to, points on corners, points on top edges, and points on bottom edges (e.g., the center of the bottom edge).

[0057] In some embodiments, the control unit 50 uses background subtraction to determine the position of the cargo 90. One process includes static background subtraction that uses a background image as a reference to detect changes in pixel values. Another process uses dynamic background subtraction, which uses a dynamically selected background image as a reference and compares the current image against changes in pixel values ​​based on that reference.

[0058] One embodiment of tracking the position of cargo 90 includes constructing a motion detector using dynamic background subtraction, performing region filtering on small moving objects (e.g., people), and then tracking the remaining pixels on which motion was detected. When the sum of the detected pixels is large enough, the output of the background subtraction is used to determine the position of cargo 90 to cluster the detections into the object. A K-means clustering algorithm is then used to find the centroid of cargo 90. When the centroid enters a specific region of interest associated with the cargo position, then cargo 90 is marked as being at that position from the perspective of a camera observing cargo 90 at that position. In some embodiments, the process includes a camera voting system in which images from multiple cameras 41 are analyzed to determine whether they include cargo 90 at a particular position. Typically, cameras 41 in the relative path 105 of the moving cargo 90 have a better field of view of the cargo.

[0059] Another embodiment of tracking cargo 90 includes leading edge detection. The method uses a combination of static and dynamic computer vision techniques to determine the effective area of ​​cargo hold 103 from a given video feed. In some embodiments, the method uses a bird's-eye view transformation to convert a 3D image into a 2D top view. The method uses both static and dynamic methods, and then determines between one of these methods. The static method uses subtraction from a static background image to determine ground displacement. If the amount of pixel difference exceeds a threshold, the method assumes the presence of cargo 90. The dynamic method uses dynamic background subtraction to extract only the moving portion of the video and performs Canny edge detection logic to determine the contour of the moving object. The method then detects the presence of cargo 90 by searching for the leading edge of the cargo within the contour. The method also determines between static and dynamic methods, and when a leading edge is detected in the region within a short timeframe before static background subtraction, the presence of cargo is determined, and the object is also marked as present. If static background subtraction marked no object presence when no leading edge was detected within a short window previously, the algorithm also clears the previous detection. In some embodiments, heuristic / bounce logic is applied to handle hysteresis conditions.

[0060] Another method for tracking cargo 90 is static background subtraction in the Hue, Saturation, and Value (HSV) space. This method uses static computer vision techniques to determine the effective area of ​​cargo hold 103 from a pre-recorded video feed. The method uses a bird's-eye view transformation to convert the 3D image into a 2D top-down view. This has several advantages, including simplifying the selection of the area of ​​interest and enabling the application of area-of-interest cropping. In some embodiments, this method uses static background subtraction to determine ground shift. The method operates in the HSV color space to handle differences in illumination intensity. The static area of ​​interest is mapped onto the floor, and the number of pixels that have changed compared to a reference background image is counted to determine if an object is occupying an area. Objects too small to be cargo are filtered out. A kernel erosion technique is used to reduce the noise floor of the static background subtraction to account for smaller pixel-level differences caused by vibration and resulting changes in light reflection. Furthermore, the arbitrator logic is updated to track state transitions to accurately infer the cargo position in the area when the cargo is not directly in the camera's field of view. Each state is recorded in JSON, and camera and locator data are fused together.

[0061] Figure 6 A schematic diagram of a cargo tracking system 15 is shown, which includes a control unit 50 that receives data from a locator 25 and a camera 41. Different components may communicate individually with the control unit 50 or via one or more data buses 59. The locator 25 and camera 41 may be powered in various ways, including but not limited to Power over Ethernet, battery power, and various other wireless and wired configurations.

[0062] In some embodiments, the cargo tracking system 15 is integrated with the vehicle 100. The control unit 50 is either a standalone device solely for monitoring the cargo 90 or may be part of another system on the vehicle 100, such as a flight control computer that monitors the operation of the vehicle 100. In some embodiments, the control unit 50 is located remotely from the vehicle 100. One embodiment includes the control unit 50 as a remote server that receives signal information from the locator 25 and receives images from the camera 41 and processes visual data.

[0063] Control unit 50 is also configured to transmit cargo information to remote node 99. Remote node 99 is located on the ground or in an aircraft and is concerned with cargo 90. Embodiments include, but are not limited to, an airline operating vehicle 100, a transportation company responsible for transporting cargo 90, and the owner of cargo 90. In some embodiments, control unit 50 maintains a record 70 of the location of cargo 90 within vehicle 100. Record 70 includes passageway 105 and cargo hold 109, in which cargo 90 is located. In some embodiments, the entire record 70 is transmitted to remote node 99. In other embodiments, discrete information from record 70 is transmitted to remote node 99. Communication with remote node 99 can be directed from control unit 50 or via a communication system on vehicle 100. Communication can be via various networks, including, but not limited to, packet data networks such as public networks (e.g., the Internet) or private networks, and mobile communication networks (e.g., WCDMA, LTE, or WiMAX networks).

[0064] The cargo tracking system 15 uses both the wireless system 20 and the vision system 40 to identify the cargo 90 during loading and / or unloading. Figure 7 A method for identifying and tracking cargo 90 is illustrated. Cargo 90 is initially identified via wireless system 20 (box 200). Identification is based on identification data transmitted from tag 21 and received by one or more locators 25. The location of cargo 90 is also tracked via wireless system 20 (box 202). Location tracking is performed again using data received from tag 21, such as RSSI calculations of signals received from the multiple locators 25. Cargo tracking system 15 also provides vision system 40 with information to track the location of cargo 90 (box 204).

[0065] In some embodiments, vision system 40 provides more precise tracking than wireless system 20, which is capable of identification and coarse positioning. With cargo 90 identified and its coarse location known, more precise movement and positioning of cargo 90 can be achieved by vision system 40. In some embodiments, vision system 40 is used to determine the final position within cargo hold 103. The final position is determined as the location of cargo 90 when it is determined that cargo 90 has stopped moving.

[0066] In some embodiments, both the wireless system 20 and the vision system 40 simultaneously track the cargo 90. The wireless system 20 is initially able to identify the cargo 90 and determine its relatively coarse location. The control unit 50 is then able to analyze images received from one or more cameras 41 to track further movement of the cargo 90.

[0067] Figure 8 A method is illustrated for simultaneously tracking the position of cargo 90 using both a wireless system 20 and a vision system 40. The process begins with cargo 90 identified based on data transmitted from tag 21 (box 210) (box 209). The position of cargo 90 is also determined based on the position of tag 21 received by one or more locators 25 (box 212). The vision system 40 and wireless system 20 simultaneously track cargo 90. Cargo is detected in an image (box 220), and its position is determined based on the image (box 222). In some embodiments, control unit 50 initially determines the position of cargo 90 via wireless system 20. Once the position is roughly known, control unit 50 analyzes an image from camera 41 with a field of view including the approximate position. Control unit 50 is then able to determine the position of cargo 90 more specifically based on the image. In some embodiments, the image is analyzed only after cargo 90 has been identified via wireless system 20. In other embodiments, the image is analyzed prior to identification via wireless system 20.

[0068] The method continues to locate cargo 90 based on the position determined from both wireless system 20 and vision system 40 (box 225). In some embodiments, if one of systems 20 or 40 fails to determine the position, the position is monitored by both. This can occur in various situations, such as, but not limited to, when locator 25 fails to receive a signal from tag 21 and the image fails to include cargo 90, such as when one or more cameras 41 are obstructed. In some embodiments, the position of cargo is determined based on vision system 40 (box 225) because it is generally a more accurate system. In some embodiments, the position of cargo 90 is based on a combination of positions determined by both systems 20 and 40 (e.g., average position).

[0069] In some embodiments, the wireless system 20 and the vision system 40 operate continuously. Figure 9 One method involves using a wireless system 20 to identify cargo 90 (box 210). The wireless system 20 is also used to determine the position of cargo 90. Once cargo 90 is identified and initially located, a vision system 40 is used to track the position of cargo 90 (box 212). The initial position determined by the wireless system 20 enables the control unit 50 to locate cargo 90 in an image. The control unit 50 then uses only the vision system 40 to track the position.

[0070] Figure 10 An embodiment of a control unit 50 is shown, comprising processing circuitry 51, memory circuitry 52, camera interface circuitry 53, locator interface circuitry 58, and communication circuitry 54. Processing circuitry 51 controls the overall operation of the cargo tracking system 15 according to program instructions stored in memory circuitry 52. ​​Processing circuitry 51 may include one or more circuits, a microcontroller, a microprocessor, hardware, or a combination thereof. Processing circuitry 51 may include varying amounts of computing power to provide the required functionality.

[0071] Memory circuitry 52 includes a non-transitory computer-readable storage medium storing program instructions, such as a computer program product, which configures processing circuitry 51 to implement one or more of the techniques discussed herein. Memory circuitry 52 may include various memory devices, such as, for example, read-only memory and flash memory. Memory circuitry 52 may be as follows: Figure 10 The memory circuit 52 can be a separate component shown, or it can be combined with the processing circuitry 51. Alternatively, the processing circuitry 51 may omit the memory circuitry 52, for example, according to at least some embodiments in which the processing circuitry 51 is dedicated and non-programmable. The memory circuitry 52 is configured to support loading images into runtime memory for real-time processing and storage. In one embodiment, the memory circuitry 52 includes a solid-state device (SSD).

[0072] Control unit 50 includes a graphics processing unit (GPU) 55. GPU 55 is a dedicated electronic circuit designed to manipulate and modify memory circuitry 52 to accelerate the creation of images in a frame buffer intended for output. GPU 55 may contain varying amounts of computing power to provide the required functionality. In one embodiment, GPU 55 has quaternary flip-flops with more than one computing power. This processing power provides large-scale machine learning. In one embodiment, computing device 50 includes a separate GPU 55. In another embodiment, the processing is performed at processing circuitry 51.

[0073] The memory circuit 52 is configured to store records 70 of the cargo 90. Records 70 include identification data on the cargo 90, including but not limited to identification alphanumeric codes, name, owner, volume, contents, origin, destination, and loading position on the vehicle 90.

[0074] Camera interface circuit 53 provides image reception from camera 41. Camera interface circuit 53 can provide one-way communication from camera 41 or two-way communication to and from camera 41. Positioner interface circuit 58 provides identification data reception from positioner 25. Positioner interface circuit 58 can be configured for one-way or two-way communication.

[0075] Communication circuit 54 provides communication to and from control unit 50. Communication may include communication with other circuitry on vehicle 100 (e.g., vehicle control system) and / or with remote node 99. Communication circuit 54 provides data transmission and reception with remote node 99.

[0076] User interface 60 provides users with access to data about cargo 90. User interface 60 includes one or more input devices 62, such as, but not limited to, a keyboard, touchpad, scroll ball, and joystick. User interface 60 also includes one or more displays 61 for displaying information about cargo 90 and / or for the operator to input commands to processing circuitry 51.

[0077] In some embodiments, the control unit 50 operates autonomously to process identification data and images. This autonomy minimizes and / or eliminates operator intervention, which could slow down the process and / or introduce errors.

[0078] The cargo tracking system 15 can be used on various vehicles 100, including but not limited to trucks, trains, ships, and airplanes. The cargo tracking system 15 can also be used in other environments. Examples include, but are not limited to, warehouses, airport loading facilities, and distribution centers.

[0079] In some embodiments, the image includes a timestamp indicating the time when the image was captured. The timestamp may be applied by the camera 41 or the control unit 50. The control unit 50 may use the timestamp to track the movement of the cargo 90 and the different images captured by the camera 41.

[0080] In some embodiments, a heat map is used to track the location of cargo 90. This functionality includes a tag 21 attached to cargo 90, configured to emit signals. These signals are received by one or more locators 25 to form a heat map of the general location of cargo 90. The heat map is used to narrow down the location of cargo 90. Once the general location is known from the heat map, one or more other systems (e.g., vision system 40, perception, hearing) are used to determine a more specific location.

[0081] Furthermore, information regarding the cargo tracking system 15 is disclosed in embodiment A.

[0082] Regarding quantities or measurements, the term "substantially" means that the feature, parameter, or value does not need to be precisely achieved. Instead, deviations or variations (including, for example, tolerances, measurement errors, measurement accuracy limitations, and other factors known to those skilled in the art) may occur in quantities that do not preclude the effects that the feature is intended to provide.

[0083] The invention may be practiced in ways other than those specifically set forth herein without departing from its essential characteristics. This embodiment is to be considered illustrative rather than restrictive in all respects, and all variations falling within the meaning and equivalents of the appended claims are intended to be included therein.

Claims

1. A method for tracking cargo (90), the method comprising: Receive wireless signals from the tag (21) on the cargo (90); The goods (90) are identified based on the wireless signal. The location of the cargo (90) is determined based on the wireless signal; and Capture images of the cargo (90) over a period of time, and monitor the position of the cargo (90) based on the images as the cargo (90) moves through an area.

2. The method according to claim 1, further comprising: The tag (21) is attached to the cargo (90) before receiving the wireless signal from the tag (21) on the cargo (90).

3. The method according to claim 1, further comprising: After determining the location of the cargo (90) based on the wireless signal, an image of the cargo (90) is captured.

4. The method according to claim 1, further comprising: The cargo (90) is identified based on the identification data contained in the wireless signal.

5. The method according to claim 1, further comprising: The location of the cargo (90) is monitored simultaneously based on the wireless signal and the image.

6. The method according to claim 1, further comprising: Based on the image, it was determined that the cargo (90) had stopped moving within the area; and When the cargo (90) stops moving, the final position of the cargo (90) is determined as the point where the cargo (90) is located.

7. The method according to claim 1, further comprising: When the cargo (90) is loaded onto the aircraft, the cargo (90) is identified and its location is determined.

8. The method according to claim 1, further comprising: The wireless signals are received at multiple locators (25), and the location of the cargo (90) is determined based on the signal strength of the wireless signals received at the multiple locators (25).

9. A method for tracking cargo (90) loaded onto a vehicle, the method comprising: Receives wireless signals at one or more of a plurality of locators (25), wherein the wireless signals are transmitted from a tag (21) attached to the cargo (90); The goods (90) are identified based on the wireless signal. When the cargo (90) moves in the vehicle, the wireless signal is received at a plurality of the locators (25) and the location of the cargo (90) is tracked based on the wireless signal; An image of the cargo (90) is captured while the cargo (90) is moving in the vehicle; and The location of the cargo (90) within the vehicle is tracked based on the image.

10. The method of claim 9, further comprising: The wireless signal is received at a plurality of the locators (25), and the location of the cargo (90) is determined based on the signal strength of the wireless signal received at the plurality of the locators (25).

11. The method of claim 9, further comprising: The wireless signal is received at a plurality of the locators (25) installed at fixed locations on the vehicle.

12. The method according to claim 9, wherein, Receiving wireless signals at one or more of the multiple locators (25) includes receiving Bluetooth Low Energy signals transmitted from a BLE tag (21) attached to the cargo (90).

13. The method of claim 9, further comprising: The points on the cargo (90) are identified based on the image, and the position of the cargo (90) is tracked based on the points identified in the image.

14. The method of claim 9, further comprising: The leading edge of the cargo (90) is identified based on the image; and Track the leading edge of the cargo (90) as it moves within the vehicle.

15. A cargo tracking system, comprising: Wireless system (20), including: Tag (21) is configured to connect to cargo (90) and is configured to transmit identification data; and The locator (25) is configured to connect to the vehicle and to receive the identification data transmitted from the tag (21); The vision system (40) includes multiple cameras positioned in the vehicle and configured to capture images of the cargo (90); The control unit (50) includes a processing circuit (51) configured to: The cargo (90) is identified and its location is tracked based on signals transmitted from the tag (21) and received by the locator (25); and The location of the cargo (90) is tracked based on the images captured by the vision system (40).

16. The cargo tracking system according to claim 15, wherein, The wireless system (20) is a Bluetooth Low Energy system.

17. The cargo tracking system according to claim 15, wherein, The control unit (50) is configured to initially identify the cargo (90) and the location of the cargo (90) based on the identification data received by one or more of the locators (25), and then track the location of the cargo (90) based on the images captured by the vision system (40).

18. The cargo tracking system according to claim 15, wherein, The control unit (50) is configured to begin capturing images of the cargo (90) after the location of the cargo (90) is determined by the wireless system (20).

19. The cargo tracking system according to claim 15, wherein, The camera is located at a fixed position inside the vehicle.

20. The cargo tracking system according to claim 15, wherein, The wireless system (20) is configured to determine the location of the cargo (90) based on the signal strength of the wireless signals received at the plurality of locators (25).

Images