An intelligent warehouse goods tracking system and method based on RFID and visual identification fusion

Abstract

This invention discloses an intelligent warehouse cargo tracking system and method based on the fusion of RFID and visual recognition, belonging to the field of logistics warehousing and automation technology. The system includes an RFID sensing unit, a visual sensing unit, an edge computing unit, and a central management unit. The visual acquisition unit is synchronously triggered with the RFID reader to acquire cargo images and radio frequency signals; the edge computing terminal identifies cargo image features through deep learning algorithms and combines RFID signal strength and phase information, utilizing a multimodal data fusion algorithm to perform identity binding, location positioning, and status detection for the cargo. This invention effectively solves the tracking failure problems caused by dense stacking of warehouse goods, metal interference, and changes in lighting by integrating the non-contact batch identification capability of RFID with the high-precision environmental perception capability of visual recognition, significantly improving the accuracy and real-time performance of cargo inventory, sorting, and inbound / outbound operations.

Description

Technical Field

[0001] This invention relates to the field of logistics and warehousing automation technology, specifically to a system and method that combines radio frequency identification (RFID) technology with computer vision technology to achieve accurate tracking, positioning and management of goods in a warehouse. Background Technology

[0002] Cumbersome operation: It relies on manual scanning of each item or requires complex mechanical structures, which severely restricts the speed of inbound and outbound operations and inventory counting.

[0003] Fragile and easily soiled: Tags are easily contaminated, wrinkled, or damaged, leading to reading failures and extremely low reliability in complex warehouse environments. Pure RFID technology: Sensing "blind spots," signal interference: Easily blocked by substances such as metal and liquids, creating signal blind spots and missed reads. Lack of status: Only provides ID information and cannot sense the physical state of goods, such as whether the packaging is damaged, whether the goods are tipped over, or whether the tags are detached—critical statuses. Pure machine vision technology: A "prisoner" of the environment. Recognition bottleneck: When goods are densely stacked, have highly similar appearances, or are subject to drastic changes in lighting or partial occlusion, the recognition accuracy drops sharply.

[0004] High computing power requirements: Real-time processing of high-definition video streams requires huge computing resources, resulting in high system costs and difficulty in guaranteeing real-time performance in complex scenarios.

[0005] A single technological approach can no longer meet the stringent requirements of modern smart warehouses. Multimodal sensing fusion, with its complementary advantages, has become an inevitable trend and a core opportunity for industry development. Summary of the Invention

[0006] To solve the above problems, the present invention provides the following technical solution: an intelligent warehouse cargo tracking system based on the fusion of RFID and visual recognition, comprising: RFID sensing unit: consists of an array of RFID readers deployed in a predetermined area of ​​the warehouse, and electronic tags attached to goods or vehicles, wherein the electronic tags store the unique identification of the goods; Visual sensing unit: includes an industrial camera and supplementary lighting equipment that are spatially synchronized with the RFID reader array, used to acquire images or video streams of goods; Edge computing unit: connected to the RFID reader array and the visual acquisition module respectively, used to receive radio frequency signals and image data, and perform local data preprocessing and feature fusion; Central Management Unit: Communicatively connected to the edge computing terminal, used to store cargo lifecycle data and issue scheduling instructions based on the fusion results.

[0007] Preferably, the edge computing unit deploys a multimodal data fusion algorithm module, the module being configured as follows: Spatial mapping submodule: Based on preset calibration parameters, the RFID signal positioning area is mapped to the image coordinate system through AOA or RSSI differential calculation; Binding and verification submodule: Calculate the overlap rate between the visual target detection box and the RFID positioning area. If the overlap rate is greater than the threshold and the cargo category information is consistent, then bind the cargo ID and visual features. Dynamic weighting submodule: Adjusts the confidence weights of RFID and visual recognition in real time based on environmental parameters.

[0008] Furthermore, the RFID reader array and visual acquisition unit are integrated on the mobile terminal, and perform dynamic inspections as the mobile device moves.

[0009] Furthermore, the visual acquisition unit employs a depth camera, which can acquire the depth information of the cargo to calculate its volume and precise coordinates in three-dimensional space.

[0010] A smart warehouse cargo tracking method based on the fusion of RFID and visual recognition includes the following steps: S1: Synchronous acquisition When goods enter the monitoring area, the sensor is triggered to synchronize the RFID reader and industrial camera to capture radio frequency signals and image data; S2: Data preprocessing involves denoising and filtering RFID signals to select valid tag IDs; target detection and segmentation are performed on image data to extract cargo appearance features; S3: Coarse positioning and matching: Calculate the approximate area of ​​the tag using the angle of arrival (AOA) or signal strength (RSSI) difference of the RFID signal, and map this area to the region of interest (ROI) in the image; S4: Fusion verification within the ROI area compares whether the visually identified goods category / code is consistent with the goods category information stored in the RFID tag; S5: Status Update and Anomaly Handling. If the verification is consistent, update the system inventory database and goods location information; if the verification is inconsistent or only a single modality data is detected, mark it as an abnormal status and issue an alarm.

[0011] Furthermore, it also includes a confidence-based weighted decision-making mechanism: assigning different initial weights to visual recognition and RFID recognition, increasing the weight of visual recognition when there is sufficient light and the goods are unobstructed, and increasing the weight of RFID recognition when the goods are densely stacked or the light is insufficient. Beneficial effects

[0012] 1. High-precision positioning: Visual recognition is used to correct the positioning ambiguity of RFID, and RFID is used to assist visual recognition to distinguish goods with similar appearances, achieving centimeter-level positioning.

[0013] 2. Anomaly detection vision systems can detect external features that RFID cannot detect, such as damaged packaging, tipped-over goods, and detached tags.

[0014] 3. Strong anti-interference capability: When RFID signals are blocked, historical trajectory is extrapolated through visual features; when vision is limited, tracking continuity is maintained by relying on RFID. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention; Figure 2 A logical diagram of the RFID subsystem and the visual recognition subsystem; Figure 3 A flowchart of a multimodal data fusion algorithm in an edge computing terminal; Figure 4 This is a flowchart illustrating the logic of the cargo tracking and status update method. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 This invention discloses an intelligent warehouse cargo tracking system and method based on the fusion of RFID and visual recognition, belonging to the field of logistics warehousing and automation technology. The system includes: an RFID sensing unit, a visual sensing unit, an edge computing unit, and a central management unit.

[0018] like Figure 2 As shown, this embodiment constructs an intelligent warehouse cargo tracking system. "Gantry-type" data collection terminals are installed at the warehouse's inlet, outlet, and the top of each aisle. The left-hand RFID sensing subsystem comprises three sub-modules: an RFID antenna array, a multi-antenna layout, signal reception radio frequency, and signal capture; position calculation is based on signal strength and phase. The middle-hand visual sensing subsystem comprises three sub-modules: a multi-view camera gantry with multiple cameras on both sides and top; simultaneous image acquisition and synchronous image capture by the multi-view cameras; target detection and cargo tracking; and identification and cargo movement tracking.

[0019] On the right: The fusion decision center contains three core sub-modules: mapping the RFID positioning area with the visual image coordinates to determine whether the time and space are aligned, feature matching, comparing the consistency between the radio frequency ID and the visual features, and making a state judgment based on a confidence threshold.

[0020] To further clarify, the multi-view camera is a movable, multi-angle shooting camera, and is also an industrial-grade camera.

[0021] The output layer data stream extends from the fusion center to the WMS (Warehouse Management System) and AGV control module (Automated Guided Vehicle scheduling). When the WMS cannot synchronize with the cloud storage system to achieve data closure, the central management service area can make judgments based on the specific goods entry time to ensure system updates and feasibility. Each terminal includes: an RFID UHF reader and four antennas covering a rectangular area below the gantry for reading tags on pallets; a line-scan or area-scan camera installed at an angle covering the overlapping area of ​​the RFID antennas to ensure the camera's field of view covers all possible tag locations; supplementary LED lights to ensure image acquisition quality; and an edge computing box connected to the camera and reader via network cable for real-time algorithm execution. The gantry spans the conveyor belt and aisle. Four RFID antennas are installed at the four corners of the gantry to form a signal coverage area. An industrial camera is installed in the center to cover the field of view. AGVs / pallets carrying RFID tags pass underneath. like Figure 3 and Figure 4 As shown, the tracking process is as follows when goods are being moved through the gantry: Triggering: The photoelectric sensor detects an object entering, triggering the RFID reader to scan continuously, while the camera captures images at 30fps. RFID Processing: The edge computing box obtains the tag ID (such as the EPC code) and uses the RSSI (Received Signal Strength Indicator) value combined with the antenna position to estimate the approximate coordinates of the tag in the horizontal plane (denoted as region A) using triangulation.

[0022] The edge computing box runs YOLO or SSD object detection algorithms to identify cargo frames in the image and read text / QR code information from the cargo packaging (denoted as visual feature B). The system establishes a coordinate system, mapping region A to the image coordinate system. It detects whether the center of the object with visual feature B falls within region A. Cross-validation: If RFID reads ID001 and visual recognition identifies the text "Apple" within area A, and the database confirms that the corresponding goods for ID001 are indeed Apple, then tracking is considered successful. If RFID reads ID001, but visual recognition finds the goods to be damaged in that area, or identifies the text "Banana," then an anomaly is identified (possibly due to incorrect labeling or mixed goods), and the system immediately sends an alert to the central server.

[0023] Dynamic inventory management integrates a handheld RFID reader and a top-view camera onto a mobile inventory robot. As the robot moves between shelves, the RFID reader quickly scans the product tags on the shelves in batches. Simultaneously, the camera scans each shelf layer. If an RFID reader misses a reading (e.g., due to signal obstruction), the system visually identifies the quantity and appearance of the goods on that layer to determine if any are missing. If the visual identification is unclear, the presence of the goods is confirmed by the RFID reader. This combination achieves an inventory accuracy rate of over 99.9%.

[0024] In this embodiment, the structural features, working principle, and specific circuit structure of the above-mentioned components are all based on existing technology and will not be described in detail here.

[0025] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A smart warehouse cargo tracking system based on the fusion of RFID and visual recognition, characterized in that, include: RFID sensing unit: consists of an array of RFID readers deployed in a predetermined area of ​​the warehouse, and electronic tags attached to goods or vehicles, wherein the electronic tags store the unique identification of the goods; Visual sensing unit: includes an industrial camera and supplementary lighting equipment that are spatially synchronized with the RFID reader array, used to acquire images or video streams of goods; The edge computing unit is connected to the RFID reader array and the visual acquisition module respectively, and is used to receive radio frequency signals and image data, and perform local data preprocessing and feature fusion. The central management unit is communicatively connected to the edge computing terminal and is used to store cargo lifecycle data and issue scheduling instructions based on the fusion results.

2. The system according to claim 1, characterized in that, The edge computing unit deploys a multimodal data fusion algorithm module, which is configured as follows: Spatial mapping submodule: Based on preset calibration parameters, the RFID signal positioning area is mapped to the image coordinate system through AOA or RSSI differential calculation; Binding and verification submodule: Calculate the overlap rate between the visual target detection box and the RFID positioning area. If the overlap rate is greater than the threshold and the cargo category information is consistent, then bind the cargo ID and visual features. Dynamic weighting submodule: Adjusts the confidence weights of RFID and visual recognition in real time based on environmental parameters.

3. The system according to claim 1, characterized in that, The RFID reader array and visual acquisition unit are integrated on the mobile terminal, and perform dynamic inspections as the mobile device moves.

4. The system according to claim 1, characterized in that, The visual acquisition unit uses a depth camera to acquire the depth information of the cargo, which is used to calculate the cargo's volume and its precise coordinates in three-dimensional space.

5. A smart warehouse cargo tracking method based on the fusion of RFID and visual recognition, implemented based on the system described in any one of claims 1-4, characterized in that, Includes the following steps: S1: Synchronous data acquisition. When goods enter the monitoring area, the sensor is triggered to synchronize with the RFID reader and industrial camera to capture radio frequency signals and image data; S2: Data preprocessing. Noise reduction and filtering of RFID signals to identify valid tag IDs; target detection and segmentation of image data to extract cargo appearance features; S3: Coarse positioning and matching. Utilizing the RFID signal's angle of arrival (AOA) or signal strength. (RSSI) Differential calculation calculates the approximate region of the label and maps that region to a region of interest (ROI) in the image; S4: Fusion Verification. Within the ROI area, compare whether the visually identified goods category / code matches the goods category information stored in the RFID tag; S5: Status Update and Exception Handling. If the verification is successful, update the system inventory database and goods location information; If the verification is inconsistent or only a single modality of data is detected, it will be marked as an abnormal state and an alarm will be triggered.

6. The method according to claim 5, characterized in that, In step S4, a confidence-based weighted decision-making mechanism is also included: assigning different initial weights to visual recognition and RFID recognition, increasing the weight of visual recognition when there is sufficient light and the goods are unobstructed, and increasing the weight of RFID recognition when the goods are densely stacked or the light is insufficient.

Images