A warehouse intelligent control system and operation method based on edge visual sensing

Abstract

The application discloses a kind of based on edge visual induction's warehouse intelligent control system, comprising: perception module, edge computing module, message bus module, WMS service module and interactive module, its operation method includes the following steps: system mirroring deployment;Carry out the configuration mode setting and interactive calibration of warehousing operation and delivery operation;Warehousing operation;Delivery operation;Abnormal correction and sample archiving;Automatic reset.The application adopts edge visual and image difference differential non-inductive trigger, replaces physical sensor and artificial code scanning input, substantially reduces non-production operation, significantly improves the smoothness and efficiency of warehouse operation, warehousing and delivery double-mode self-adaptation, through heterogeneous vision parallel identification and real-time message push, realizes system active drive logistics, sub-second response and real-time interception of wrong picking, carries double-link correction and sample self-evolution closed loop, data can be traced throughout, supports remote visual operation and maintenance, and identification accuracy continues to improve with use.

Description

Technical Field

[0002] This invention relates to the field of computer vision technology, and in particular to a warehouse intelligent control system and operation method based on edge vision sensing. Background Technology

[0004] Intelligent warehouse control is an intelligent management and control system that relies on technologies such as the Internet of Things, artificial intelligence, big data, and automated equipment to realize the perception, decision-making, execution, and feedback of the entire warehousing process. It aims to solve the pain points of traditional warehousing models such as low efficiency, high cost, data lag, and poor coordination, and adapt to the needs of modern supply chains for high-frequency, precise, and flexible operations.

[0005] Traditional warehouse management systems (WMS) primarily rely on an interaction model of "PDA / terminal + barcode scanner + manual data entry" in core operational processes such as inbound and outbound picking. This model suffers from severe performance bottlenecks in high-frequency industrial scenarios, specifically manifested in the following three dimensions of inconsistency:

[0006] 1. The lack of coordination between physical actions and system feedback in the "high-frequency switching flow" results in a cycle of "putting down materials - picking up equipment - aligning with the barcode - putting down equipment - typing on the keyboard - confirming the document", which makes non-productive actions account for more than 60% in a single material flow.

[0007] 2. The inconsistency between the "strong coupling and serialization" of the form interaction process: The form operation has a strict logical sequence. Once a scanning step fails (such as a damaged barcode), the entire workflow is immediately interrupted. The operator must manually delete the erroneous data and re-enter it, which results in the throughput of the operation being limited by the interaction level of the software page.

[0008] 3. The inconsistency between the recorded results and the actual operation process, where the system's perception of material flow lags behind physical reality, leads to the data's authenticity relying excessively on the operator's sense of responsibility and physical condition.

[0009] A warehousing and logistics management method and system disclosed in Chinese patent document CN114169838B includes the following steps: When the WMS system receives an inbound plan, it scans the barcodes of the products in the inbound plan and compares the results with the corresponding list information. If the comparison is successful, it allocates storage locations and generates an inbound task. The WCS system breaks down the inbound task and sends inbound scheduling instructions to the AGVs. The WCS system feeds back the message to the WMS system, causing the WMS system to update the inventory. When the WMS system receives an outbound plan, it allocates storage locations and generates an outbound task. The WCS system breaks down the outbound task and sends outbound scheduling instructions to the AGVs. When products are transported to a designated area, the WMS system scans the barcodes of the products transported to the designated area and compares the results with the corresponding list information. If the comparison is successful, the WMS system updates the inventory. This invention can improve the storage efficiency of warehousing and logistics management, as well as the degree of automation and intelligence. However, the warehousing and logistics management method and system require manual scanning and verification of incoming and outgoing goods. Operators need to repeatedly switch between handheld devices, screens, and materials, which is cumbersome.

[0010] A method and system for intelligent warehouse management based on the Internet of Things (IoT) and artificial intelligence (AI) is disclosed in Chinese patent document CN120106754A. This method and system includes: 1) intelligent inventory management, which utilizes cargo box tags and image recognition technology to automatically identify and track each item entering and leaving the warehouse; and uses AI technology combined with big data analysis to predict future inventory demand and automatically adjust inventory levels to reduce inventory costs and stockout risks; 2) automated operation processes, which use IoT technology to monitor equipment status in real time to ensure efficient equipment operation and reduce malfunctions. Downtime; 3:) Real-time data analysis and decision support: collect various data generated during the warehousing process, including inventory data, order data, and equipment operation data; use artificial intelligence technology to perform deep learning and analysis on warehousing data to achieve intelligent decision-making such as inventory forecasting and route planning, and optimize warehousing management and logistics distribution processes; 4) Cross-system collaboration and data sharing: interconnect information from multiple systems such as warehousing, transportation, and sales to form end-to-end supply chain collaboration. However, this intelligent warehousing management method and system based on the Internet of Things and artificial intelligence requires real-time monitoring or periodic sampling, and cannot achieve event triggering and on-demand calculation, resulting in greater consumption.

[0011] To address the shortcomings of the existing technologies, providing a warehouse intelligent control system and operation method based on edge vision sensing is a worthwhile research topic. Summary of the Invention

[0013] The purpose of this invention is to overcome the shortcomings of low interaction efficiency and complex hardware deployment, and to provide a warehouse intelligent control system and operation method based on edge vision sensing, which achieves the technical effect of non-sensory operation and improved efficiency.

[0014] The objective of this invention is achieved through the following technical solution:

[0015] A warehouse intelligent control system and operation method based on edge vision sensing includes: a sensing module, an edge computing module, a message bus module, a WMS service module, and an interaction module;

[0016] The sensing module includes a material camera, a barcode camera, a visual light source, and a light source controller. The sensing module is used to acquire barcode images of materials and containers.

[0017] The edge computing module uses NVIDIA Jetson Orin Nano Super and is equipped with a YOLO recognition engine, ZBar barcode engine, image differential state machine component and Redis asynchronous communication component.

[0018] The message bus module adopts Redis Stream+Pub / Sub and is used for decoupled asynchronous message passing between the edge end and the server.

[0019] The WMS service module includes a business logic server, a MySQL database, and an SSE front-end push module. The WMS service module is used for automatic document wake-up and automatic form backfilling, and is deployed in a containerized manner.

[0020] The interaction module is an industrial touch display terminal. It provides an interface for ROI calibration, manual intervention, and status display, and builds a hardware and software integrated edge vision warehouse intelligent control architecture to realize material and barcode image acquisition, edge AI inference, asynchronous message transmission, WMS automatic linkage, and visual interaction, replacing the traditional barcode scanning + manual data entry mode.

[0021] Optionally, the intelligent warehouse control system adopts a mirrored deployment strategy to achieve rapid delivery and high consistency of workstations. Pre-installed edge system images are directly imported into Jetson Orin Nano Super. These images integrate the YOLOAI recognition engine, ZBar barcode detection plugin, image differential state machine module, and Redis asynchronous communication components. The system then connects with the upper-level warehouse management system (WMS) on the backend server of the workstation, directly pulling the system image. This image includes a core business logic server, a MySQL relational database, and a Redis message bus.

[0022] By using mounting technology to map server disks to the edge, second-level local storage of identification samples is achieved, ensuring that the edge host and WMS server are on the same local area network. This enables rapid deployment of workstations, high environmental consistency, second-level local storage of samples, reduced network transmission pressure, and guaranteed system stability and deployment efficiency.

[0023] A warehouse intelligent control operation method based on edge vision sensing includes the following steps:

[0024] Step 1: System image deployment. Import the system image pre-installed with the AI ​​engine and communication components into the edge device, and containerize and deploy WMS, database and message bus on the server side.

[0025] Step 2: Configure and interactively calibrate the inbound and outbound operations;

[0026] Step 3: Warehouse entry operation adopts the logic of "synchronous visual data collection and real-time correlation between accounts and physical inventory". First, synchronous sensing and data entry are performed, then physical entry is confirmed, and finally, anomaly correction is performed.

[0027] Step 4: Outbound operation adopts the asynchronous dual-trigger logic of "box to pop-up window, material through verification". First, the container is identified and the document is activated, then the material is picked and visually verified, and finally the task is completed.

[0028] Step 5: Anomaly correction and sample archiving, triggering manual intervention or post-event reversal and correction, saving difficult samples for model self-evolution;

[0029] Step Six: Automatic Reset. After the materials are removed, the ROI is restored to the baseline. The system resets within seconds and waits for the next operation, forming a standardized closed-loop operation process. It is compatible with both inbound and outbound scenarios, and is fully automated with manual backup to achieve consistency between accounts and physical inventory and continuous operation.

[0030] Optionally, when setting the warehousing operation configuration mode in step two, enter the system configuration page, set the mode parameter to inbound, save and restart the edge system to make it effective, observe the real-time preview window of the material camera on the monitor, and accurately align the green ROI frame with the center area of ​​the detection station through interface interaction. Align the green ROI frame of the material camera with the center of the detection station to establish the differential trigger benchmark and achieve material ROI calibration. In the warehousing mode, there is no green ROI frame in the barcode preview window. The operator only needs to ensure that the container barcode is completely displayed in the real-time preview window of the camera to complete the barcode positioning.

[0031] When setting the outbound operation configuration mode, enter the system configuration page, set the mode parameter to outbound, save and restart the edge system to make it effective, observe the real-time preview window of the material camera on the monitor, and accurately align the green ROI box with the center area of ​​the detection station through interface interaction to achieve material ROI calibration. Align the green ROI area of ​​the barcode camera with the location where the barcode of the material box usually appears to complete the barcode positioning. The barcode does not need to be completely placed within the green ROI box, but it must be fully presented in the entire preview screen of the camera to ensure the global retrieval capability of the barcode recognition engine.

[0032] Optionally, the synchronous sensing and input process in step three includes: the operator first places the material on the detection table, and the system simultaneously drives the material and the barcode camera so that the barcode and material recognition results are transmitted to the WMS warehouse entry page in real time. Then, the warehouse entry page renders the detailed specification information and detection quantity of the material in real time under the corresponding container barcode.

[0033] The physical warehousing confirmation process includes: the operator removes the inspected materials and puts them into a container. After all materials are placed, the operator clicks the WMS "warehousing" button, the container is moved into the warehouse, and the system returns to its initial state to prepare for the next order. If barcode recognition fails, materials are missed, or quantity errors occur during the warehousing process, the manual intervention area at the edge is used to correct the errors in real time to ensure that the "accounts and actual items match". There is no barcode scanning or manual data entry during the entire warehousing process. Data is bound to the container in real time, and anomalies are corrected immediately to ensure that the warehousing data is accurate and efficient.

[0034] Optionally, the container identification and document wake-up process in step four includes: automatic triggering: when the container moves to the designated location, the barcode camera detects ROI occlusion, the state machine enters the ready state, and after the preset trigger delay time expires, the system automatically interprets the container barcode. After successful barcode recognition, it is transmitted to WMS in real time, and the corresponding picking list pops up on the page. The upper part of the picking list is a folded table (recording all materials in the container), which can be expanded by clicking; the lower part displays the task details to be picked. If recognition fails, the edge intervention area is activated, and the process continues after manual barcode input. WMS still synchronizes documents in real time.

[0035] The picking and visual verification process includes: the operator places the material on the inspection table according to the document. After the trigger delay ends, the system automatically identifies the material category and quantity. The result is displayed in real time on the right side of the edge and synchronized to WMS. The "actual picking quantity" box of the corresponding material in the WMS picking list automatically accumulates the quantity. When the inspection fails, the manual intervention area is activated, and the operator manually fills in the category and quantity to correct the data.

[0036] After all materials have been picked, the operator clicks the "Picking Complete" button at the bottom of the WMS. The bin is moved away, the system automatically resets and waits for the next container, achieving zero-action alignment of people, bins, and orders. The picking process is automatically checked, reducing operation steps and improving outbound efficiency and accuracy.

[0037] Optionally, the intelligent warehouse control operation method adopts a non-intrusive trigger recognition method;

[0038] The non-intrusive trigger recognition method includes the following steps;

[0039] S1. Interactive visual calibration: After the system starts, the dynamic ROI box is overlaid and displayed in real time on the edge interface. Users can fine-tune the ROI coordinates by adjusting the camera angle or interacting with the interface to align the detection area with the work center. After the user confirms the alignment, the system acquires an empty scene image as a background template frame to establish the system in zero state.

[0040] S2. State machine determination and adaptive obstacle avoidance delay: The system continuously compares the pixel features of the current video frame ROI region with the background template frame. When the difference value exceeds the preset threshold, it enters the pre-trigger state; it starts a configurable delay countdown with a delay range of 0.5s to 3s, waiting for the operator's hands to leave the detection area.

[0041] S3. Parallel recognition and non-blocking reporting: After the delay ends, the system drives the material camera and barcode camera to continuously acquire multiple frames of images at high speed. It simultaneously calls the YOLO model for material classification and counting and the ZBar engine for barcode recognition. The recognition results are forcibly associated with barcode information to ensure that the material recognition results are bound to the corresponding container identity and prevent data silos. An independent background thread writes the detection results and image samples to the database and publishes messages through the Redis channel to achieve asynchronous non-blocking reporting. This process does not occupy camera sampling and AI inference resources, ensuring that the front end can still maintain high-frequency perception while the detection station processes data in the background.

[0042] S4. Locking and Automatic Reset: After identification, if the differential value of the ROI area does not recover to the baseline, the system enters the result locking state to prevent repeated triggering. When the material is removed and the ROI area returns to the background template state, the system automatically resets to the waiting-to-trigger state. In industrial warehousing environments, traditional triggering methods rely on physical sensors (such as infrared photoelectric switches) or manual clicking of interface buttons. This invention uses real-time image processing technology at the edge computing end (Jetson) to achieve automatic triggering using the differential state machine of the region of interest (ROI). This technology not only saves additional hardware costs, but also achieves "seamless" connection of the workflow through software-level state logic design.

[0043] Optionally, the intelligent warehouse control operation method adopts a heterogeneous concurrent identification architecture;

[0044] The heterogeneous concurrency identification architecture method includes the following steps:

[0045] S1. Based on WMS business instructions or initial configuration, the system is divided into two independent visual recognition pipelines: inbound mode and outbound mode.

[0046] S2. The warehousing mode adopts static container drive and only monitors the ROI area of ​​the material detection station. When the ROI of the material detection station is blocked, the material camera is driven to collect N frames of images and the barcode camera is driven to collect M frames of images to complete the synchronous recognition of materials and barcodes, reduce the idle power consumption of the barcode camera, and only bind the identity when the material passes the inspection.

[0047] S3, the outbound mode adopts dynamic container drive, and performs dual ROI collaborative calibration on the material station and barcode area. When the material box arrives at the barcode area and the barcode camera ROI is blocked, the container ID is first identified and pushed to the Redis channel, so that WMS automatically pops up the corresponding picking list; when the material is placed on the detection station and triggers the material ROI, the container barcode and material data are collected again to form a complete evidence chain for push.

[0048] S4. Robust processing is performed on heterogeneous data collected from multiple frames. On the material side, a multi-frame median algorithm is used to determine the final quantity, and on the barcode side, a multi-frame merging and deduplication algorithm is used to eliminate recognition jitter. The detection stability is determined by variance monitoring.

[0049] Material side (visual median suppression): Using the median of the N-frame detection results as the final quantity determination, this algorithm can effectively eliminate the missed or over-detection errors caused by the flickering of light and shadow in a single frame or the instantaneous overlap of materials.

[0050] Barcode side (multi-frame merging and deduplication): Perform "set union operation" on the barcode results of M frames to eliminate inter-frame recognition jitter caused by barcode reflection while preserving the ability of multiple barcodes to coexist.

[0051] Stability assessment (variance monitoring): The system calculates the mean and variance of multi-frame detection results in real time. If the variance exceeds the safety threshold, it is judged as "unstable detection" and automatically marked as a difficult sample for reporting. However, the detection values ​​are not forcibly changed to ensure the traceability of the data.

[0052] S5. The recognition results are transmitted to the WMS backend for business verification. This invention designs a heterogeneous concurrent recognition architecture with an adaptive business mode for complex material flow logic in industrial scenarios. By decoupling the perception logic of the material camera and the barcode camera, the system can automatically switch between two modes: "inbound (single-end driven)" and "outbound (dual-end linkage)". It uses multi-frame filtering algorithm and asynchronous reporting mechanism to solve the problems of recognition accuracy and response latency in complex dynamic environments.

[0053] Optionally, the intelligent warehouse control operation method adopts a real-time collaborative architecture consisting of a distributed asynchronous message bus and server-side event push;

[0054] The real-time collaborative architecture method includes the following steps:

[0055] S1. Edge-end anomaly pre-judgment and manual intervention: After the edge computing device acquires the visual inspection results, it performs anomaly pre-judgment in the GUI main thread. It assesses whether manual intervention is needed based on confidence level, missed detection status and result stability. It enforces barcode binding constraints on the detection results. If a barcode is missing, it forces a pop-up window to fill in the missing barcode to ensure the determinism of data services.

[0056] S2. Data decoupling persistence and task snapshot encapsulation: Task snapshot generation: The system encapsulates material attributes (category, quantity), container identity (barcode), time consumption statistics and manual correction marks into standardized business snapshots.

[0057] To address the issues of data loss and backlog in high-concurrency web scenarios, this invention employs a combined Redis Stream + Pub / Sub message bus delivery model, utilizing S3 and Redis.

[0058] S4 and SSE proactive push and automatic form-driven: The WMS backend listens for Redis notifications, retrieves the complete snapshot from the Stream based on the message_id, and proactively pushes it to the front-end page through the SSE (Server-Sent Events) long connection.

[0059] Optionally, the intelligent warehouse control operation method adopts a dual-link correction mechanism;

[0060] The dual-link correction mechanism includes the following steps:

[0061] S1. Construct a dual-link data correction system involving the perception layer and the business layer:

[0062] Link A: Real-time evaluation of detection quality at the edge. When there are missed detections, low confidence, or insufficient stability of multi-frame results, they are judged as difficult samples, the pipeline is blocked, and a pop-up window is forced for manual intervention. The manual real value is used to cover the model prediction result, thus completing the source correction.

[0063] Link B: Perform one-click cancellation on the reported false detection data, retrieve the original snapshot for correction, send an overwrite message to WMS via Redis to reset the business status, and retain comparison records for negative sample analysis;

[0064] S2, Automated sample collection and asset management;

[0065] S3, edge-end operational status and model quality monitoring, through a closed-loop logic of "failure perception - manual correction - sample collection - training evolution", the system has the characteristic of continuously improving accuracy with the increase of usage time. It transforms the black-box operational status of the edge end into transparent digital reports, enabling maintenance personnel to remotely diagnose model performance and hardware load without on-site intervention. Complete correction traces are retained, ensuring that every piece of inbound / outbound data has original images and manual correction records available when correction occurs, meeting the stringent data auditing requirements of industrial production.

[0066] Positive and beneficial effects: 1. This intelligent warehouse control system based on edge vision sensing adopts edge vision and image differential non-sensory triggering to replace physical sensors and manual barcode scanning and data entry, which greatly reduces non-production operations and significantly improves the smoothness and efficiency of warehouse operations.

[0067] 2. This intelligent warehouse control operation method based on edge vision sensing features dual-mode adaptive inbound and outbound operations. Through heterogeneous visual parallel recognition and real-time message push, the system proactively drives logistics, provides sub-second response, and intercepts mispicked items in real time.

[0068] 3. This intelligent warehouse control operation method based on edge vision sensing is equipped with dual-link correction and sample self-evolution closed loop, data can be traced throughout the process, supports remote visual operation and maintenance, and the recognition accuracy continues to improve with use. Attached Figure Description

[0070] Figure 1 This is a system architecture diagram of the present invention;

[0071] Figure 2 This is a flowchart of the interactive ROI calibration and triggering method of the present invention;

[0072] Figure 3 This is a timing diagram of the dual ROI linkage under the outbound mode of the present invention;

[0073] Figure 4 This is a logic diagram of heterogeneous data multidimensional aggregation and smoothing processing in this invention;

[0074] Figure 5 This is a diagram of the dual-link correction and model self-evolution closed-loop architecture of the present invention. Detailed Implementation

[0076] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0077] Example 1

[0078] like Figures 1 to 5 As shown, a warehouse intelligent control system and operation method based on edge vision sensing includes: a sensing module, an edge computing module, a message bus module, a WMS service module, and an interaction module;

[0079] The sensing module includes a material camera, a barcode camera, a visual light source, and a light source controller. The sensing module is used to acquire barcode images of materials and containers.

[0080] The edge computing module uses NVIDIA Jetson Orin Nano Super and is equipped with a YOLO recognition engine, ZBar barcode engine, image differential state machine component and Redis asynchronous communication component;

[0081] The message bus module uses Redis Stream + Pub / Sub and is used for decoupling asynchronous message passing between the edge and the server.

[0082] The WMS service module includes a business logic server, a MySQL database, and an SSE front-end push module. The WMS service module is used for automatic document wake-up and automatic form filling, and is deployed in a containerized manner.

[0083] The interaction module is an industrial touch display terminal. It provides an interface for ROI calibration, manual intervention and status display, and builds a hardware and software integrated edge vision warehouse intelligent control architecture to realize material and barcode image acquisition, edge AI inference, asynchronous message transmission, WMS automatic linkage and visual interaction, replacing the traditional barcode scanning + manual data entry mode.

[0084] Furthermore, the sensing module uses dual independent cameras to collect data in parallel. The material camera is responsible for identifying the material type and quantity, while the barcode camera is responsible for identifying the container. The two data streams do not interfere with each other and are processed synchronously, improving the parallelism and accuracy of the identification.

[0085] The intelligent warehouse control system adopts a mirrored deployment strategy to achieve rapid delivery and high consistency at workstations. Pre-installed edge system images are directly imported into Jetson Orin Nano Super. These images integrate the YOLO AI recognition engine, ZBar barcode detection plugin, image differential state machine module, and Redis asynchronous communication components. The system then connects with the upper-level warehouse management system (WMS) on the backend server of the workstation, enabling data exchange and business collaboration. The system image, which includes a core business logic server, a MySQL relational database, and a Redis message bus, is directly pulled from the backend server of the workstation.

[0086] By using mounting technology to map the server disk to the edge, the system achieves second-level local storage of identification samples, ensuring that the edge host and the WMS server are on the same local area network. This enables rapid deployment of workstations, a high degree of environmental consistency, and second-level local storage of samples, reducing network transmission pressure and ensuring system stability and deployment efficiency.

[0087] like Figures 1 to 5 As shown, a warehouse intelligent control operation method based on edge vision sensing includes the following steps:

[0088] Step 1: System image deployment. Import the system image pre-installed with the AI ​​engine and communication components into the edge device, and containerize and deploy WMS, database and message bus on the server side.

[0089] Step 2: Configure and interactively calibrate the inbound and outbound operations;

[0090] Step 3: Warehouse entry operation adopts the logic of "synchronous visual data collection and real-time correlation between accounts and physical inventory". First, synchronous sensing and data entry are performed, then physical entry is confirmed, and finally, anomaly correction is performed.

[0091] Step 4: Outbound operation adopts the asynchronous dual-trigger logic of "box to pop-up window, material through verification". First, the container is identified and the document is activated, then the material is picked and visually verified, and finally the task is completed.

[0092] Step 5: Anomaly correction and sample archiving, triggering manual intervention or post-event reversal and correction, saving difficult samples for model self-evolution;

[0093] Step Six: Automatic Reset. After the materials are removed, the ROI is restored to the baseline. The system resets within seconds and waits for the next operation, forming a standardized closed-loop operation process. It is compatible with both inbound and outbound scenarios, and is fully automated with manual backup to achieve consistency between accounts and physical inventory and continuous operation.

[0094] Example 2

[0095] When setting the inbound operation configuration mode in step two, go to the system configuration page, set the mode parameter to inbound, save and restart the edge system to make it effective, observe the real-time preview window of the material camera on the monitor, and accurately align the green ROI box with the center area of ​​the detection station through interface interaction. Align the green ROI box of the material camera with the center of the detection station to establish the differential trigger benchmark and achieve material ROI calibration. In the inbound mode, there is no green ROI box in the barcode preview window. The operator only needs to ensure that the container barcode is fully displayed in the real-time preview window of the camera to complete the barcode positioning.

[0096] When configuring the outbound operation mode, enter the system configuration page, set the mode parameter to outbound, save and restart the edge system to make it effective, observe the real-time preview window of the material camera on the monitor, and accurately align the green ROI box with the center area of ​​the detection station through interface interaction to achieve material ROI calibration. Align the green ROI area of ​​the barcode camera with the location where the barcode of the material box usually appears to complete the barcode positioning; the barcode does not need to be completely placed within the green ROI box, but it must be fully presented in the entire preview screen of the camera to ensure the global retrieval capability of the barcode recognition engine;

[0097] After the benchmark confirms the completion of ROI physical alignment, click "Confirm Calibration". The system will capture the current empty field image as the background template frame and officially enter the "Ready" state.

[0098] In the inbound mode, the inbound parameter is set to calibrate only the ROI of the material and not the ROI of the barcode; in the outbound mode, the outbound parameter is set to calibrate both ROIs. After calibration, an empty field image is captured as the background template frame to adapt to different business logics of static triggering in inbound and dual-end linkage in outbound. The ROI is accurately located, and the background template provides a benchmark for differential triggering.

[0099] The synchronous sensing and data entry process in step three includes: the operator first places the material on the detection table, and the system simultaneously drives the material and barcode camera so that the barcode and material recognition results are transmitted to the WMS warehouse entry page in real time. Then, the warehouse entry page renders the detailed specification information and detection quantity of the material in real time under the corresponding container barcode.

[0100] The physical warehousing confirmation process includes: the operator removes the inspected materials and puts them into a container. After all materials are placed, the operator clicks the WMS "Inbound" button, the container is moved into the warehouse, and the system returns to its initial state to prepare for the next order. If barcode recognition fails, materials are missed, or quantity errors occur during the warehousing process, the manual intervention area at the edge is used to correct the errors in real time to ensure that the records match the actual inventory. The entire warehousing process is done without scanning or manual data entry. Data is bound to the container in real time, and anomalies are corrected immediately to ensure that the warehousing data is accurate and efficient.

[0101] Step four, the container identification and document wake-up process, includes: automatic triggering, the container moves to the designated location, the barcode camera detects ROI occlusion, the state machine enters the ready state, and after the preset trigger delay time expires, the system automatically interprets the container barcode. After successful barcode recognition, it is transmitted to WMS in real time, and the corresponding picking list pops up on the page. The upper part of the picking list is a collapsible table (recording all materials in the container), which can be expanded by clicking; the lower part displays the task details to be picked. If recognition fails, the edge intervention area is activated, and the process continues after manual barcode input. WMS still synchronizes documents in real time.

[0102] The material picking and visual verification process includes: the operator places the material on the inspection station according to the document. After the trigger delay ends, the system automatically identifies the material category and quantity. The result is displayed in real time on the right side of the edge and synchronized to WMS. The "actual picking quantity" box of the corresponding material in the WMS picking list automatically accumulates the quantity. When the inspection fails, the manual intervention area is activated, and the operator manually enters the category and quantity to correct the data.

[0103] After all materials have been picked, the operator clicks the "Picking Complete" button at the bottom of the WMS. The bin is moved away, the system automatically resets and waits for the next container, achieving zero-action alignment of people, bins, and orders. The picking process is automatically checked, reducing operation steps and improving outbound efficiency and accuracy.

[0104] Example 3

[0105] like Figure 2 As shown, the intelligent warehouse control operation method adopts a contactless trigger recognition method;

[0106] The non-intrusive trigger recognition method includes the following steps;

[0107] S1. Interactive visual calibration: After the system starts, the dynamic ROI box is overlaid and displayed in real time on the edge interface. Users can fine-tune the ROI coordinates by adjusting the camera angle or interacting with the interface to align the detection area with the work center. After the user confirms the alignment, the system acquires an empty scene image as a background template frame to establish the system in zero state.

[0108] S2. State machine determination and adaptive obstacle avoidance delay: The system continuously compares the pixel features of the current video frame ROI region with the background template frame. When the difference value exceeds the preset threshold, it enters the pre-trigger state; it starts a configurable delay countdown with a delay range of 0.5s to 3s, waiting for the operator's hands to leave the detection area.

[0109] S3. Parallel recognition and non-blocking reporting: After the delay ends, the system drives the material camera and barcode camera to continuously acquire multiple frames of images at high speed. It simultaneously calls the YOLO model for material classification and counting and the ZBar engine for barcode recognition. The recognition results are forcibly associated with barcode information to ensure that the material recognition results are bound to the corresponding container identity and prevent data silos. An independent background thread writes the detection results and image samples to the database and publishes messages through the Redis channel to achieve asynchronous non-blocking reporting. This process does not occupy camera sampling and AI inference resources, ensuring that the front end can still maintain high-frequency perception while the detection station processes data in the background.

[0110] S4. Locking and Automatic Reset: After identification, if the ROI area difference value does not recover to the baseline, the system enters the result locking state to prevent repeated triggering. When the material is removed and the ROI area returns to the background template state, the system automatically resets to the waiting-to-trigger state. Through the above logic, the system constructs a "placement triggers, removal resets" assembly line work mode. Operators are completely freed from the mouse, keyboard, and barcode scanner. The only operation they need to perform is the physical action of "picking up material - placing down for inspection - taking away for packing". The complex form operations of the WMS system are compressed to two ends - "start work" and "final one-click completion (warehousing / picking)". All data flow in the intermediate process is automatically driven by visual sensing, which greatly reduces the cognitive load and operation time in industrial scenarios.

[0111] In industrial warehousing environments, traditional triggering methods rely on physical sensors (such as infrared photoelectric switches) or manual clicking of interface buttons. This invention uses real-time image processing technology at the edge computing end (Jetson) to achieve automatic triggering using the differential state machine of the region of interest (ROI). This technology not only saves additional hardware costs, but also achieves "seamless" connection of the work process through software-level state logic design.

[0112] Example 4

[0113] like Figure 3 As shown, the intelligent warehouse control operation method adopts a heterogeneous concurrent recognition architecture;

[0114] The heterogeneous concurrency identification architecture method includes the following steps:

[0115] S1. Based on WMS business instructions or initial configuration, the system is divided into two independent visual recognition pipelines: inbound mode and outbound mode.

[0116] S2. The warehousing mode adopts static container drive and only monitors the ROI area of ​​the material detection station. When the ROI of the material detection station is blocked, the material camera is driven to collect N frames of images and the barcode camera is driven to collect M frames of images to complete the synchronous recognition of materials and barcodes, reduce the idle power consumption of the barcode camera, and only bind the identity when the material passes the inspection.

[0117] S3, the outbound mode adopts dynamic container drive, and performs dual ROI collaborative calibration on the material station and barcode area. When the material box arrives at the barcode area and the barcode camera ROI is blocked, the container ID is first identified and pushed to the Redis channel, so that WMS automatically pops up the corresponding picking list; when the material is placed on the detection station and triggers the material ROI, the container barcode and material data are collected again to form a complete evidence chain for push.

[0118] S4. Robust processing is performed on heterogeneous data collected from multiple frames. On the material side, a multi-frame median algorithm is used to determine the final quantity, and on the barcode side, a multi-frame merging and deduplication algorithm is used to eliminate recognition jitter. The detection stability is determined by variance monitoring.

[0119] Material side (visual median suppression): Using the median of the N-frame detection results as the final quantity determination, this algorithm can effectively eliminate the missed or over-detection errors caused by the flickering of light and shadow in a single frame or the instantaneous overlap of materials.

[0120] Barcode side (multi-frame merging and deduplication): Perform "set union operation" on the barcode results of M frames to eliminate inter-frame recognition jitter caused by barcode reflection while preserving the ability of multiple barcodes to coexist.

[0121] Stability assessment (variance monitoring): The system calculates the mean and variance of multi-frame detection results in real time. If the variance exceeds the safety threshold, it is judged as "unstable detection" and automatically marked as a difficult sample for reporting. However, the detection values ​​are not forcibly changed to ensure the traceability of the data.

[0122] S5. Transmit the identification results to the WMS backend for business verification, automatic backfilling and verification: After the data is transmitted to the WMS through the specified channel, the backend performs "picking list matching and verification". If the materials are consistent, the "actual picking quantity" input box is automatically filled in using SSE.

[0123] Real-time interception of incorrect picking: If an operator picks up materials that are not required by the picking list, the WMS will immediately execute a visual feedback mechanism, such as flashing a red box on the page or popping up a window to intercept the operation and prevent illegal operations.

[0124] Assembly line reset: After picking is completed, the operator clicks to confirm, the system drives the conveyor line back to the warehouse, the barcode camera ROI is released from obstruction, the state machine automatically returns to the starting position, and it is ready to capture the next bin;

[0125] By using "barcode-triggered documents, visual automatic verification, and system automatic backfilling", the dozens of clicks required for data entry in traditional WMS are compressed into three actions: "look once, put down once, and click once". One set of hardware is compatible with two completely different business logics: inbound and outbound. The combination of ROI state machines enables the algorithm to have a deep understanding of the business.

[0126] This invention addresses the complex material flow logic in industrial scenarios by designing a heterogeneous concurrent recognition architecture with an adaptive business mode. By decoupling the perception logic of the material camera and the barcode camera, the system can automatically switch between two modes: "inbound (single-end driven)" and "outbound (dual-end linkage)". It utilizes multi-frame filtering algorithms and asynchronous reporting mechanisms to solve the problems of recognition accuracy and response latency in complex dynamic environments.

[0127] Example 5

[0128] like Figure 4 As shown, the intelligent warehouse control operation method adopts a real-time collaborative architecture consisting of a distributed asynchronous message bus and server-side event push.

[0129] The real-time collaborative architecture approach includes the following steps:

[0130] S1. Edge-end anomaly pre-judgment and manual intervention: After the edge computing device acquires the visual inspection results, it performs anomaly pre-judgment in the GUI main thread. It assesses whether manual intervention is needed based on confidence level, missed detection status and result stability. It enforces barcode binding constraints on the detection results. If a barcode is missing, it forces a pop-up window to fill in the missing barcode to ensure the determinism of data services.

[0131] S2. Data decoupling and persistence, along with task snapshot encapsulation, ensures the continuity of visual perception. The system employs a queue mechanism to deliver the recognition results to an independent background IO thread, achieving complete decoupling between UI rendering and data persistence.

[0132] Multimodal data is solidified, and the IO thread executes the actions of "saving the result image", "writing to the MySQL business database" and "archiving difficult samples" in sequence, establishing a unique correspondence between the task and the local physical image;

[0133] Task snapshot generation: The system encapsulates material attributes (category, quantity), container identity (barcode), time consumption statistics, and manual correction marks into a standardized business snapshot.

[0134] To address data loss and backlog issues in high-concurrency web scenarios, this invention employs a combined Redis Stream + Pub / Sub message bus delivery model, utilizing S3 and Redis.

[0135] Persistent storage (Stream) uses the XADD command to write business snapshots to the Stream, achieving ordered arrangement and traceable storage of messages;

[0136] Lightweight notifications (Pub / Sub) are only published in the subscribed channel, and the system only publishes lightweight notifications containing message_id. This design of "large body stored in stream, small ID sent notification" greatly reduces the listening load on the WMS side and avoids large packet loss caused by network fluctuations.

[0137] S4 and SSE proactive push and automatic form-driven: The WMS backend listens for Redis notifications, retrieves the complete snapshot from the Stream based on the message_id, and proactively pushes it to the front-end page through the SSE (Server-Sent Events) long connection;

[0138] The page automatically redirects and wakes up; the front end automatically retrieves and activates the corresponding inbound / outbound task order based on the barcode in the message, without the need for manual document search.

[0139] Active mapping at the form field level allows the recognition results (such as material SKU name and real-time detection quantity) to be directly injected into the corresponding input fields of the HTML form. The system transforms the original action of "manual keyboard typing" into automatic backfilling driven by "visual perception", enabling the WMS page to have the ability to actively perceive physical reality.

[0140] To address the inefficient interaction patterns in traditional WMS systems, such as frequent manual page switching, manual focus on input boxes, and repetitive data entry, this invention designs a real-time collaborative architecture based on edge sensing, a distributed asynchronous message bus (RedisStream), and server-side event push (SSE). The core innovation of this architecture lies in transforming unstructured visual data generated by physical sensing at the edge into logical driving instructions for the WMS business system through multi-level asynchronous links.

[0141] The complex interactive logic that originally involved "searching, locating, scanning, inputting, verifying, and confirming" is compressed into a single physical action through a visual link. By combining Redis and SSE, a sub-second response is achieved from "material placement" to "screen data backfilling," eliminating the latency caused by traditional HTTP polling. This technology liberates warehouse operators from the heavy role of "data entry worker" and transforms them into "workflow monitors," greatly reducing the cognitive load and operational error rate in high-frequency industrial operation scenarios.

[0142] Example 6

[0143] like Figure 5 As shown, the intelligent warehouse control operation method adopts a dual-link correction mechanism;

[0144] The dual-link correction mechanism includes the following steps:

[0145] S1. Construct a dual-link data correction system involving the perception layer and the business layer:

[0146] Link A: Real-time evaluation of detection quality at the edge. When there are missed detections, low confidence, or insufficient stability of multi-frame results, they are judged as difficult samples, the pipeline is blocked, and a pop-up window is forced for manual intervention. The manual real value is used to cover the model prediction result, thus completing the source correction.

[0147] Link B: Perform one-click cancellation on the reported false detection data, retrieve the original snapshot for correction, send an overwrite message to WMS via Redis to reset the business status, and retain comparison records for negative sample analysis;

[0148] S2. Automated Sample Collection and Asset Management:

[0149] The system transforms every manual correction action into valuable model training data, turning "waste into fuel":

[0150] Physical layer: Mounted fast disk writing, the edge end adopts the technology of directly mounting the server disk directory to the local machine, realizing the writing of high-resolution original image and result image in seconds, without the need to transfer large files over the network, ensuring system performance under high frequency operation;

[0151] Data layer: Multimodal association storage: Each difficult sample is indexed by the task ID, and the original image, the predicted bounding box result image, and the manually corrected JSON label are saved synchronously. This structured storage of "image + automatic label" enables WMS to export high-quality training datasets with one click, which greatly shortens the algorithm iteration cycle.

[0152] S3. Edge-end operational status and model quality monitoring:

[0153] The WMS platform integrates a "monitoring and reporting system" for edge devices, which aggregates the massive amounts of underlying data into the following key metrics.

[0154] Load monitoring: Real-time analysis of task count and total frame count to monitor whether edge devices are receiving orders stably and whether there are any load anomalies;

[0155] Performance monitoring (inference time): Statistically analyze the average inference time and the distribution of long-tailed slow frames to assess whether the edge computing power meets the requirements of real-time production;

[0156] Quality monitoring (automatic recognition rate): By using the "difficult sample cause distribution map" and "automatic pass / manual coverage ratio", the system can accurately locate the scenarios in which the model is most prone to errors on site (such as lighting interference or specific SKU occlusion), providing quantitative basis for hardware supplementary lighting optimization or algorithm strategy adjustment.

[0157] To address the recognition fluctuations of industrial vision systems caused by factors such as lighting, occlusion, and contamination in complex environments, this invention designs a self-evolving architecture integrating "real-time error correction," "post-event traceability," and "sample closed-loop training." This architecture ensures the consistency between inventory and physical inventory data through a dual-link error correction mechanism (real-time interception of difficult samples and post-event reversal of false detections), and utilizes automated sample archiving technology to achieve a data closed loop from "manual error correction" to "model evolution."

[0158] Through a closed-loop logic of "failure perception - manual correction - sample collection - training evolution", the system has the characteristic of continuously improving accuracy with the increase of usage time. It transforms the black-box operation status at the edge into transparent digital reports, enabling maintenance personnel to remotely diagnose model performance and hardware load without on-site intervention. Complete correction traces are retained, ensuring that every piece of inbound / outbound data has original images and manual correction records available when correction occurs, meeting the stringent data auditing requirements of industrial production.

[0159] The above is only used to illustrate the technical solution of the present invention and not to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A warehouse intelligent control system based on edge vision sensing, characterized in that, include: The module comprises a perception module, an edge computing module, a message bus module, a WMS service module, and an interaction module. The sensing module includes a material camera, a barcode camera, a visual light source, and a light source controller. The sensing module is used to acquire barcode images of materials and containers. The edge computing module is equipped with a YOLO recognition engine, a ZBar barcode engine, an image differential state machine component, and a Redis asynchronous communication component. The message bus module is used for decoupling asynchronous message transmission between the edge end and the server end; The WMS service module includes a business logic server, a MySQL database, and an SSE front-end push module. The WMS service module is used for automatic document wake-up and automatic form filling. The interaction module is an industrial touch display terminal, which provides an interface for ROI calibration, manual intervention, and status display.

2. The warehouse intelligent control system based on edge vision sensing according to claim 1, characterized in that: The intelligent warehouse control system adopts a mirrored deployment strategy. The pre-installed edge system image is directly imported into Jetson Orin NanoSuper. The image integrates the YOLO AI recognition engine, ZBar barcode detection plugin, image differential state machine module, and Redis asynchronous communication component. The system image is then directly pulled from the backend server of the workstation using containerization technology. This image includes the core business logic server, MySQL relational database, and Redis message bus. By using mounting technology to map the server disk to the edge, second-level local storage of identification samples is achieved, ensuring that the edge host and the WMS server are on the same local area network.

3. A warehouse intelligent control operation method based on edge vision sensing as described in any one of claims 1 to 2, characterized in that, Includes the following steps: Step 1: System image deployment. Import the system image pre-installed with the AI ​​engine and communication components into the edge device, and containerize and deploy WMS, database and message bus on the server side. Step 2: Configure and interactively calibrate the inbound and outbound operations; Step 3: Warehouse entry operation adopts the logic of "synchronous visual data collection and real-time correlation between accounts and physical inventory". First, synchronous sensing and data entry are performed, then physical entry is confirmed, and finally, anomaly correction is performed. Step 4: Outbound operation adopts the asynchronous dual-trigger logic of "box to pop-up window, material through verification". First, the container is identified and the document is activated, then the material is picked and visually verified, and finally the task is completed. Step 5: Anomaly correction and sample archiving, triggering manual intervention or post-event reversal and correction, saving difficult samples for model self-evolution; Step Six: Automatic Reset. After the material is removed, the ROI returns to the baseline, and the system resets within seconds to await the next operation.

4. The warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: When setting the configuration mode for the warehousing operation in step two, enter the system configuration page, set the mode parameter to inbound, save and restart the edge system to make it effective, realize material ROI calibration, and complete barcode positioning; When setting the configuration mode for the outbound operation, enter the system configuration page, set the mode parameter to outbound, save and restart the edge system to make it effective, realize material ROI calibration, and complete barcode positioning.

5. The warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The synchronous sensing and input process in step three includes: the operator first places the material on the detection table, and the system simultaneously drives the material and the barcode camera so that the barcode and material recognition results are transmitted to the WMS warehouse entry page in real time. Then, the warehouse entry page renders the detailed specification information and detection quantity of the material in real time under the corresponding container barcode. The physical warehousing confirmation process includes: the operator takes away the inspected materials and puts them into a container. After all the materials have been put in, the operator clicks the WMS "warehousing" button, the container is moved into the warehouse, and the system returns to its initial state to prepare for the next order.

6. The warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The container identification and document wake-up process in step four includes: automatic triggering, the container moves to the designated location, the barcode camera detects ROI occlusion, the state machine enters the ready state, and after the preset trigger delay time expires, the system automatically interprets the container barcode. After successful barcode recognition, it is transmitted to WMS in real time, and the corresponding picking list pops up on the page. The upper part of the picking list is a folded table that can be expanded by clicking; the lower part displays the task details to be picked. If recognition fails, the edge intervention area is activated, and the process continues after manual barcode input. WMS still synchronizes documents in real time. The picking and visual verification process includes: the operator places the material on the inspection table according to the document. After the trigger delay ends, the system automatically identifies the material category and quantity. The result is displayed in real time on the right side of the edge and synchronized to WMS. The "actual picking quantity" box of the corresponding material in the WMS picking list automatically accumulates the quantity. When the inspection fails, the manual intervention area is activated, and the operator manually fills in the category and quantity to correct the data. After all materials have been picked, the operator clicks the "Pick Complete" button below the WMS, the bin is moved away, the system automatically resets and waits for the next container.

7. The warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The intelligent warehouse control operation method adopts a non-contact trigger recognition method; The non-intrusive trigger recognition method includes the following steps; S1. Interactive visual calibration: After the system starts, the dynamic ROI box is displayed in real time on the edge interface. Users can fine-tune the ROI coordinates by adjusting the camera angle or interacting with the interface to align the detection area with the work center. After the user confirms the alignment, the system acquires an empty scene image as a background template frame to establish a system zero state. S2. State machine determination and adaptive obstacle avoidance delay: The system continuously compares the pixel features of the current video frame ROI region with the background template frame. When the difference value exceeds the preset threshold, it enters the pre-trigger state; it starts a configurable delay countdown with a delay range of 0.5s to 3s, waiting for the operator's hands to leave the detection area. S3. Parallel recognition and non-blocking reporting: After the delay ends, the system drives the material camera and barcode camera to continuously acquire multiple frames of images at high speed, and simultaneously calls the YOLO model for material classification and counting, and the ZBar engine for barcode recognition; the recognition results are forcibly associated with barcode information to ensure that the material recognition results are bound to the corresponding container identity; The detection results and image samples are written to the database by an independent background thread, and the message is published through the Redis channel to achieve asynchronous non-blocking reporting; S4. Locking and Automatic Reset: After identification is completed, if the difference value of the ROI area does not return to the baseline, the system enters the result locking state to prevent repeated triggering; when the material is removed and the ROI area returns to the background template state, the system automatically resets to the waiting trigger state.

8. The warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The intelligent warehouse control operation method adopts a heterogeneous concurrent identification architecture; The heterogeneous concurrency identification architecture includes the following steps: S1. Based on WMS business instructions or initial configuration, the system is divided into two independent visual recognition pipelines: inbound mode and outbound mode. S2. The warehousing mode adopts static container drive and only monitors the ROI area of ​​the material detection station. When the ROI of the material detection station is blocked, the material camera is driven to collect N frames of images and the barcode camera is driven to collect M frames of images to complete the synchronous recognition of materials and barcodes. S3, the outbound mode adopts dynamic container drive, and performs dual ROI collaborative calibration on the material station and barcode area. When the material box arrives at the barcode area and the barcode camera ROI is blocked, the container ID is first identified and pushed to the Redis channel, so that WMS automatically pops up the corresponding picking list; when the material is placed on the detection station and triggers the material ROI, the container barcode and material data are collected again to form a complete evidence chain for push. S4. Robust processing is performed on heterogeneous data collected from multiple frames. On the material side, a multi-frame median algorithm is used to determine the final quantity, and on the barcode side, a multi-frame merging and deduplication algorithm is used to eliminate recognition jitter. The detection stability is determined by variance monitoring. S5. Pass the recognition results to the WMS backend for business verification.

9. A warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The intelligent warehouse control operation method adopts a real-time collaborative architecture consisting of a distributed asynchronous message bus and server-side event push. The real-time collaborative architecture includes the following steps: S1. Edge-end anomaly pre-judgment and manual intervention: After the edge computing device acquires the visual inspection results, it performs anomaly pre-judgment in the GUI main thread. It assesses whether manual intervention is needed based on confidence level, missed detection status and result stability. It enforces barcode binding constraints on the detection results. If a barcode is missing, it forces a pop-up window to fill in the missing barcode to ensure the determinism of data services. S2, Data decoupling persistence and task snapshot encapsulation; S3 and Redis dual-layer message bus delivery; S4 and SSE proactive push and automatic form driving.

10. A warehouse intelligent control operation method based on edge vision sensing according to claim 3, characterized in that: The intelligent warehouse control operation method adopts a dual-link correction mechanism; The dual-link correction mechanism includes the following steps: S1. Construct a dual-link data correction system involving the perception layer and the business layer: Link A: Real-time evaluation of detection quality at the edge. When there are missed detections, low confidence, or insufficient stability of multi-frame results, they are judged as difficult samples, the pipeline is blocked, and a pop-up window is forced for manual intervention. The manual real value is used to cover the model prediction result, thus completing the source correction. Link B: Perform one-click cancellation on the reported false detection data, retrieve the original snapshot for correction, send an overwrite message to WMS via Redis to reset the business status, and retain comparison records for negative sample analysis; S2, Automated sample collection and asset management; S3, edge-end operation status and model quality monitoring.

Images